2024-08-20T00:47:02.606 INFO:root:teuthology version: 1.2.0 2024-08-20T00:47:02.632 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-20T00:47:02.731 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298 branch: wip-hemanth-testing-2024-08-19-1357 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7861298' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-08-19-1357 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug cephsqlite: 20 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy mgr-modules: - snap_schedule sha1: ef4275410d8483698016fe484b2a3c664080b4c1 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: ef4275410d8483698016fe484b2a3c664080b4c1 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-08-19-1357 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 owner: scheduled_hyelloji@teuthology priority: 10 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7244 sha1: ef4275410d8483698016fe484b2a3c664080b4c1 sleep_before_teardown: 0 suite: fs suite_branch: wip-hemanth-testing-2024-08-19-1357 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: ef4275410d8483698016fe484b2a3c664080b4c1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGtcIxzrXOKDKy/tQev3eRtiVzzw611JteP8ovZ4PguLbj26DphKV/fpH9cJ7p18F55olp7Bg9aKN0PsThKcaQ= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTwKN0d/LiU6YLOqptevKhKjCkjJBi/nmVx0s+/IJlT+/RiAv6/VkiEyRM/1m3h/O5HLbOU2zAic6g/v3RHeOY= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyffimhgJRJwueEoanC/bsPggJtKVVv5MlpklRrTrAKq0OaxzlD9IfSOH3ybVuBi6coszYXzPStmKUphWDQHdw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 67db276914d29505c9707d04f3a3b8fb9d8aee4b timestamp: 2024-08-19_19:57:23 tube: smithi user: hyelloji verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-08-20T00:47:02.732 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa; will attempt to use it 2024-08-20T00:47:02.732 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks 2024-08-20T00:47:02.733 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-08-20T00:47:02.734 INFO:teuthology.task.internal:Checking packages... 2024-08-20T00:47:02.756 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ef4275410d8483698016fe484b2a3c664080b4c1' 2024-08-20T00:47:02.756 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-08-20T00:47:02.756 INFO:teuthology.packaging:ref: None 2024-08-20T00:47:02.757 INFO:teuthology.packaging:tag: None 2024-08-20T00:47:02.757 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-08-19-1357 2024-08-20T00:47:02.757 INFO:teuthology.packaging:sha1: ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:47:02.757 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-08-19-1357 2024-08-20T00:47:02.898 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-5872.gef427541 2024-08-20T00:47:02.900 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-08-20T00:47:02.906 INFO:teuthology.task.internal:no buildpackages task found 2024-08-20T00:47:02.906 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-08-20T00:47:02.941 INFO:teuthology.task.internal:Saving configuration 2024-08-20T00:47:02.962 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-08-20T00:47:02.968 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-08-20T00:47:02.994 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 00:41:53.789746', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGtcIxzrXOKDKy/tQev3eRtiVzzw611JteP8ovZ4PguLbj26DphKV/fpH9cJ7p18F55olp7Bg9aKN0PsThKcaQ='} 2024-08-20T00:47:03.018 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 00:41:53.788625', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKTwKN0d/LiU6YLOqptevKhKjCkjJBi/nmVx0s+/IJlT+/RiAv6/VkiEyRM/1m3h/O5HLbOU2zAic6g/v3RHeOY='} 2024-08-20T00:47:03.039 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-08-20 00:41:53.787291', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHyffimhgJRJwueEoanC/bsPggJtKVVv5MlpklRrTrAKq0OaxzlD9IfSOH3ybVuBi6coszYXzPStmKUphWDQHdw='} 2024-08-20T00:47:03.039 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-08-20T00:47:03.046 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-08-20T00:47:03.046 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-08-20T00:47:03.046 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-08-20T00:47:03.046 INFO:teuthology.run_tasks:Running task console_log... 2024-08-20T00:47:03.143 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f00594ebbe0>, signals=[15]) 2024-08-20T00:47:03.143 INFO:teuthology.run_tasks:Running task internal.connect... 2024-08-20T00:47:03.355 INFO:teuthology.task.internal:Opening connections... 2024-08-20T00:47:03.356 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T00:47:03.357 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:47:03.436 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-08-20T00:47:03.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:47:03.514 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-08-20T00:47:03.514 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:47:03.590 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-08-20T00:47:03.596 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-08-20T00:47:03.614 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-08-20T00:47:03.615 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-08-20T00:47:03.672 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-08-20T00:47:03.672 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T00:47:03.673 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T00:47:03.674 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-08-20T00:47:03.698 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-08-20T00:47:03.716 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-08-20T00:47:03.716 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-08-20T00:47:03.775 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="CentOS Stream" 2024-08-20T00:47:03.775 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="9" 2024-08-20T00:47:03.775 INFO:teuthology.orchestra.run.smithi142.stdout:ID="centos" 2024-08-20T00:47:03.775 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="rhel fedora" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="9" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el9" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:LOGO="fedora-logo-icon" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://centos.org/" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T00:47:03.776 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T00:47:03.777 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-08-20T00:47:03.798 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-08-20T00:47:03.816 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-08-20T00:47:03.816 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="CentOS Stream" 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="9" 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:ID="centos" 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="rhel fedora" 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="9" 2024-08-20T00:47:03.875 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el9" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="CentOS Stream 9" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:LOGO="fedora-logo-icon" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://centos.org/" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-08-20T00:47:03.876 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-08-20T00:47:03.876 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-08-20T00:47:03.898 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-08-20T00:47:03.908 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-08-20T00:47:03.927 INFO:teuthology.task.internal:Checking for old test directory... 2024-08-20T00:47:03.927 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-08-20T00:47:03.932 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-08-20T00:47:03.935 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-08-20T00:47:03.953 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-08-20T00:47:03.961 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-08-20T00:47:03.962 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-08-20T00:47:03.990 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-08-20T00:47:03.992 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-08-20T00:47:04.108 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-08-20T00:47:04.182 INFO:teuthology.run_tasks:Running task kernel... 2024-08-20T00:47:04.205 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-08-20T00:47:04.205 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-08-20T00:47:04.205 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-08-20T00:47:04.206 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T00:47:04.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:47:04.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-08-20T00:47:04.413 DEBUG:teuthology.task.kernel:sha1 for client.0 is 7e35574567e19d2ca345caad671b98560d512bea 2024-08-20T00:47:04.413 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "7e35574567e19d2ca345caad671b98560d512bea"... 2024-08-20T00:47:04.413 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-08-20T00:47:04.431 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-496.el9.x86_64 2024-08-20T00:47:04.432 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-496.el9.x86_64 vs 7e35574567e19d2ca345caad671b98560d512bea 2024-08-20T00:47:04.432 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-08-20T00:47:04.432 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum upgrade -y linux-firmware 2024-08-20T00:47:10.584 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-08-20T00:47:18.646 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 20 MB 00:06 2024-08-20T00:47:29.208 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 989 kB/s | 6.6 MB 00:06 2024-08-20T00:47:30.826 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 18 kB 00:00 2024-08-20T00:47:31.655 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 31 MB/s | 22 MB 00:00 2024-08-20T00:47:37.103 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 8.2 kB/s | 1.7 kB 00:00 2024-08-20T00:47:39.108 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-08-20T00:47:39.109 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-08-20T00:47:39.109 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-08-20T00:47:39.217 INFO:teuthology.task.kernel:Downloading kernel 7e35574567e19d2ca345caad671b98560d512bea on client.0... 2024-08-20T00:47:39.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7e35574567e19d2ca345caad671b98560d512bea 2024-08-20T00:47:39.362 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/7e35574567e19d2ca345caad671b98560d512bea/centos/9/flavors/default/x86_64/ 2024-08-20T00:47:39.363 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc3_g7e35574567e1-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://2.chacra.ceph.com/r/kernel/testing/7e35574567e19d2ca345caad671b98560d512bea/centos/9/flavors/default/x86_64/ --input-file=- 2024-08-20T00:47:39.365 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi138.front.sepia.ceph.com to complete... 2024-08-20T00:47:45.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20 00:47:45 URL:https://2.chacra.ceph.com/r/kernel/testing/7e35574567e19d2ca345caad671b98560d512bea/centos/9/flavors/default/x86_64/kernel-6.11.0_rc3_g7e35574567e1-1.x86_64.rpm [360142159/360142159] -> "/tmp/kernel.x86_64.rpm" [1] 2024-08-20T00:47:45.702 INFO:teuthology.orchestra.run.smithi138.stderr:FINISHED --2024-08-20 00:47:45-- 2024-08-20T00:47:45.702 INFO:teuthology.orchestra.run.smithi138.stderr:Total wall clock time: 6.3s 2024-08-20T00:47:45.702 INFO:teuthology.orchestra.run.smithi138.stderr:Downloaded: 1 files, 343M in 6.0s (57.0 MB/s) 2024-08-20T00:47:45.705 INFO:teuthology.task.kernel:Installing kernel 7e35574567e19d2ca345caad671b98560d512bea on client.0... 2024-08-20T00:47:45.705 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-08-20T00:47:47.832 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying... ######################################## 2024-08-20T00:47:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:Preparing... ######################################## 2024-08-20T00:47:48.113 INFO:teuthology.orchestra.run.smithi138.stdout:Updating / installing... 2024-08-20T00:47:56.147 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-6.11.0_rc3_g7e35574567e1-1 ######################################## 2024-08-20T00:48:37.128 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi138.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-08-20T00:48:37.128 DEBUG:teuthology.orchestra.run.smithi138:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-08-20T00:48:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/System.map-6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/config-6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:/boot/vmlinuz-6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.213 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/System.map 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/config 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-08-20T00:48:37.214 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/crc32c-intel.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/serpent-avx2.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-08-20T00:48:37.215 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kernel 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kernel/cpu 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kernel/cpu/mce 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kvm 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kvm/kvm-amd.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kvm/kvm-intel.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/arch/x86/kvm/kvm.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/ansi_cprng.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/anubis.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/arc4.ko 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx 2024-08-20T00:48:37.216 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/async_memcpy.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/async_pq.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/async_raid6_recov.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/async_tx.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/async_xor.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/async_tx/raid6test.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/authenc.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/authencesn.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/blake2b_generic.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/blowfish_common.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/blowfish_generic.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/camellia_generic.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/cast5_generic.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/cast6_generic.ko 2024-08-20T00:48:37.217 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/cast_common.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/ccm.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/cmac.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/crc32_generic.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/crypto_engine.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/crypto_user.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/cts.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/des_generic.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/dh_generic.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/ecc.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/ecdh_generic.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/echainiv.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/essiv.ko 2024-08-20T00:48:37.218 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/fcrypt.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/khazad.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/lz4.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/lz4hc.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/md4.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/michael_mic.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/pcbc.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/pcrypt.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/rmd160.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/seed.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/serpent_generic.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/tcrypt.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/tea.ko 2024-08-20T00:48:37.219 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/twofish_common.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/twofish_generic.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/vmac.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/wp512.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/xcbc.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/xor.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/crypto/xxhash_generic.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/acpi_ipmi.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/acpi_pad.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/ec_sys.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/platform_profile.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/sbs.ko 2024-08-20T00:48:37.220 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/sbshc.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/acpi/video.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/acard-ahci.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/ahci_platform.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/ata_generic.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/libahci_platform.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_acpi.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_ali.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_amd.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_artop.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_atiixp.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_atp867x.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_cmd640.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_cmd64x.ko 2024-08-20T00:48:37.221 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_cypress.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_efar.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_hpt366.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_hpt37x.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_hpt3x2n.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_hpt3x3.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_it8213.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_it821x.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_jmicron.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_marvell.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_mpiix.ko 2024-08-20T00:48:37.222 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_netcell.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_ninja32.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_ns87410.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_ns87415.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_oldpiix.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_opti.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_optidma.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_pcmcia.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_pdc2027x.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_piccolo.ko 2024-08-20T00:48:37.223 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_rdc.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_sch.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_serverworks.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_sil680.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_sis.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_sl82c105.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_triflex.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pata_via.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/pdc_adma.ko 2024-08-20T00:48:37.224 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_inic162x.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_mv.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_nv.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_promise.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_qstor.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_sil.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_sil24.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_sis.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_svw.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_sx4.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_uli.ko 2024-08-20T00:48:37.225 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_via.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ata/sata_vsc.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/auxdisplay 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/auxdisplay/ks0108.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/base 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/base/regmap 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/base/regmap/regmap-i2c.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/base/regmap/regmap-mmio.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/bcma 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/bcma/bcma.ko 2024-08-20T00:48:37.226 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/aoe 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/aoe/aoe.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/brd.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/drbd 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/drbd/drbd.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/floppy.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/loop.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/mtip32xx 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-08-20T00:48:37.227 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/nbd.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/null_blk 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/null_blk/null_blk.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/pktcdvd.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/rbd.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/virtio_blk.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/xen-blkback 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/xen-blkfront.ko 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/zram 2024-08-20T00:48:37.228 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/block/zram/zram.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hangcheck-timer.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random/amd-rng.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random/intel-rng.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random/via-rng.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/hw_random/virtio-rng.ko 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi 2024-08-20T00:48:37.229 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_si.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/lp.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/mwave 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/mwave/mwave.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/ppdev.ko 2024-08-20T00:48:37.230 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tlclk.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm_atmel.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm_infineon.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm_nsc.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm_tis.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/uv_mmtimer.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/char/virtio_console.ko 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq 2024-08-20T00:48:37.231 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/p4-clockmod.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/powernow-k8.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/cpufreq/speedstep-lib.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/ccp 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/ccp/ccp.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel/qat 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel/qat/qat_common 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-08-20T00:48:37.232 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/padlock-aes.ko 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/padlock-sha.ko 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/virtio 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dca 2024-08-20T00:48:37.233 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dca/dca.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/devfreq 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dma 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dma/dw 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dma/dw/dw_dmac.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dma/ioat 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/dma/ioat/ioatdma.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/amd64_edac.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/e752x_edac.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/edac_mce_amd.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i3000_edac.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i3200_edac.ko 2024-08-20T00:48:37.234 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i5100_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i5400_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i7300_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i7core_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/i82975x_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/ie31200_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/sb_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/edac/x38_edac.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/firmware 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/firmware/edd.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/firmware/iscsi_ibft.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/gpio 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/gpio/gpio-ich.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/gpio/gpio-viperboard.ko 2024-08-20T00:48:37.235 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-appleir.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-aureal.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-axff.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-dr.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-elecom.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-elo.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-emsff.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-gaff.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-gt683r.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-gyration.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-holtek-kbd.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-holtek-mouse.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-holtekff.ko 2024-08-20T00:48:37.236 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-hyperv.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-icade.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-keytouch.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-kye.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-lcpower.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-led.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-lenovo.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-logitech-dj.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-multitouch.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-ortek.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-penmount.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-petalynx.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-picolcd.ko 2024-08-20T00:48:37.237 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-pl.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-plantronics.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-primax.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-rmi.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-arvo.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-common.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-isku.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-kone.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-konepure.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-lua.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-pyra.ko 2024-08-20T00:48:37.238 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-ryos.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat-savu.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-roccat.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-saitek.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-samsung.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-sensor-hub.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-sjoy.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-sony.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-speedlink.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-steelseries.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-sunplus.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-thrustmaster.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-tivo.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-tmff.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-topseed.ko 2024-08-20T00:48:37.239 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-twinhan.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-uclogic.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-waltop.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-wiimote.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-xinmo.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-zpff.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/hid-zydacron.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/uhid.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hid/wacom.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hv 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hv/hv_balloon.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hv/hv_utils.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hv/hv_vmbus.ko 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon 2024-08-20T00:48:37.240 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/abituguru.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/abituguru3.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/acpi_power_meter.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ad7414.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ad7418.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adc128d818.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adm1025.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adm1026.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adm1029.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adm1031.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adm9240.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ads7828.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7410.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7411.ko 2024-08-20T00:48:37.241 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7462.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7470.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7475.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/adt7x10.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/amc6821.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/applesmc.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/asb100.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/asc7621.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/asus_atk0110.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/atxp1.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/coretemp.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/dme1737.ko 2024-08-20T00:48:37.242 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ds1621.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ds620.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/emc1403.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/emc6w201.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/f71805f.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/f71882fg.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/f75375s.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/fam15h_power.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/fschmd.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/g760a.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/g762.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/gl518sm.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/gl520sm.ko 2024-08-20T00:48:37.243 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/hwmon-vid.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/i5500_temp.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/i5k_amb.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ibmaem.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ibmpex.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ina209.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ina2xx.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/it87.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/k10temp.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/k8temp.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lineage-pem.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm63.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm73.ko 2024-08-20T00:48:37.244 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm75.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm77.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm78.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm80.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm83.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm85.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm87.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm90.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm92.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm93.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm95234.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm95241.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/lm95245.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc2945.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4151.ko 2024-08-20T00:48:37.245 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4215.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4222.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4245.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4260.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/ltc4261.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max16065.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max1619.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max1668.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max197.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max6639.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max6650.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/max6697.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/mcp3021.ko 2024-08-20T00:48:37.246 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/nct6683.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/nct6775-core.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/nct6775.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/nct7802.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pc87360.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pc87427.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pcf8591.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/max16064.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/max34440.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/max8688.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-08-20T00:48:37.247 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/powr1220.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sch5627.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sch5636.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sch56xx-common.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sht15.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sht21.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/shtc1.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/sis5595.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/smsc47b397.ko 2024-08-20T00:48:37.248 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/smsc47m1.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/smsc47m192.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/thmc50.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/tmp102.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/tmp103.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/tmp401.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/tmp421.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/via-cputemp.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/via686a.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/vt1211.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/vt8231.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83627ehf.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83627hf.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83781d.ko 2024-08-20T00:48:37.249 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83791d.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83792d.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83793.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83795.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83l785ts.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/hwmon/w83l786ng.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/algos 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-08-20T00:48:37.250 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-i801.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-isch.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-parport.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-08-20T00:48:37.251 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-via.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/i2c-dev.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/i2c-mux.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/i2c-smbus.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/i2c/i2c-stub.ko 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband 2024-08-20T00:48:37.252 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/ib_cm.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/ib_core.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/ib_umad.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/iw_cm.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/rdma_cm.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/cxgb4 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mlx4 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mlx5 2024-08-20T00:48:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mthca 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/ipoib 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/iser 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/isert 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/srp 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/srpt 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-08-20T00:48:37.254 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/gameport 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/gameport/fm801-gp.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/gameport/gameport.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/joydev.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/keyboard 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/keyboard/gpio_keys.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/appletouch.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/bcm5974.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/cyapatp.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/elan_i2c.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/sermouse.ko 2024-08-20T00:48:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/synaptics_usb.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/mouse/vsxxxaa.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/rmi4 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/rmi4/rmi_core.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/serio 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/serio/altera_ps2.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/serio/arc_ps2.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/serio/serio_raw.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/input/sparse-keymap.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/leds-blinkm.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/leds-lm3530.ko 2024-08-20T00:48:37.256 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/leds-lp3944.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/leds-ss4200.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/bcache 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/bcache/bcache.ko 2024-08-20T00:48:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-bio-prison.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-cache-smq.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-cache.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-crypt.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-delay.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-flakey.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-log-userspace.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-multipath.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-queue-length.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-raid.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-round-robin.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-service-time.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-switch.ko 2024-08-20T00:48:37.258 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-thin-pool.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/dm-verity.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/persistent-data 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/raid0.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/raid1.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/raid10.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/md/raid456.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptbase.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptctl.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptfc.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptlan.ko 2024-08-20T00:48:37.259 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptsas.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptscsih.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/message/fusion/mptspi.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/lpc_ich.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/lpc_sch.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/mfd-core.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/sm501.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/viperboard.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/vx855.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/mfd/wl1273-core.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/altera-stapl 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-08-20T00:48:37.260 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/apds9802als.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/apds990x.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/bh1770glc.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/cb710 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/cb710/cb710.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/eeprom 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/eeprom/at24.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/eeprom/max6875.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/enclosure.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/hpilo.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/isl29003.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/isl29020.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/lis3lv02d 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-08-20T00:48:37.261 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/mei 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/mei/mei-me.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/mei/mei-txe.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/mei/mei.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-gru 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-gru/gru.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-xp 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-xp/xp.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-xp/xpc.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/tifm_7xx1.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/tifm_core.ko 2024-08-20T00:48:37.262 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/tsl2550.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/vmw_balloon.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/vmw_vmci 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/bonding 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/bonding/bonding.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/b53 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/b53/b53_common.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/bcm-sf2.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/mv88e6060.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/mv88e6xxx 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/dummy.ko 2024-08-20T00:48:37.263 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/eql.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/3com 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/8390 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/8390/8390.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/adaptec 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-08-20T00:48:37.264 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/agere 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/agere/et131x.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/alteon 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/altera 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/amd 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/alx 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atl1c 2024-08-20T00:48:37.265 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atl1e 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atlx 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/genet 2024-08-20T00:48:37.266 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/brocade 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/brocade/bna 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-08-20T00:48:37.267 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/cisco 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/cisco/enic 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dlink 2024-08-20T00:48:37.268 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/dnet.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/emulex 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/emulex/benet 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/ethoc.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/fealnx.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/e100.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/e1000 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/e1000e 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/fm10k 2024-08-20T00:48:37.269 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/i40e 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/iavf 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/igb 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/igbvf 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/ixgbe 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/ixgbevf 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-08-20T00:48:37.270 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/libeth 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/libie 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/jme.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/marvell 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/marvell/skge.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx4 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx5 2024-08-20T00:48:37.271 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/micrel 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/myricom 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/myricom/myri10ge 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/natsemi 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/neterion 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-08-20T00:48:37.272 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/nvidia 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/packetengines 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic/netxen 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-08-20T00:48:37.273 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/rdc 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/realtek 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/realtek/atp.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/rocker 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sfc 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/silan 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sis 2024-08-20T00:48:37.274 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sis/sis190.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sis/sis900.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/smsc 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/stmicro 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/stmicro/stmmac 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sun 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sun/cassini.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sun/niu.ko 2024-08-20T00:48:37.275 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sun/sungem.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/tehuti 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/ti 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/ti/tlan.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/via 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/wiznet 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/xircom 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-08-20T00:48:37.276 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/geneve.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/hyperv 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ieee802154 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ieee802154/fakelb.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ifb.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ipvlan 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ipvlan/ipvlan.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/macvlan.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/macvtap.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mdio 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mdio.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-08-20T00:48:37.277 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mdio/mdio-regmap.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/mii.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/net_failover.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/netconsole.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/nlmon.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/pcs 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/pcs/pcs-lynx.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/amd.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/bcm7xxx.ko 2024-08-20T00:48:37.278 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/bcm87xx.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/broadcom.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/cicada.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/davicom.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/dp83640.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/et1011c.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/icplus.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/lxt.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/marvell.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/micrel.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/national.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/phylink.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/qsemi.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/realtek.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/smsc.ko 2024-08-20T00:48:37.279 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/ste10Xp.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/phy/vitesse.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/bsd_comp.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/ppp_async.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/ppp_deflate.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/ppp_generic.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/ppp_mppe.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/ppp_synctty.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/pppoe.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/pppox.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/ppp/pptp.ko 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/slip 2024-08-20T00:48:37.280 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/slip/slhc.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/slip/slip.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/sungem_phy.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/tap.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team_mode_activebackup.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team_mode_broadcast.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team_mode_random.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/tun.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/veth.ko 2024-08-20T00:48:37.281 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/virtio_net.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/vmxnet3 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/vxlan 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/vxlan/vxlan.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/xen-netback 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/xen-netback/xen-netback.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/net/xen-netfront.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ntb 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ntb/ntb.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/nvme 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/nvme/host 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/nvme/host/nvme-core.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/nvme/host/nvme.ko 2024-08-20T00:48:37.282 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/parport 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/parport/parport.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/parport/parport_cs.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/parport/parport_pc.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/parport/parport_serial.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci/hotplug 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci/pcie 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci/pcie/aer_inject.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pci/xen-pcifront.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pcmcia 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pcmcia/i82092.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pcmcia/pd6729.ko 2024-08-20T00:48:37.283 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pcmcia/yenta_socket.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pinctrl 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pinctrl/intel 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/acer-wmi.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/acerhdf.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/apple-gmux.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/asus-laptop.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/asus-wmi.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/classmate-laptop.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-08-20T00:48:37.284 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/hdaps.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/intel 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/intel_ips.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/msi-wmi.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/mxm-wmi.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/samsung-laptop.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/samsung-q10.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/topstar-laptop.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-08-20T00:48:37.285 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/toshiba_haps.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/wmi-bmof.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/platform/x86/wmi.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/powercap 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/powercap/intel_rapl_common.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/powercap/intel_rapl_msr.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps/clients 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps/clients/pps-gpio.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps/clients/pps-ldisc.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps/clients/pps_parport.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/pps/pps_core.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ptp 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ptp/ptp.ko 2024-08-20T00:48:37.286 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ptp/ptp_kvm.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ras 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ras/amd 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ras/amd/atl 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ras/amd/fmpm.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-bq32k.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1286.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1307.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1374.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1511.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1553.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1672.ko 2024-08-20T00:48:37.287 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds1742.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds2404.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-ds3232.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-em3027.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-fm3130.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-isl12022.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-isl1208.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-m41t80.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-m48t35.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-m48t59.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-max6900.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-msm6242.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-pcf2127.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-pcf85063.ko 2024-08-20T00:48:37.288 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-pcf8523.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-pcf8563.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-pcf8583.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-rp5c01.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-rs5c372.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-rx8025.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-rx8581.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/rtc/rtc-x1205.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/3w-9xxx.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/3w-sas.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/3w-xxxx.ko 2024-08-20T00:48:37.289 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/BusLogic.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/a100u2w.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/aacraid 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/aacraid/aacraid.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/advansys.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/aic7xxx 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/am53c974.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/arcmsr 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/atp870u.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/be2iscsi 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-08-20T00:48:37.290 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bfa 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bfa/bfa.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bnx2fc 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bnx2i 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/ch.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/csiostor 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/csiostor/csiostor.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi/cxgb3i 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi/cxgb4i 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-08-20T00:48:37.291 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/dc395x.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/device_handler 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/dmx3191d.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/esas2r 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/esas2r/esas2r.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/esp_scsi.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/fcoe 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/fcoe/fcoe.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-08-20T00:48:37.292 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/fnic 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/fnic/fnic.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/hpsa.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/hptiop.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/hv_storvsc.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/initio.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/ipr.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/ips.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/isci 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/isci/isci.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/iscsi_tcp.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libfc 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libfc/libfc.ko 2024-08-20T00:48:37.293 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libiscsi.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libiscsi_tcp.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libsas 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/libsas/libsas.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/lpfc 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/lpfc/lpfc.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/megaraid 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/megaraid.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/mpt3sas 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/mvsas 2024-08-20T00:48:37.294 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/mvsas/mvsas.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/mvumi.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/pm8001 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/pmcraid.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla1280.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla2xxx 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla4xxx 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/raid_class.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_debug.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_transport_fc.ko 2024-08-20T00:48:37.295 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_transport_sas.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_transport_spi.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/scsi_transport_srp.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/ses.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/st.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/stex.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/sym53c8xx_2 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/virtio_scsi.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/vmw_pvscsi.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/scsi/wd719x.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ssb 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ssb/ssb.ko 2024-08-20T00:48:37.296 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/iscsi 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/loopback 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/loopback/tcm_loop.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/target_core_file.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/target_core_iblock.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/target_core_mod.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/target_core_pscsi.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/target_core_user.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/tcm_fc 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel 2024-08-20T00:48:37.297 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-08-20T00:48:37.298 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/ipwireless 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/n_gsm.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/n_hdlc.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/nozomi.ko 2024-08-20T00:48:37.299 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial/8250 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial/arc_uart.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial/jsm 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/tty/serial/jsm/jsm.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ufs 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ufs/core 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ufs/core/ufshcd-core.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ufs/host 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio/uio.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio/uio_aec.ko 2024-08-20T00:48:37.300 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio/uio_cif.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio/uio_pci_generic.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/uio/uio_sercos3.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/class 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/class/cdc-acm.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/class/cdc-wdm.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/class/usblp.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/class/usbtmc.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/host 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/host/sl811-hcd.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/image 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/image/mdc800.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/image/microtek.ko 2024-08-20T00:48:37.301 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/adutux.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/appledisplay.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/emi26.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/emi62.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/ezusb.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/idmouse.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/iowarrior.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/isight_firmware.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/ldusb.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/legousbtower.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/sisusbvga 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-08-20T00:48:37.302 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/trancevibrator.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/usb3503.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/usblcd.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/usbsevseg.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/uss720.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/misc/yurex.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/phy 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/phy/phy-generic.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/aircable.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ark3116.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/belkin_sa.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ch341.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/cp210x.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/cyberjack.ko 2024-08-20T00:48:37.303 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/cypress_m8.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/digi_acceleport.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/empeg.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ftdi_sio.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/garmin_gps.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/io_edgeport.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/io_ti.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ipaq.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ipw.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ir-usb.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/keyspan.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/keyspan_pda.ko 2024-08-20T00:48:37.304 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/kl5kusb105.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/kobil_sct.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/mct_u232.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/mos7720.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/mos7840.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/navman.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/omninet.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/opticon.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/option.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/oti6858.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/pl2303.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/qcaux.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/qcserial.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/quatech2.ko 2024-08-20T00:48:37.305 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/safe_serial.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/sierra.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/spcp8x5.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ssu100.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/symbolserial.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/usb_debug.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/usb_wwan.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/visor.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/whiteheat.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/serial/xsens_mt.ko 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage 2024-08-20T00:48:37.306 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/uas.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-alauda.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-cypress.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-datafab.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-freecom.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-isd200.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-karma.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-onetouch.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-realtek.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-sddr09.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-sddr55.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/ums-usbat.ko 2024-08-20T00:48:37.307 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/storage/usb-storage.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/usbip 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/usbip/usbip-core.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/usbip/usbip-host.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio/pci 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio/pci/vfio-pci.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio/vfio.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vhost 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vhost/vhost.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vhost/vhost_iotlb.ko 2024-08-20T00:48:37.308 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vhost/vhost_net.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/vhost/vhost_scsi.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/video 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/video/backlight 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/video/backlight/apple_bl.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/video/backlight/lcd.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/video/backlight/platform_lcd.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_balloon.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_mmio.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_pci.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-08-20T00:48:37.309 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/virtio/virtio_ring.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/masters 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/masters/ds2482.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/masters/ds2490.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-08-20T00:48:37.310 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_smem.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/slaves/w1_therm.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/w1/wire.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/alim1535_wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/alim7101_wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/f71808e_wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/hpwdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/i6300esb.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/iTCO_wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/ib700wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/ibmasr.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-08-20T00:48:37.311 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/it8712f_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/it87_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/machzwd.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/nv_tco.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/pcwd_pci.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/pcwd_usb.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/sch311x_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/softdog.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/sp5100_tco.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/via_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/w83877f_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/w83977f_wdt.ko 2024-08-20T00:48:37.312 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/wdt_pci.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/watchdog/xen_wdt.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-acpi-processor.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-evtchn.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-gntalloc.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-gntdev.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-pciback 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xen-privcmd.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xenfs 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/drivers/xen/xenfs/xenfs.ko 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/9p 2024-08-20T00:48:37.313 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/9p/9p.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/affs 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/affs/affs.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/befs 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/befs/befs.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/binfmt_misc.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/btrfs 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/btrfs/btrfs.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/cachefiles 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/cachefiles/cachefiles.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ceph 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ceph/ceph.ko 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/coda 2024-08-20T00:48:37.314 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/coda/coda.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/cramfs 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/cramfs/cramfs.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/dlm 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/dlm/dlm.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ecryptfs 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ecryptfs/ecryptfs.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/f2fs 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/f2fs/f2fs.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fat 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fat/fat.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fat/msdos.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fat/vfat.ko 2024-08-20T00:48:37.315 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fuse 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fuse/cuse.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/fuse/fuse.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/gfs2 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/gfs2/gfs2.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/hfs 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/hfs/hfs.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/hfsplus 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/hfsplus/hfsplus.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/isofs 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/isofs/isofs.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/jfs 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/jfs/jfs.ko 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/lockd 2024-08-20T00:48:37.316 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/lockd/lockd.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/minix 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/minix/minix.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/netfs 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/netfs/netfs.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/blocklayout 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/filelayout 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/flexfilelayout 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/nfs.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/nfsv3.ko 2024-08-20T00:48:37.317 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs/nfsv4.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs_common 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs_common/grace.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfs_common/nfs_acl.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfsd 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nfsd/nfsd.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nilfs2 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nilfs2/nilfs2.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-celtic.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-centeuro.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-croatian.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-cyrillic.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-gaelic.ko 2024-08-20T00:48:37.318 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-greek.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-iceland.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-inuit.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-roman.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-romanian.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/mac-turkish.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp1250.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp1251.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp1255.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp737.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp775.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp850.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp852.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp855.ko 2024-08-20T00:48:37.319 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp857.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp860.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp861.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp862.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp863.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp864.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp865.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp866.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp869.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp874.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp932.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp936.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp949.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_cp950.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_euc-jp.ko 2024-08-20T00:48:37.320 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-1.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-13.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-14.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-15.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-2.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-3.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-4.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-5.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-6.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-7.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_iso8859-9.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_koi8-r.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_koi8-ru.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_koi8-u.ko 2024-08-20T00:48:37.321 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_ucs2_utils.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/nls/nls_utf8.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/cluster 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/dlm 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/dlmfs 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/ocfs2.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/overlayfs 2024-08-20T00:48:37.322 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/overlayfs/overlay.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/pstore 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/pstore/ramoops.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/reiserfs 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/reiserfs/reiserfs.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/romfs 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/romfs/romfs.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb/client 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb/client/cifs.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb/common 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb/common/cifs_arc4.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/smb/common/cifs_md4.ko 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/squashfs 2024-08-20T00:48:37.323 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/squashfs/squashfs.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/sysv 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/sysv/sysv.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/udf 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/udf/udf.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ufs 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/ufs/ufs.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/xfs 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/fs/xfs/xfs.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel/rcu 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel/rcu/rcutorture.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel/torture.ko 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel/trace 2024-08-20T00:48:37.324 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/asn1_encoder.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crc-itu-t.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crc8.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crypto 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crypto/libaescfb.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crypto/libarc4.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/crypto/libdes.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/libcrc32c.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/lru_cache.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/lz4 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/lz4/lz4_compress.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/lz4/lz4hc_compress.ko 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/math 2024-08-20T00:48:37.325 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/math/cordic.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/raid6 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/raid6/raid6_pq.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/reed_solomon 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/reed_solomon/reed_solomon.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/ts_bm.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/ts_fsm.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/ts_kmp.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/zstd 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/lib/zstd/zstd_compress.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/mm 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/mm/hwpoison-inject.ko 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net 2024-08-20T00:48:37.326 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/6lowpan.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_dest.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_fragment.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_hop.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_ipv6.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_mobility.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_routing.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/6lowpan/nhc_udp.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802/garp.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802/mrp.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802/p8022.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802/psnap.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/802/stp.ko 2024-08-20T00:48:37.327 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/8021q 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/8021q/8021q.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/9p 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/9p/9pnet.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/9p/9pnet_fd.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/9p/9pnet_rdma.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/9p/9pnet_virtio.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/batman-adv 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/batman-adv/batman-adv.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/br_netfilter.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/bridge.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-08-20T00:48:37.328 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_among.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_arp.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_ip.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_limit.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_log.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_mark.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_snat.ko 2024-08-20T00:48:37.329 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_stp.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/ebtables.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ceph 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ceph/libceph.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/core 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/core/failover.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/core/pktgen.ko 2024-08-20T00:48:37.330 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dccp 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dccp/dccp.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dccp/dccp_diag.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dccp/dccp_ipv4.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dccp/dccp_ipv6.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dns_resolver 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dns_resolver/dns_resolver.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa/dsa_core.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa/tag_brcm.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa/tag_dsa.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa/tag_none.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/dsa/tag_trailer.ko 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ieee802154 2024-08-20T00:48:37.331 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ieee802154/6lowpan 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ieee802154/ieee802154.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ieee802154/ieee802154_socket.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ah4.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/esp4.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/fou.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/gre.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/inet_diag.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ip_gre.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ip_tunnel.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ip_vti.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ipcomp.ko 2024-08-20T00:48:37.332 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/ipip.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/arp_tables.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/iptable_security.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-08-20T00:48:37.333 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_bic.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_dctcp.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_diag.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_highspeed.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_htcp.ko 2024-08-20T00:48:37.334 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_hybla.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_illinois.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_lp.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_scalable.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_vegas.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_veno.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_westwood.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tcp_yeah.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/tunnel4.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/udp_diag.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/udp_tunnel.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv4/xfrm4_tunnel.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/ah6.ko 2024-08-20T00:48:37.335 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/esp6.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/fou6.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/ip6_tunnel.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/ip6_vti.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/ipcomp6.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-08-20T00:48:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-08-20T00:48:37.337 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/sit.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/tunnel6.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/ipv6/xfrm6_tunnel.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/key 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/key/af_key.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_core.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_debugfs.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_eth.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_ip.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_ip6.ko 2024-08-20T00:48:37.338 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_netlink.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/l2tp/l2tp_ppp.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/llc 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/llc/llc.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/mac802154 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/mac802154/mac802154.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/mpls 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/mpls/mpls_gso.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-08-20T00:48:37.339 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-08-20T00:48:37.340 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conncount.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack.ko 2024-08-20T00:48:37.341 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_h323.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_irc.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_sane.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_sip.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_dup_netdev.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_flow_table.ko 2024-08-20T00:48:37.342 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_flow_table_inet.ko 2024-08-20T00:48:37.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_log_syslog.ko 2024-08-20T00:48:37.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat.ko 2024-08-20T00:48:37.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat_amanda.ko 2024-08-20T00:48:37.345 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat_ftp.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat_irc.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat_sip.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_nat_tftp.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_synproxy_core.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nf_tables.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nfnetlink.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nfnetlink_acct.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nfnetlink_log.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nfnetlink_osf.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nfnetlink_queue.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_chain_nat.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_compat.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_ct.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_dup_netdev.ko 2024-08-20T00:48:37.346 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_fib.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_fwd_netdev.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_hash.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_limit.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_log.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_masq.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_nat.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_numgen.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_queue.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_quota.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_redir.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_reject.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_reject_inet.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_socket.ko 2024-08-20T00:48:37.347 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/nft_tunnel.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_AUDIT.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_CHECKSUM.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_CLASSIFY.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_CT.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_DSCP.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_HL.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_HMARK.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_IDLETIMER.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_LED.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_LOG.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_MASQUERADE.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_NETMAP.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_NFLOG.ko 2024-08-20T00:48:37.348 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_NFQUEUE.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_RATEEST.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_REDIRECT.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_SECMARK.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_TCPMSS.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_TEE.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_TPROXY.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_TRACE.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_addrtype.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_bpf.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_cgroup.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_cluster.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_comment.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_connbytes.ko 2024-08-20T00:48:37.349 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_connlabel.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_connlimit.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_connmark.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_conntrack.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_cpu.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_dccp.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_devgroup.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_dscp.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_ecn.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_esp.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_hashlimit.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_helper.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_hl.ko 2024-08-20T00:48:37.350 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_ipcomp.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_iprange.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_ipvs.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_l2tp.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_length.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_limit.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_mac.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_mark.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_multiport.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_nat.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_nfacct.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_osf.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_owner.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_physdev.ko 2024-08-20T00:48:37.351 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_pkttype.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_policy.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_quota.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_rateest.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_realm.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_recent.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_sctp.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_set.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_socket.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_state.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_statistic.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_string.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_tcpmss.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_time.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netfilter/xt_u32.ko 2024-08-20T00:48:37.352 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netlink 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/netlink/netlink_diag.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/nsh 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/nsh/nsh.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/openvswitch 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/openvswitch/openvswitch.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/openvswitch/vport-geneve.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/openvswitch/vport-gre.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/openvswitch/vport-vxlan.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/packet 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/packet/af_packet_diag.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/rds 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/rds/rds.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/rds/rds_rdma.ko 2024-08-20T00:48:37.353 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/rds/rds_tcp.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_csum.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_gact.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_mirred.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_nat.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_pedit.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_police.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_simple.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_skbedit.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/act_vlan.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_basic.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_bpf.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_flow.ko 2024-08-20T00:48:37.354 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_fw.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_route.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/cls_u32.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_cmp.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_ipset.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_meta.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_nbyte.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_text.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/em_u32.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_choke.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_codel.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_drr.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_fq.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_gred.ko 2024-08-20T00:48:37.355 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_hfsc.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_hhf.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_htb.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_ingress.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_mqprio.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_mqprio_lib.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_multiq.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_netem.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_pie.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_plug.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_prio.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_qfq.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_red.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_sfb.ko 2024-08-20T00:48:37.356 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_sfq.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_tbf.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sched/sch_teql.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sctp 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sctp/sctp.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sctp/sctp_diag.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/auth_gss 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/sunrpc.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/xprtrdma 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/tipc 2024-08-20T00:48:37.357 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/tipc/tipc.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/tipc/tipc_diag.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/unix 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/unix/unix_diag.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock/vsock.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock/vsock_diag.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/vmw_vsock/vsock_loopback.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/xfrm 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/net/xfrm/xfrm_ipcomp.ko 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security/keys 2024-08-20T00:48:37.358 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security/keys/encrypted-keys 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security/keys/trusted-keys 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/kernel/security/keys/trusted-keys/trusted.ko 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.alias 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.alias.bin 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.builtin 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.builtin.alias.bin 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.builtin.bin 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.builtin.modinfo 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.dep 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.dep.bin 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.devname 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.order 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.softdep 2024-08-20T00:48:37.359 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.symbols 2024-08-20T00:48:37.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.symbols.bin 2024-08-20T00:48:37.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/modules.weakdep 2024-08-20T00:48:37.360 INFO:teuthology.orchestra.run.smithi138.stdout:/lib/modules/6.11.0-rc3-g7e35574567e1/vmlinuz 2024-08-20T00:48:37.364 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.364 DEBUG:teuthology.orchestra.run.smithi138:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout:postinstall scriptlet (using /bin/sh): 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/bin/kernel-install add 6.11.0-rc3-g7e35574567e1 /lib/modules/6.11.0-rc3-g7e35574567e1/vmlinuz 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout:for file in vmlinuz System.map config; do 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc3-g7e35574567e1/${file}" "/boot/${file}-6.11.0-rc3-g7e35574567e1"; then 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout: cp "/lib/modules/6.11.0-rc3-g7e35574567e1/${file}" "/boot/${file}-6.11.0-rc3-g7e35574567e1" 2024-08-20T00:48:37.389 INFO:teuthology.orchestra.run.smithi138.stdout: fi 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout:done 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -d "/lib/modules/6.11.0-rc3-g7e35574567e1/dtb" ] && \ 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout: ! diff -rq "/lib/modules/6.11.0-rc3-g7e35574567e1/dtb" "/boot/dtb-6.11.0-rc3-g7e35574567e1" >/dev/null 2>&1; then 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout: rm -rf "/boot/dtb-6.11.0-rc3-g7e35574567e1" 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout: cp -r "/lib/modules/6.11.0-rc3-g7e35574567e1/dtb" "/boot/dtb-6.11.0-rc3-g7e35574567e1" 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout:if [ ! -e "/lib/modules/6.11.0-rc3-g7e35574567e1/modules.dep" ]; then 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout: /usr/sbin/depmod 6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.390 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-08-20T00:48:37.391 INFO:teuthology.orchestra.run.smithi138.stdout:preuninstall scriptlet (using /bin/sh): 2024-08-20T00:48:37.391 INFO:teuthology.orchestra.run.smithi138.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-08-20T00:48:37.391 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-install remove 6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.391 INFO:teuthology.orchestra.run.smithi138.stdout:fi 2024-08-20T00:48:37.391 DEBUG:teuthology.orchestra.run.smithi138:> sudo rpm -qi grub2-tools 2024-08-20T00:48:37.462 INFO:teuthology.orchestra.run.smithi138.stdout:Name : grub2-tools 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Epoch : 1 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Version : 2.06 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Release : 82.el9 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Architecture: x86_64 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Install Date: Mon 19 Aug 2024 09:46:08 PM UTC 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Group : Unspecified 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Size : 8274670 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:License : GPLv3+ 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-08-20T00:48:37.463 INFO:teuthology.orchestra.run.smithi138.stdout:Packager : builder@centos.org 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:Vendor : CentOS 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:URL : http://www.gnu.org/software/grub/ 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:Summary : Support tools for GRUB. 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:Description : 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:customizable bootloader with modular architecture. It supports a rich 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:variety of kernel formats, file systems, computer architectures and 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:hardware devices. 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:48:37.464 INFO:teuthology.orchestra.run.smithi138.stdout:This subpackage provides tools for support of all platforms. 2024-08-20T00:48:37.465 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-08-20T00:48:37.465 INFO:teuthology.task.kernel:Updating grub on smithi138 to boot 6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:37.465 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-08-20T00:48:38.021 INFO:teuthology.orchestra.run.smithi138.stderr:Generating grub configuration file ... 2024-08-20T00:48:39.401 INFO:teuthology.orchestra.run.smithi138.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-08-20T00:48:39.426 INFO:teuthology.orchestra.run.smithi138.stderr:done 2024-08-20T00:48:39.428 DEBUG:teuthology.orchestra.run.smithi138:> mktemp 2024-08-20T00:48:39.444 INFO:teuthology.orchestra.run.smithi138.stdout:/tmp/tmp.JlHz2CJNLo 2024-08-20T00:48:39.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JlHz2CJNLo 2024-08-20T00:48:39.511 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0666 /tmp/tmp.JlHz2CJNLo 2024-08-20T00:48:39.665 DEBUG:teuthology.orchestra.remote:smithi138:/tmp/tmp.JlHz2CJNLo is 6KB 2024-08-20T00:48:39.713 DEBUG:teuthology.orchestra.run.smithi138:> rm -fr /tmp/tmp.JlHz2CJNLo 2024-08-20T00:48:39.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo /bin/ls /boot/loader/entries || true 2024-08-20T00:48:39.790 INFO:teuthology.orchestra.run.smithi138.stdout:b82e8f2bb04e44c8ac70748537791130-0-rescue.conf 2024-08-20T00:48:39.791 INFO:teuthology.orchestra.run.smithi138.stdout:b82e8f2bb04e44c8ac70748537791130-5.14.0-239.el9.x86_64.conf 2024-08-20T00:48:39.791 INFO:teuthology.orchestra.run.smithi138.stdout:b82e8f2bb04e44c8ac70748537791130-5.14.0-496.el9.x86_64.conf 2024-08-20T00:48:39.791 INFO:teuthology.orchestra.run.smithi138.stdout:b82e8f2bb04e44c8ac70748537791130-6.11.0-rc3-g7e35574567e1.conf 2024-08-20T00:48:39.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo grub2-set-default b82e8f2bb04e44c8ac70748537791130-6.11.0-rc3-g7e35574567e1 2024-08-20T00:48:39.944 DEBUG:teuthology.orchestra.run.smithi138:> sudo shutdown -r now 2024-08-20T00:49:09.947 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-08-20T00:49:09.948 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:49:09.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:49:44.414 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-08-20T00:49:53.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:49:53.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:50:11.807 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-08-20T00:50:23.807 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:50:23.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:50:26.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.138 2024-08-20T00:50:41.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:50:41.880 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:50:42.237 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-08-20T00:50:42.717 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:50:42.717 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc3-g7e35574567e1"... 2024-08-20T00:50:42.717 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-08-20T00:50:42.752 INFO:teuthology.orchestra.run.smithi138.stdout:6.11.0-rc3-g7e35574567e1 2024-08-20T00:50:42.753 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc3-g7e35574567e1 vs 6.11.0-rc3-g7e35574567e1 2024-08-20T00:50:42.753 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-08-20T00:50:42.753 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-08-20T00:50:43.754 DEBUG:teuthology.parallel:result is None 2024-08-20T00:50:43.755 INFO:teuthology.run_tasks:Running task internal.base... 2024-08-20T00:50:43.764 INFO:teuthology.task.internal:Creating test directory... 2024-08-20T00:50:43.764 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T00:50:43.768 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T00:50:43.771 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-08-20T00:50:43.802 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-08-20T00:50:43.808 INFO:teuthology.run_tasks:Running task internal.archive... 2024-08-20T00:50:43.833 INFO:teuthology.task.internal:Creating archive directory... 2024-08-20T00:50:43.833 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T00:50:43.847 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T00:50:43.849 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-08-20T00:50:43.887 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-08-20T00:50:43.896 INFO:teuthology.task.internal:Enabling coredump saving... 2024-08-20T00:50:43.897 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T00:50:43.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:50:43.936 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-08-20T00:50:43.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:50:43.953 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T00:50:43.957 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T00:50:43.979 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-08-20T00:50:44.034 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.050 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.055 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.067 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.103 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.129 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-08-20T00:50:44.132 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-08-20T00:50:44.140 INFO:teuthology.task.internal:Configuring sudo... 2024-08-20T00:50:44.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T00:50:44.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T00:50:44.146 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-08-20T00:50:44.195 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-08-20T00:50:44.204 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-08-20T00:50:44.204 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T00:50:44.208 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T00:50:44.214 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-08-20T00:50:44.239 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.341 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.424 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:44.487 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:44.556 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:50:44.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T00:50:44.610 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.656 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.717 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:44.759 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:44.803 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:50:44.803 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T00:50:44.871 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.917 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-08-20T00:50:44.977 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:45.020 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-08-20T00:50:45.064 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:50:45.064 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-08-20T00:50:45.130 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-08-20T00:50:45.133 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-08-20T00:50:45.136 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-08-20T00:50:45.169 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T00:50:45.204 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T00:50:45.210 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T00:50:45.601 INFO:teuthology.run_tasks:Running task internal.timer... 2024-08-20T00:50:45.609 INFO:teuthology.task.internal:Starting timer... 2024-08-20T00:50:45.609 INFO:teuthology.run_tasks:Running task pcp... 2024-08-20T00:50:45.662 INFO:teuthology.run_tasks:Running task selinux... 2024-08-20T00:50:45.695 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-08-20T00:50:45.695 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-08-20T00:50:45.790 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-08-20T00:50:45.793 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-08-20T00:50:45.866 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-08-20T00:50:45.869 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2024-08-20T00:50:45.909 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2024-08-20T00:50:45.912 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-08-20T00:50:45.912 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-08-20T00:50:45.949 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-08-20T00:50:45.949 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-08-20T00:50:45.970 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-08-20T00:50:45.970 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2024-08-20T00:50:45.995 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2024-08-20T00:50:45.995 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi138.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2024-08-20T00:50:45.995 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T00:50:46.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:50:46.052 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T00:50:46.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:50:46.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T00:50:46.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T00:50:46.110 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-08-20T00:50:46.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-08-20T00:50:46.169 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-08-20T00:50:46.198 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2024-08-20T00:50:46.223 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-08-20T00:50:46.233 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-08-20T00:50:46.236 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-08-20T00:50:46.343 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-08-20T00:50:46.354 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-08-20T00:50:46.370 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi138.front.sepia.ceph.com,smithi142.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-08-20T00:52:58.732 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi138.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-08-20T00:52:58.734 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:52:58.735 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:52:58.829 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-08-20T00:52:58.936 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-08-20T00:52:58.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-20T00:52:58.937 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:52:59.011 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-08-20T00:52:59.090 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-08-20T00:52:59.090 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-20T00:52:59.091 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-08-20T00:52:59.167 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-08-20T00:52:59.252 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-08-20T00:52:59.252 INFO:teuthology.run_tasks:Running task clock... 2024-08-20T00:52:59.263 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-08-20T00:52:59.263 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T00:52:59.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T00:52:59.268 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T00:52:59.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T00:52:59.271 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-08-20T00:52:59.271 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T00:52:59.299 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T00:52:59.314 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T00:52:59.324 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T00:52:59.325 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-08-20T00:52:59.338 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-08-20T00:52:59.339 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T00:52:59.350 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-08-20T00:52:59.355 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-08-20T00:52:59.364 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T00:52:59.365 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2024-08-20T00:52:59.377 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2024-08-20T00:52:59.378 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T00:52:59.392 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T00:52:59.405 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-08-20T00:52:59.407 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T00:52:59.429 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-08-20T00:52:59.438 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.441 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.458 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-08-20T00:52:59.463 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.466 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.487 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-08-20T00:52:59.578 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.586 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T00:52:59.587 INFO:teuthology.run_tasks:Running task install... 2024-08-20T00:52:59.595 DEBUG:teuthology.task.install:project ceph 2024-08-20T00:52:59.595 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'}} 2024-08-20T00:52:59.595 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1'} 2024-08-20T00:52:59.595 INFO:teuthology.task.install:Using flavor: default 2024-08-20T00:52:59.601 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-08-20T00:52:59.601 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-08-20T00:52:59.601 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T00:52:59.602 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:52:59.604 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T00:52:59.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:52:59.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'tag': None, 'wait_for_package': False} 2024-08-20T00:52:59.605 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:52:59.747 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T00:52:59.747 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T00:52:59.756 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T00:52:59.756 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T00:52:59.759 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/ 2024-08-20T00:52:59.759 INFO:teuthology.task.install.rpm:Package version is 19.0.0-5872.gef427541 2024-08-20T00:52:59.858 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T00:52:59.858 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:52:59.858 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T00:52:59.868 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T00:52:59.868 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:52:59.868 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T00:52:59.871 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-08-20T00:52:59.871 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:52:59.871 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-08-20T00:52:59.894 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T00:52:59.895 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T00:52:59.897 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T00:52:59.898 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T00:52:59.915 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-08-20T00:52:59.915 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ef4275410d8483698016fe484b2a3c664080b4c1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-08-20T00:52:59.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T00:52:59.964 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T00:53:00.012 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-08-20T00:53:00.038 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T00:53:00.039 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T00:53:00.090 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-08-20T00:53:00.102 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2024-08-20T00:53:00.103 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-08-20T00:53:00.104 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2024-08-20T00:53:00.105 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-08-20T00:53:00.145 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-08-20T00:53:00.149 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-08-20T00:53:00.331 INFO:teuthology.orchestra.run.smithi167.stdout:45 files removed 2024-08-20T00:53:00.332 INFO:teuthology.orchestra.run.smithi142.stdout:45 files removed 2024-08-20T00:53:00.358 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T00:53:00.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T00:53:00.437 INFO:teuthology.orchestra.run.smithi138.stdout:45 files removed 2024-08-20T00:53:00.470 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-08-20T00:53:00.894 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 324 kB/s | 84 kB 00:00 2024-08-20T00:53:00.905 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-08-20T00:53:01.154 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 289 kB/s | 84 kB 00:00 2024-08-20T00:53:01.170 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 103 kB/s | 24 kB 00:00 2024-08-20T00:53:01.189 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-08-20T00:53:01.437 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-08-20T00:53:01.465 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-20T00:53:01.470 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 89 kB/s | 24 kB 00:00 2024-08-20T00:53:01.747 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-08-20T00:53:07.713 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - BaseOS 1.3 MB/s | 8.2 MB 00:06 2024-08-20T00:53:07.745 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 1.4 MB/s | 8.2 MB 00:05 2024-08-20T00:53:08.349 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - BaseOS 1.2 MB/s | 8.2 MB 00:06 2024-08-20T00:53:15.701 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - AppStream 2.9 MB/s | 20 MB 00:06 2024-08-20T00:53:16.120 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - AppStream 3.0 MB/s | 20 MB 00:06 2024-08-20T00:53:17.380 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-08-20T00:53:24.047 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - CRB 1.4 MB/s | 6.6 MB 00:04 2024-08-20T00:53:24.922 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - CRB 1.3 MB/s | 6.6 MB 00:05 2024-08-20T00:53:25.095 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.6 MB 00:03 2024-08-20T00:53:25.523 INFO:teuthology.orchestra.run.smithi167.stdout:CentOS Stream 9 - Extras packages 82 kB/s | 18 kB 00:00 2024-08-20T00:53:26.569 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 24 MB/s | 22 MB 00:00 2024-08-20T00:53:26.583 INFO:teuthology.orchestra.run.smithi142.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 18 kB 00:00 2024-08-20T00:53:26.717 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 18 kB 00:00 2024-08-20T00:53:27.487 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 28 MB/s | 22 MB 00:00 2024-08-20T00:53:27.688 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 26 MB/s | 22 MB 00:00 2024-08-20T00:53:31.999 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-08-20T00:53:33.026 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 5.7 kB/s | 1.7 kB 00:00 2024-08-20T00:53:33.936 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-08-20T00:53:34.171 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:34.172 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:34.186 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T00:53:34.187 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T00:53:34.188 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T00:53:34.189 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T00:53:34.191 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T00:53:34.194 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T00:53:34.195 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T00:53:34.255 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T00:53:34.270 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T00:53:34.271 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T00:53:34.272 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T00:53:34.273 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T00:53:34.274 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T00:53:34.275 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T00:53:34.276 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T00:53:34.277 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T00:53:34.278 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T00:53:34.279 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T00:53:34.280 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T00:53:34.281 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout:Installing weak dependencies: 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout:======================================================================================= 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout:Install 148 Packages 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 2 Packages 2024-08-20T00:53:34.282 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:53:34.283 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 232 M 2024-08-20T00:53:34.283 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2024-08-20T00:53:35.001 INFO:teuthology.orchestra.run.smithi167.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 50 kB/s | 6.6 kB 00:00 2024-08-20T00:53:35.177 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:35.178 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:35.184 INFO:teuthology.orchestra.run.smithi167.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 4.8 MB/s | 901 kB 00:00 2024-08-20T00:53:35.189 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T00:53:35.190 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T00:53:35.191 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T00:53:35.192 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T00:53:35.193 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T00:53:35.196 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T00:53:35.197 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T00:53:35.257 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-08-20T00:53:35.260 INFO:teuthology.orchestra.run.smithi167.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 1.9 MB/s | 145 kB 00:00 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T00:53:35.274 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T00:53:35.275 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:35.276 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T00:53:35.277 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T00:53:35.278 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T00:53:35.279 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T00:53:35.280 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T00:53:35.281 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T00:53:35.282 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T00:53:35.283 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T00:53:35.284 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T00:53:35.285 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:======================================================================================= 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Install 148 Packages 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 2 Packages 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 232 M 2024-08-20T00:53:35.286 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-08-20T00:53:35.360 INFO:teuthology.orchestra.run.smithi167.stdout:(4/150): ceph-base-19.0.0-5872.gef427541.el9.x8 11 MB/s | 5.5 MB 00:00 2024-08-20T00:53:35.443 INFO:teuthology.orchestra.run.smithi167.stdout:(5/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 13 MB/s | 2.4 MB 00:00 2024-08-20T00:53:35.494 INFO:teuthology.orchestra.run.smithi167.stdout:(6/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 11 MB/s | 1.5 MB 00:00 2024-08-20T00:53:35.836 INFO:teuthology.orchestra.run.smithi167.stdout:(7/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 12 MB/s | 4.8 MB 00:00 2024-08-20T00:53:35.994 INFO:teuthology.orchestra.run.smithi142.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-08-20T00:53:36.153 INFO:teuthology.orchestra.run.smithi167.stdout:(8/150): ceph-common-19.0.0-5872.gef427541.el9. 17 MB/s | 22 MB 00:01 2024-08-20T00:53:36.220 INFO:teuthology.orchestra.run.smithi167.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 378 kB/s | 25 kB 00:00 2024-08-20T00:53:36.401 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:36.402 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-08-20T00:53:36.414 INFO:teuthology.orchestra.run.smithi138.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-08-20T00:53:36.415 INFO:teuthology.orchestra.run.smithi138.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-08-20T00:53:36.417 INFO:teuthology.orchestra.run.smithi138.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-08-20T00:53:36.418 INFO:teuthology.orchestra.run.smithi138.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-08-20T00:53:36.420 INFO:teuthology.orchestra.run.smithi138.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-08-20T00:53:36.422 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-08-20T00:53:36.423 INFO:teuthology.orchestra.run.smithi138.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-08-20T00:53:36.487 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:19.0.0-5872.gef427541.el9 ceph 6.6 k 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 901 k 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-5872.gef427541.el9 ceph 145 k 2024-08-20T00:53:36.509 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:19.0.0-5872.gef427541.el9 ceph 1.5 M 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 150 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 14 M 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 7.4 M 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 49 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 11 M 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:19.0.0-5872.gef427541.el9 ceph 50 M 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 285 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 772 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 225 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell x86_64 2:19.0.0-5872.gef427541.el9 ceph 27 k 2024-08-20T00:53:36.510 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 18 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 32 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 755 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:19.0.0-5872.gef427541.el9 ceph 126 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel x86_64 1:3.2.2-2.el9 appstream 4.4 M 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:19.0.0-5872.gef427541.el9 ceph 188 k 2024-08-20T00:53:36.511 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:19.0.0-5872.gef427541.el9 ceph 371 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 355 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:19.0.0-5872.gef427541.el9 ceph 116 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:19.0.0-5872.gef427541.el9 ceph 85 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:19.0.0-5872.gef427541.el9 ceph 170 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.5 M 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 3.2 M 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 22 M 2024-08-20T00:53:36.512 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 34 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:19.0.0-5872.gef427541.el9 ceph 2.4 M 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 284 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:19.0.0-5872.gef427541.el9 ceph 4.8 M 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:19.0.0-5872.gef427541.el9 ceph 17 M 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:19.0.0-5872.gef427541.el9 ceph-noarch 16 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:19.0.0-5872.gef427541.el9 ceph 25 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-08-20T00:53:36.513 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:19.0.0-5872.gef427541.el9 ceph 163 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:19.0.0-5872.gef427541.el9 ceph 508 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:19.0.0-5872.gef427541.el9 ceph 5.5 M 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-08-20T00:53:36.514 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.26-1.el9 appstream 38 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.26-1.el9 appstream 4.9 M 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-08-20T00:53:36.515 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:19.0.0-5872.gef427541.el9 ceph 45 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:19.0.0-5872.gef427541.el9 ceph 138 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.19-7.el9 appstream 245 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.33.0-1.el9 epel 220 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-08-20T00:53:36.516 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-08-20T00:53:36.517 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-08-20T00:53:36.518 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-08-20T00:53:36.519 INFO:teuthology.orchestra.run.smithi138.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout:======================================================================================= 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout:Install 148 Packages 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-08-20T00:53:36.520 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:53:36.521 INFO:teuthology.orchestra.run.smithi142.stdout:(2/150): ceph-base-19.0.0-5872.gef427541.el9.x8 6.9 MB/s | 5.5 MB 00:00 2024-08-20T00:53:36.521 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 232 M 2024-08-20T00:53:36.521 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-08-20T00:53:36.577 INFO:teuthology.orchestra.run.smithi142.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 2.1 MB/s | 145 kB 00:00 2024-08-20T00:53:36.595 INFO:teuthology.orchestra.run.smithi167.stdout:(10/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 16 MB/s | 17 MB 00:01 2024-08-20T00:53:36.728 INFO:teuthology.orchestra.run.smithi142.stdout:(4/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 1.2 MB/s | 901 kB 00:00 2024-08-20T00:53:36.728 INFO:teuthology.orchestra.run.smithi167.stdout:(11/150): ceph-radosgw-19.0.0-5872.gef427541.el 12 MB/s | 11 MB 00:00 2024-08-20T00:53:36.754 INFO:teuthology.orchestra.run.smithi167.stdout:(12/150): cephfs-mirror-19.0.0-5872.gef427541.e 1.4 MB/s | 225 kB 00:00 2024-08-20T00:53:36.769 INFO:teuthology.orchestra.run.smithi142.stdout:(5/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 12 MB/s | 2.4 MB 00:00 2024-08-20T00:53:36.779 INFO:teuthology.orchestra.run.smithi167.stdout:(13/150): cephfs-shell-19.0.0-5872.gef427541.el 542 kB/s | 27 kB 00:00 2024-08-20T00:53:36.804 INFO:teuthology.orchestra.run.smithi167.stdout:(14/150): libcephfs-devel-19.0.0-5872.gef427541 630 kB/s | 32 kB 00:00 2024-08-20T00:53:36.863 INFO:teuthology.orchestra.run.smithi167.stdout:(15/150): libcephfs2-19.0.0-5872.gef427541.el9. 8.8 MB/s | 755 kB 00:00 2024-08-20T00:53:36.888 INFO:teuthology.orchestra.run.smithi167.stdout:(16/150): libcephsqlite-19.0.0-5872.gef427541.e 1.9 MB/s | 163 kB 00:00 2024-08-20T00:53:36.913 INFO:teuthology.orchestra.run.smithi167.stdout:(17/150): librados-devel-19.0.0-5872.gef427541. 2.5 MB/s | 126 kB 00:00 2024-08-20T00:53:36.956 INFO:teuthology.orchestra.run.smithi167.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 7.4 MB/s | 508 kB 00:00 2024-08-20T00:53:37.039 INFO:teuthology.orchestra.run.smithi167.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 542 kB/s | 45 kB 00:00 2024-08-20T00:53:37.096 INFO:teuthology.orchestra.run.smithi138.stdout:(1/150): ceph-19.0.0-5872.gef427541.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-08-20T00:53:37.123 INFO:teuthology.orchestra.run.smithi167.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 1.6 MB/s | 138 kB 00:00 2024-08-20T00:53:37.136 INFO:teuthology.orchestra.run.smithi142.stdout:(6/150): ceph-mon-19.0.0-5872.gef427541.el9.x86 13 MB/s | 4.8 MB 00:00 2024-08-20T00:53:37.190 INFO:teuthology.orchestra.run.smithi167.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 2.8 MB/s | 188 kB 00:00 2024-08-20T00:53:37.265 INFO:teuthology.orchestra.run.smithi167.stdout:(22/150): librgw2-19.0.0-5872.gef427541.el9.x86 16 MB/s | 5.5 MB 00:00 2024-08-20T00:53:37.290 INFO:teuthology.orchestra.run.smithi167.stdout:(23/150): python3-rados-19.0.0-5872.gef427541.e 3.6 MB/s | 371 kB 00:00 2024-08-20T00:53:37.315 INFO:teuthology.orchestra.run.smithi167.stdout:(24/150): python3-rbd-19.0.0-5872.gef427541.el9 6.9 MB/s | 355 kB 00:00 2024-08-20T00:53:37.341 INFO:teuthology.orchestra.run.smithi167.stdout:(25/150): python3-rgw-19.0.0-5872.gef427541.el9 2.3 MB/s | 116 kB 00:00 2024-08-20T00:53:37.366 INFO:teuthology.orchestra.run.smithi167.stdout:(26/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.7 MB/s | 85 kB 00:00 2024-08-20T00:53:37.378 INFO:teuthology.orchestra.run.smithi142.stdout:(7/150): ceph-common-19.0.0-5872.gef427541.el9. 13 MB/s | 22 MB 00:01 2024-08-20T00:53:37.425 INFO:teuthology.orchestra.run.smithi167.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 2.9 MB/s | 170 kB 00:00 2024-08-20T00:53:37.496 INFO:teuthology.orchestra.run.smithi138.stdout:(2/150): ceph-fuse-19.0.0-5872.gef427541.el9.x8 2.2 MB/s | 901 kB 00:00 2024-08-20T00:53:37.508 INFO:teuthology.orchestra.run.smithi167.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 413 kB/s | 34 kB 00:00 2024-08-20T00:53:37.583 INFO:teuthology.orchestra.run.smithi167.stdout:(29/150): rbd-mirror-19.0.0-5872.gef427541.el9. 13 MB/s | 3.2 MB 00:00 2024-08-20T00:53:37.609 INFO:teuthology.orchestra.run.smithi167.stdout:(30/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 1.5 MB/s | 150 kB 00:00 2024-08-20T00:53:37.630 INFO:teuthology.orchestra.run.smithi138.stdout:(3/150): ceph-immutable-object-cache-19.0.0-587 1.1 MB/s | 145 kB 00:00 2024-08-20T00:53:38.212 INFO:teuthology.orchestra.run.smithi142.stdout:(8/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 1.0 MB/s | 1.5 MB 00:01 2024-08-20T00:53:38.276 INFO:teuthology.orchestra.run.smithi167.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-08-20T00:53:38.337 INFO:teuthology.orchestra.run.smithi142.stdout:(9/150): ceph-radosgw-19.0.0-5872.gef427541.el9 11 MB/s | 11 MB 00:00 2024-08-20T00:53:38.342 INFO:teuthology.orchestra.run.smithi167.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 4.2 MB/s | 284 kB 00:00 2024-08-20T00:53:38.362 INFO:teuthology.orchestra.run.smithi142.stdout:(10/150): ceph-selinux-19.0.0-5872.gef427541.el 167 kB/s | 25 kB 00:00 2024-08-20T00:53:38.443 INFO:teuthology.orchestra.run.smithi167.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 493 kB/s | 49 kB 00:00 2024-08-20T00:53:38.518 INFO:teuthology.orchestra.run.smithi167.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 219 kB/s | 16 kB 00:00 2024-08-20T00:53:38.585 INFO:teuthology.orchestra.run.smithi167.stdout:(35/150): ceph-volume-19.0.0-5872.gef427541.el9 4.2 MB/s | 285 kB 00:00 2024-08-20T00:53:38.654 INFO:teuthology.orchestra.run.smithi142.stdout:(11/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 11 MB/s | 17 MB 00:01 2024-08-20T00:53:38.677 INFO:teuthology.orchestra.run.smithi167.stdout:(36/150): cephadm-19.0.0-5872.gef427541.el9.noa 8.2 MB/s | 772 kB 00:00 2024-08-20T00:53:38.679 INFO:teuthology.orchestra.run.smithi142.stdout:(12/150): cephfs-mirror-19.0.0-5872.gef427541.e 709 kB/s | 225 kB 00:00 2024-08-20T00:53:38.713 INFO:teuthology.orchestra.run.smithi142.stdout:(13/150): cephfs-shell-19.0.0-5872.gef427541.el 462 kB/s | 27 kB 00:00 2024-08-20T00:53:38.738 INFO:teuthology.orchestra.run.smithi142.stdout:(14/150): libcephfs-devel-19.0.0-5872.gef427541 542 kB/s | 32 kB 00:00 2024-08-20T00:53:38.738 INFO:teuthology.orchestra.run.smithi138.stdout:(4/150): ceph-mds-19.0.0-5872.gef427541.el9.x86 2.1 MB/s | 2.4 MB 00:01 2024-08-20T00:53:38.772 INFO:teuthology.orchestra.run.smithi142.stdout:(15/150): libcephfs2-19.0.0-5872.gef427541.el9. 13 MB/s | 755 kB 00:00 2024-08-20T00:53:38.828 INFO:teuthology.orchestra.run.smithi167.stdout:(37/150): ceph-mgr-dashboard-19.0.0-5872.gef427 11 MB/s | 14 MB 00:01 2024-08-20T00:53:38.830 INFO:teuthology.orchestra.run.smithi142.stdout:(16/150): librados-devel-19.0.0-5872.gef427541. 2.1 MB/s | 126 kB 00:00 2024-08-20T00:53:38.847 INFO:teuthology.orchestra.run.smithi142.stdout:(17/150): libcephsqlite-19.0.0-5872.gef427541.e 1.5 MB/s | 163 kB 00:00 2024-08-20T00:53:38.853 INFO:teuthology.orchestra.run.smithi167.stdout:(38/150): cephfs-top-19.0.0-5872.gef427541.el9. 104 kB/s | 18 kB 00:00 2024-08-20T00:53:38.897 INFO:teuthology.orchestra.run.smithi142.stdout:(18/150): libradosstriper1-19.0.0-5872.gef42754 7.4 MB/s | 508 kB 00:00 2024-08-20T00:53:38.908 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:38.956 INFO:teuthology.orchestra.run.smithi142.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 775 kB/s | 45 kB 00:00 2024-08-20T00:53:38.986 INFO:teuthology.orchestra.run.smithi167.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 504 kB/s | 80 kB 00:00 2024-08-20T00:53:39.014 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:39.015 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:39.023 INFO:teuthology.orchestra.run.smithi142.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 2.0 MB/s | 138 kB 00:00 2024-08-20T00:53:39.042 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:39.058 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:39.081 INFO:teuthology.orchestra.run.smithi142.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 3.1 MB/s | 188 kB 00:00 2024-08-20T00:53:39.087 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:39.097 INFO:teuthology.orchestra.run.smithi138.stdout:(5/150): ceph-base-19.0.0-5872.gef427541.el9.x8 2.4 MB/s | 5.5 MB 00:02 2024-08-20T00:53:39.138 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:39.148 INFO:teuthology.orchestra.run.smithi142.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 5.4 MB/s | 371 kB 00:00 2024-08-20T00:53:39.160 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:39.170 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:39.194 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:39.224 INFO:teuthology.orchestra.run.smithi142.stdout:(23/150): python3-rbd-19.0.0-5872.gef427541.el9 4.6 MB/s | 355 kB 00:00 2024-08-20T00:53:39.230 INFO:teuthology.orchestra.run.smithi138.stdout:(6/150): ceph-mgr-19.0.0-5872.gef427541.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-08-20T00:53:39.242 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:39.266 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:39.274 INFO:teuthology.orchestra.run.smithi142.stdout:(24/150): python3-rgw-19.0.0-5872.gef427541.el9 2.3 MB/s | 116 kB 00:00 2024-08-20T00:53:39.332 INFO:teuthology.orchestra.run.smithi142.stdout:(25/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 1.4 MB/s | 85 kB 00:00 2024-08-20T00:53:39.401 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:39.427 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:39.599 INFO:teuthology.orchestra.run.smithi142.stdout:(26/150): rbd-mirror-19.0.0-5872.gef427541.el9. 12 MB/s | 3.2 MB 00:00 2024-08-20T00:53:39.650 INFO:teuthology.orchestra.run.smithi142.stdout:(27/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 3.3 MB/s | 170 kB 00:00 2024-08-20T00:53:39.709 INFO:teuthology.orchestra.run.smithi142.stdout:(28/150): ceph-grafana-dashboards-19.0.0-5872.g 585 kB/s | 34 kB 00:00 2024-08-20T00:53:39.728 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:39.753 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:39.759 INFO:teuthology.orchestra.run.smithi142.stdout:(29/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 2.9 MB/s | 150 kB 00:00 2024-08-20T00:53:39.789 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:39.812 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:39.831 INFO:teuthology.orchestra.run.smithi138.stdout:(7/150): ceph-common-19.0.0-5872.gef427541.el9. 7.2 MB/s | 22 MB 00:02 2024-08-20T00:53:39.915 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:39.936 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:39.950 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:39.971 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:39.988 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:40.007 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:40.007 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:40.025 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:40.068 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:40.085 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:40.183 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:40.187 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:40.689 INFO:teuthology.orchestra.run.smithi138.stdout:(8/150): ceph-radosgw-19.0.0-5872.gef427541.el9 13 MB/s | 11 MB 00:00 2024-08-20T00:53:40.740 INFO:teuthology.orchestra.run.smithi138.stdout:(9/150): ceph-selinux-19.0.0-5872.gef427541.el9 505 kB/s | 25 kB 00:00 2024-08-20T00:53:40.742 INFO:teuthology.orchestra.run.smithi142.stdout:(30/150): ceph-mgr-dashboard-19.0.0-5872.gef427 14 MB/s | 14 MB 00:00 2024-08-20T00:53:40.814 INFO:teuthology.orchestra.run.smithi138.stdout:(10/150): ceph-mon-19.0.0-5872.gef427541.el9.x8 2.8 MB/s | 4.8 MB 00:01 2024-08-20T00:53:40.928 INFO:teuthology.orchestra.run.smithi167.stdout:(40/150): libquadmath-11.5.0-2.el9.x86_64.rpm 98 kB/s | 191 kB 00:01 2024-08-20T00:53:40.982 INFO:teuthology.orchestra.run.smithi138.stdout:(11/150): cephfs-mirror-19.0.0-5872.gef427541.e 1.3 MB/s | 225 kB 00:00 2024-08-20T00:53:41.070 INFO:teuthology.orchestra.run.smithi167.stdout:(41/150): libgfortran-11.5.0-2.el9.x86_64.rpm 361 kB/s | 801 kB 00:02 2024-08-20T00:53:41.090 INFO:teuthology.orchestra.run.smithi138.stdout:(12/150): cephfs-shell-19.0.0-5872.gef427541.el 250 kB/s | 27 kB 00:00 2024-08-20T00:53:41.128 INFO:teuthology.orchestra.run.smithi167.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-08-20T00:53:41.199 INFO:teuthology.orchestra.run.smithi138.stdout:(13/150): libcephfs-devel-19.0.0-5872.gef427541 293 kB/s | 32 kB 00:00 2024-08-20T00:53:41.304 INFO:teuthology.orchestra.run.smithi167.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-08-20T00:53:41.317 INFO:teuthology.orchestra.run.smithi142.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 13 MB/s | 7.4 MB 00:00 2024-08-20T00:53:41.329 INFO:teuthology.orchestra.run.smithi167.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 531 kB/s | 106 kB 00:00 2024-08-20T00:53:41.355 INFO:teuthology.orchestra.run.smithi167.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-08-20T00:53:41.380 INFO:teuthology.orchestra.run.smithi167.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-08-20T00:53:41.410 INFO:teuthology.orchestra.run.smithi142.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 3.0 MB/s | 284 kB 00:00 2024-08-20T00:53:41.414 INFO:teuthology.orchestra.run.smithi167.stdout:(47/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-20T00:53:41.493 INFO:teuthology.orchestra.run.smithi142.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 592 kB/s | 49 kB 00:00 2024-08-20T00:53:41.533 INFO:teuthology.orchestra.run.smithi138.stdout:(14/150): libcephfs2-19.0.0-5872.gef427541.el9. 2.2 MB/s | 755 kB 00:00 2024-08-20T00:53:41.577 INFO:teuthology.orchestra.run.smithi142.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 197 kB/s | 16 kB 00:00 2024-08-20T00:53:41.589 INFO:teuthology.orchestra.run.smithi167.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-08-20T00:53:41.616 INFO:teuthology.orchestra.run.smithi138.stdout:(15/150): libcephsqlite-19.0.0-5872.gef427541.e 1.9 MB/s | 163 kB 00:00 2024-08-20T00:53:41.623 INFO:teuthology.orchestra.run.smithi167.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-08-20T00:53:41.648 INFO:teuthology.orchestra.run.smithi167.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-20T00:53:41.665 INFO:teuthology.orchestra.run.smithi167.stdout:(51/150): elfutils-libelf-devel-0.191-4.el9.x86 518 kB/s | 22 kB 00:00 2024-08-20T00:53:41.698 INFO:teuthology.orchestra.run.smithi167.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-08-20T00:53:41.699 INFO:teuthology.orchestra.run.smithi138.stdout:(16/150): librados-devel-19.0.0-5872.gef427541. 1.5 MB/s | 126 kB 00:00 2024-08-20T00:53:41.715 INFO:teuthology.orchestra.run.smithi167.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-08-20T00:53:41.749 INFO:teuthology.orchestra.run.smithi167.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 546 kB/s | 18 kB 00:00 2024-08-20T00:53:41.783 INFO:teuthology.orchestra.run.smithi167.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 335 kB/s | 11 kB 00:00 2024-08-20T00:53:41.833 INFO:teuthology.orchestra.run.smithi167.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-08-20T00:53:41.858 INFO:teuthology.orchestra.run.smithi167.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-08-20T00:53:41.883 INFO:teuthology.orchestra.run.smithi167.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-08-20T00:53:41.917 INFO:teuthology.orchestra.run.smithi167.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-08-20T00:53:41.942 INFO:teuthology.orchestra.run.smithi167.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 9.7 MB/s | 581 kB 00:00 2024-08-20T00:53:41.958 INFO:teuthology.orchestra.run.smithi138.stdout:(17/150): libradosstriper1-19.0.0-5872.gef42754 1.9 MB/s | 508 kB 00:00 2024-08-20T00:53:41.968 INFO:teuthology.orchestra.run.smithi167.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-08-20T00:53:41.993 INFO:teuthology.orchestra.run.smithi167.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 943 kB/s | 47 kB 00:00 2024-08-20T00:53:41.994 INFO:teuthology.orchestra.run.smithi142.stdout:(35/150): ceph-test-19.0.0-5872.gef427541.el9.x 14 MB/s | 50 MB 00:03 2024-08-20T00:53:42.024 INFO:teuthology.orchestra.run.smithi142.stdout:(36/150): ceph-volume-19.0.0-5872.gef427541.el9 637 kB/s | 285 kB 00:00 2024-08-20T00:53:42.026 INFO:teuthology.orchestra.run.smithi167.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-08-20T00:53:42.051 INFO:teuthology.orchestra.run.smithi167.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-08-20T00:53:42.052 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:42.081 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:42.092 INFO:teuthology.orchestra.run.smithi142.stdout:(37/150): librgw2-19.0.0-5872.gef427541.el9.x86 1.7 MB/s | 5.5 MB 00:03 2024-08-20T00:53:42.117 INFO:teuthology.orchestra.run.smithi142.stdout:(38/150): cephfs-top-19.0.0-5872.gef427541.el9. 199 kB/s | 18 kB 00:00 2024-08-20T00:53:42.132 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:42.153 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:42.155 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:42.163 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:42.184 INFO:teuthology.orchestra.run.smithi142.stdout:(39/150): cephadm-19.0.0-5872.gef427541.el9.noa 4.0 MB/s | 772 kB 00:00 2024-08-20T00:53:42.184 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:42.193 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:42.210 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:42.215 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:42.226 INFO:teuthology.orchestra.run.smithi142.stdout:(40/150): fuse-2.9.9-16.el9.x86_64.rpm 596 kB/s | 80 kB 00:00 2024-08-20T00:53:42.226 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:42.232 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:42.244 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:42.261 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://iad.mirror.rackspace.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 72.4.120.219) 2024-08-20T00:53:42.282 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:42.318 INFO:teuthology.orchestra.run.smithi142.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-08-20T00:53:42.326 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:42.371 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.382 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:42.392 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.418 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:42.468 INFO:teuthology.orchestra.run.smithi142.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-08-20T00:53:42.551 INFO:teuthology.orchestra.run.smithi142.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-08-20T00:53:42.552 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.579 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.635 INFO:teuthology.orchestra.run.smithi142.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-08-20T00:53:42.709 INFO:teuthology.orchestra.run.smithi138.stdout:(18/150): ceph-test-19.0.0-5872.gef427541.el9.x 26 MB/s | 50 MB 00:01 2024-08-20T00:53:42.759 INFO:teuthology.orchestra.run.smithi138.stdout:(19/150): python3-ceph-argparse-19.0.0-5872.gef 926 kB/s | 45 kB 00:00 2024-08-20T00:53:42.769 INFO:teuthology.orchestra.run.smithi142.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 989 kB/s | 125 kB 00:00 2024-08-20T00:53:42.809 INFO:teuthology.orchestra.run.smithi138.stdout:(20/150): python3-ceph-common-19.0.0-5872.gef42 2.7 MB/s | 138 kB 00:00 2024-08-20T00:53:42.859 INFO:teuthology.orchestra.run.smithi138.stdout:(21/150): python3-cephfs-19.0.0-5872.gef427541. 3.7 MB/s | 188 kB 00:00 2024-08-20T00:53:42.877 INFO:teuthology.orchestra.run.smithi142.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 2.0 MB/s | 215 kB 00:00 2024-08-20T00:53:42.884 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.918 INFO:teuthology.orchestra.run.smithi138.stdout:(22/150): python3-rados-19.0.0-5872.gef427541.e 6.2 MB/s | 371 kB 00:00 2024-08-20T00:53:42.944 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:42.959 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:42.968 INFO:teuthology.orchestra.run.smithi138.stdout:(23/150): python3-rbd-19.0.0-5872.gef427541.el9 7.0 MB/s | 355 kB 00:00 2024-08-20T00:53:42.988 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:43.018 INFO:teuthology.orchestra.run.smithi138.stdout:(24/150): python3-rgw-19.0.0-5872.gef427541.el9 2.3 MB/s | 116 kB 00:00 2024-08-20T00:53:43.044 INFO:teuthology.orchestra.run.smithi142.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-08-20T00:53:43.061 INFO:teuthology.orchestra.run.smithi138.stdout:(25/150): rbd-fuse-19.0.0-5872.gef427541.el9.x8 2.0 MB/s | 85 kB 00:00 2024-08-20T00:53:43.102 INFO:teuthology.orchestra.run.smithi167.stdout:(65/150): ceph-test-19.0.0-5872.gef427541.el9.x 7.3 MB/s | 50 MB 00:06 2024-08-20T00:53:43.102 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:43.103 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:43.103 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:43.136 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.136 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.141 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:43.147 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.185 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.208 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.211 INFO:teuthology.orchestra.run.smithi138.stdout:(26/150): rbd-mirror-19.0.0-5872.gef427541.el9. 21 MB/s | 3.2 MB 00:00 2024-08-20T00:53:43.217 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.222 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.259 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.320 INFO:teuthology.orchestra.run.smithi138.stdout:(27/150): librgw2-19.0.0-5872.gef427541.el9.x86 4.0 MB/s | 5.5 MB 00:01 2024-08-20T00:53:43.328 INFO:teuthology.orchestra.run.smithi142.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-08-20T00:53:43.345 INFO:teuthology.orchestra.run.smithi138.stdout:(28/150): rbd-nbd-19.0.0-5872.gef427541.el9.x86 1.2 MB/s | 170 kB 00:00 2024-08-20T00:53:43.368 INFO:teuthology.orchestra.run.smithi167.stdout:(66/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-08-20T00:53:43.370 INFO:teuthology.orchestra.run.smithi138.stdout:(29/150): ceph-grafana-dashboards-19.0.0-5872.g 687 kB/s | 34 kB 00:00 2024-08-20T00:53:43.386 INFO:teuthology.orchestra.run.smithi142.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-08-20T00:53:43.390 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.396 INFO:teuthology.orchestra.run.smithi138.stdout:(30/150): ceph-mgr-cephadm-19.0.0-5872.gef42754 3.0 MB/s | 150 kB 00:00 2024-08-20T00:53:43.428 INFO:teuthology.orchestra.run.smithi142.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 519 kB/s | 22 kB 00:00 2024-08-20T00:53:43.429 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:43.429 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:43.450 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:43.493 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.512 INFO:teuthology.orchestra.run.smithi142.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-08-20T00:53:43.562 INFO:teuthology.orchestra.run.smithi142.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 923 kB/s | 33 kB 00:00 2024-08-20T00:53:43.599 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:43.635 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:43.654 INFO:teuthology.orchestra.run.smithi138.stdout:(31/150): ceph-mgr-diskprediction-local-19.0.0- 29 MB/s | 7.4 MB 00:00 2024-08-20T00:53:43.704 INFO:teuthology.orchestra.run.smithi138.stdout:(32/150): ceph-mgr-modules-core-19.0.0-5872.gef 5.6 MB/s | 284 kB 00:00 2024-08-20T00:53:43.755 INFO:teuthology.orchestra.run.smithi138.stdout:(33/150): ceph-mgr-rook-19.0.0-5872.gef427541.e 991 kB/s | 49 kB 00:00 2024-08-20T00:53:43.764 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:43.771 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:43.795 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.803 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.805 INFO:teuthology.orchestra.run.smithi138.stdout:(34/150): ceph-prometheus-alerts-19.0.0-5872.ge 328 kB/s | 16 kB 00:00 2024-08-20T00:53:43.827 INFO:teuthology.orchestra.run.smithi167.stdout:(67/150): openblas-0.3.26-1.el9.x86_64.rpm 21 kB/s | 38 kB 00:01 2024-08-20T00:53:43.828 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:43.849 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.855 INFO:teuthology.orchestra.run.smithi138.stdout:(35/150): ceph-volume-19.0.0-5872.gef427541.el9 5.6 MB/s | 285 kB 00:00 2024-08-20T00:53:43.868 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.875 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.879 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:43.901 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:43.906 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:43.908 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:43.910 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:43.914 INFO:teuthology.orchestra.run.smithi138.stdout:(36/150): cephadm-19.0.0-5872.gef427541.el9.noa 13 MB/s | 772 kB 00:00 2024-08-20T00:53:43.924 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.931 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:43.940 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:43.964 INFO:teuthology.orchestra.run.smithi138.stdout:(37/150): cephfs-top-19.0.0-5872.gef427541.el9. 364 kB/s | 18 kB 00:00 2024-08-20T00:53:43.967 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:43.970 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.011 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.028 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:44.044 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:44.070 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.078 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.125 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.198 INFO:teuthology.orchestra.run.smithi138.stdout:(38/150): ceph-osd-19.0.0-5872.gef427541.el9.x8 3.5 MB/s | 17 MB 00:04 2024-08-20T00:53:44.223 INFO:teuthology.orchestra.run.smithi138.stdout:(39/150): fuse-2.9.9-16.el9.x86_64.rpm 309 kB/s | 80 kB 00:00 2024-08-20T00:53:44.255 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.258 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.304 INFO:teuthology.orchestra.run.smithi142.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.0 MB/s | 3.0 MB 00:00 2024-08-20T00:53:44.361 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.362 INFO:teuthology.orchestra.run.smithi142.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 422 kB/s | 18 kB 00:00 2024-08-20T00:53:44.371 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:44.405 INFO:teuthology.orchestra.run.smithi142.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-08-20T00:53:44.471 INFO:teuthology.orchestra.run.smithi142.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-08-20T00:53:44.513 INFO:teuthology.orchestra.run.smithi142.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-08-20T00:53:44.522 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.532 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.705 INFO:teuthology.orchestra.run.smithi142.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-08-20T00:53:44.857 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.867 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:44.889 INFO:teuthology.orchestra.run.smithi142.stdout:(59/150): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-08-20T00:53:44.893 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:44.902 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:44.914 INFO:teuthology.orchestra.run.smithi142.stdout:(60/150): libquadmath-11.5.0-2.el9.x86_64.rpm 70 kB/s | 191 kB 00:02 2024-08-20T00:53:44.939 INFO:teuthology.orchestra.run.smithi142.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-08-20T00:53:44.951 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:44.960 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:44.981 INFO:teuthology.orchestra.run.smithi142.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 714 kB/s | 47 kB 00:00 2024-08-20T00:53:45.048 INFO:teuthology.orchestra.run.smithi142.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-08-20T00:53:45.073 INFO:teuthology.orchestra.run.smithi142.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-08-20T00:53:45.074 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:45.082 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.089 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:45.089 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:45.091 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.105 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:45.145 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:45.160 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:45.237 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:45.249 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:45.252 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:45.253 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:45.273 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:45.288 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:45.331 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.346 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.458 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.473 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:45.518 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:45.533 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:45.588 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:45.593 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirror.facebook.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 163.114.132.32) 2024-08-20T00:53:45.615 INFO:teuthology.orchestra.run.smithi142.stdout:(65/150): libgfortran-11.5.0-2.el9.x86_64.rpm 229 kB/s | 801 kB 00:03 2024-08-20T00:53:45.632 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.636 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.648 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:45.649 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:45.711 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libquadmath-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.721 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] libgfortran-11.5.0-2.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.5.0-2.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.844 INFO:teuthology.orchestra.run.smithi167.stdout:(68/150): postgresql-contrib-13.16-1.el9.x86_64 451 kB/s | 910 kB 00:02 2024-08-20T00:53:45.866 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.887 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:45.949 INFO:teuthology.orchestra.run.smithi138.stdout:(40/150): ceph-mgr-dashboard-19.0.0-5872.gef427 5.5 MB/s | 14 MB 00:02 2024-08-20T00:53:46.207 INFO:teuthology.orchestra.run.smithi138.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 982 kB/s | 253 kB 00:00 2024-08-20T00:53:46.224 INFO:teuthology.orchestra.run.smithi142.stdout:(66/150): openblas-0.3.26-1.el9.x86_64.rpm 33 kB/s | 38 kB 00:01 2024-08-20T00:53:46.300 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:46.332 INFO:teuthology.orchestra.run.smithi138.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 9.9 MB/s | 1.2 MB 00:00 2024-08-20T00:53:46.356 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:46.388 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:46.391 INFO:teuthology.orchestra.run.smithi138.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-08-20T00:53:46.436 INFO:teuthology.orchestra.run.smithi167.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 530 kB/s | 1.6 MB 00:03 2024-08-20T00:53:46.450 INFO:teuthology.orchestra.run.smithi138.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-08-20T00:53:46.457 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:46.479 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:46.479 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:46.508 INFO:teuthology.orchestra.run.smithi138.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-08-20T00:53:46.532 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:46.566 INFO:teuthology.orchestra.run.smithi138.stdout:(46/150): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-08-20T00:53:46.593 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:46.622 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:46.633 INFO:teuthology.orchestra.run.smithi138.stdout:(47/150): libquadmath-11.5.0-2.el9.x86_64.rpm 79 kB/s | 191 kB 00:02 2024-08-20T00:53:46.652 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:46.682 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:46.751 INFO:teuthology.orchestra.run.smithi138.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-08-20T00:53:46.799 INFO:teuthology.orchestra.run.smithi142.stdout:(67/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-08-20T00:53:46.801 INFO:teuthology.orchestra.run.smithi138.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-08-20T00:53:46.802 INFO:teuthology.orchestra.run.smithi167.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 143 kB/s | 137 kB 00:00 2024-08-20T00:53:46.815 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:46.843 INFO:teuthology.orchestra.run.smithi138.stdout:(50/150): elfutils-libelf-devel-0.191-4.el9.x86 524 kB/s | 22 kB 00:00 2024-08-20T00:53:46.844 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:46.859 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:46.876 INFO:teuthology.orchestra.run.smithi138.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-08-20T00:53:46.887 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:46.901 INFO:teuthology.orchestra.run.smithi138.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-08-20T00:53:46.917 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:46.927 INFO:teuthology.orchestra.run.smithi138.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 648 kB/s | 33 kB 00:00 2024-08-20T00:53:46.934 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.siena.edu/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 199.223.246.113) 2024-08-20T00:53:46.961 INFO:teuthology.orchestra.run.smithi138.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 554 kB/s | 18 kB 00:00 2024-08-20T00:53:46.967 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:46.994 INFO:teuthology.orchestra.run.smithi138.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 336 kB/s | 11 kB 00:00 2024-08-20T00:53:46.997 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.003 INFO:teuthology.orchestra.run.smithi167.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 5.1 MB/s | 1.0 MB 00:00 2024-08-20T00:53:47.053 INFO:teuthology.orchestra.run.smithi167.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-08-20T00:53:47.078 INFO:teuthology.orchestra.run.smithi138.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-08-20T00:53:47.128 INFO:teuthology.orchestra.run.smithi138.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-08-20T00:53:47.154 INFO:teuthology.orchestra.run.smithi138.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-08-20T00:53:47.204 INFO:teuthology.orchestra.run.smithi138.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 662 kB 00:00 2024-08-20T00:53:47.220 INFO:teuthology.orchestra.run.smithi167.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 36 MB/s | 6.0 MB 00:00 2024-08-20T00:53:47.238 INFO:teuthology.orchestra.run.smithi138.stdout:(60/150): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-08-20T00:53:47.253 INFO:teuthology.orchestra.run.smithi167.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-08-20T00:53:47.263 INFO:teuthology.orchestra.run.smithi138.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-08-20T00:53:47.275 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.288 INFO:teuthology.orchestra.run.smithi138.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 954 kB/s | 47 kB 00:00 2024-08-20T00:53:47.297 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.313 INFO:teuthology.orchestra.run.smithi138.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-08-20T00:53:47.339 INFO:teuthology.orchestra.run.smithi138.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-08-20T00:53:47.339 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.360 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.361 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.382 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.448 INFO:teuthology.orchestra.run.smithi138.stdout:(65/150): libgfortran-11.5.0-2.el9.x86_64.rpm 247 kB/s | 801 kB 00:03 2024-08-20T00:53:47.448 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.452 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.484 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.525 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.554 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.561 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.570 INFO:teuthology.orchestra.run.smithi167.stdout:(75/150): openblas-openmp-0.3.26-1.el9.x86_64.r 918 kB/s | 4.9 MB 00:05 2024-08-20T00:53:47.591 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.610 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.621 INFO:teuthology.orchestra.run.smithi167.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-08-20T00:53:47.622 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.671 INFO:teuthology.orchestra.run.smithi167.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-08-20T00:53:47.713 INFO:teuthology.orchestra.run.smithi167.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-08-20T00:53:47.714 INFO:teuthology.orchestra.run.smithi138.stdout:(66/150): openssl-devel-3.2.2-2.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-08-20T00:53:47.744 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.747 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.764 INFO:teuthology.orchestra.run.smithi167.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-20T00:53:47.778 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.783 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.799 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:47.805 INFO:teuthology.orchestra.run.smithi167.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 841 kB/s | 35 kB 00:00 2024-08-20T00:53:47.830 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:47.860 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.889 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:47.905 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:47.918 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.923 INFO:teuthology.orchestra.run.smithi167.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-08-20T00:53:47.943 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:47.947 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:47.973 INFO:teuthology.orchestra.run.smithi167.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-08-20T00:53:48.023 INFO:teuthology.orchestra.run.smithi167.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-08-20T00:53:48.033 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:48.056 INFO:teuthology.orchestra.run.smithi167.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2024-08-20T00:53:48.098 INFO:teuthology.orchestra.run.smithi167.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-20T00:53:48.149 INFO:teuthology.orchestra.run.smithi167.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-08-20T00:53:48.201 INFO:teuthology.orchestra.run.smithi167.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-08-20T00:53:48.239 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:48.249 INFO:teuthology.orchestra.run.smithi167.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-08-20T00:53:48.284 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:48.291 INFO:teuthology.orchestra.run.smithi167.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-20T00:53:48.303 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.333 INFO:teuthology.orchestra.run.smithi167.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-08-20T00:53:48.358 INFO:teuthology.orchestra.run.smithi167.stdout:(91/150): python3-devel-3.9.19-7.el9.x86_64.rpm 221 kB/s | 245 kB 00:01 2024-08-20T00:53:48.366 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:48.416 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.457 INFO:teuthology.orchestra.run.smithi142.stdout:(68/150): postgresql-13.16-1.el9.x86_64.rpm 727 kB/s | 1.6 MB 00:02 2024-08-20T00:53:48.462 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.467 INFO:teuthology.orchestra.run.smithi167.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 386 kB/s | 42 kB 00:00 2024-08-20T00:53:48.474 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.474 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.489 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.494 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.529 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.546 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.574 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.621 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.634 INFO:teuthology.orchestra.run.smithi167.stdout:(93/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 54 MB/s | 16 MB 00:00 2024-08-20T00:53:48.640 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:48.655 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.659 INFO:teuthology.orchestra.run.smithi167.stdout:(94/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 230 kB/s | 44 kB 00:00 2024-08-20T00:53:48.671 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] openblas-openmp-0.3.26-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/openblas-openmp-0.3.26-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.684 INFO:teuthology.orchestra.run.smithi167.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-20T00:53:48.699 INFO:teuthology.orchestra.run.smithi142.stdout:(69/150): postgresql-contrib-13.16-1.el9.x86_64 479 kB/s | 910 kB 00:01 2024-08-20T00:53:48.710 INFO:teuthology.orchestra.run.smithi167.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-08-20T00:53:48.715 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.731 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:48.794 INFO:teuthology.orchestra.run.smithi167.stdout:(97/150): postgresql-server-13.16-1.el9.x86_64. 2.5 MB/s | 5.8 MB 00:02 2024-08-20T00:53:48.818 INFO:teuthology.orchestra.run.smithi167.stdout:(98/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 354 kB/s | 47 kB 00:00 2024-08-20T00:53:48.861 INFO:teuthology.orchestra.run.smithi167.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-08-20T00:53:48.886 INFO:teuthology.orchestra.run.smithi167.stdout:(100/150): socat-1.7.4.1-6.el9.x86_64.rpm 1.7 MB/s | 304 kB 00:00 2024-08-20T00:53:48.886 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:48.911 INFO:teuthology.orchestra.run.smithi167.stdout:(101/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-08-20T00:53:48.911 INFO:teuthology.orchestra.run.smithi167.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:48.936 INFO:teuthology.orchestra.run.smithi167.stdout:(102/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-08-20T00:53:49.095 INFO:teuthology.orchestra.run.smithi167.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-08-20T00:53:49.145 INFO:teuthology.orchestra.run.smithi167.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-20T00:53:49.179 INFO:teuthology.orchestra.run.smithi167.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 12 MB/s | 309 kB 00:00 2024-08-20T00:53:49.204 INFO:teuthology.orchestra.run.smithi167.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 782 kB/s | 19 kB 00:00 2024-08-20T00:53:49.215 INFO:teuthology.orchestra.run.smithi138.stdout:(67/150): openblas-0.3.26-1.el9.x86_64.rpm 20 kB/s | 38 kB 00:01 2024-08-20T00:53:49.216 INFO:teuthology.orchestra.run.smithi142.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 181 kB/s | 137 kB 00:00 2024-08-20T00:53:49.293 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:49.313 INFO:teuthology.orchestra.run.smithi167.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-20T00:53:49.338 INFO:teuthology.orchestra.run.smithi167.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-20T00:53:49.355 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:49.364 INFO:teuthology.orchestra.run.smithi167.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-20T00:53:49.380 INFO:teuthology.orchestra.run.smithi167.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 4.0 MB/s | 67 kB 00:00 2024-08-20T00:53:49.399 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:49.414 INFO:teuthology.orchestra.run.smithi167.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-08-20T00:53:49.433 INFO:teuthology.orchestra.run.smithi142.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-08-20T00:53:49.456 INFO:teuthology.orchestra.run.smithi167.stdout:(112/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-08-20T00:53:49.490 INFO:teuthology.orchestra.run.smithi167.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-08-20T00:53:49.491 INFO:teuthology.orchestra.run.smithi142.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-08-20T00:53:49.506 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 216.31.2.234) 2024-08-20T00:53:49.515 INFO:teuthology.orchestra.run.smithi167.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T00:53:49.540 INFO:teuthology.orchestra.run.smithi167.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T00:53:49.557 INFO:teuthology.orchestra.run.smithi167.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 846 kB/s | 14 kB 00:00 2024-08-20T00:53:49.582 INFO:teuthology.orchestra.run.smithi167.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-08-20T00:53:49.616 INFO:teuthology.orchestra.run.smithi167.stdout:(118/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-08-20T00:53:49.637 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:49.641 INFO:teuthology.orchestra.run.smithi167.stdout:(119/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 116 kB/s | 96 kB 00:00 2024-08-20T00:53:49.683 INFO:teuthology.orchestra.run.smithi167.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-08-20T00:53:49.695 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-chi.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 64.50.236.52) 2024-08-20T00:53:49.717 INFO:teuthology.orchestra.run.smithi167.stdout:(121/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.0 MB/s | 303 kB 00:00 2024-08-20T00:53:49.750 INFO:teuthology.orchestra.run.smithi167.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-08-20T00:53:49.776 INFO:teuthology.orchestra.run.smithi167.stdout:(123/150): python3-google-auth-2.33.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-08-20T00:53:49.793 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:49.801 INFO:teuthology.orchestra.run.smithi167.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-08-20T00:53:49.826 INFO:teuthology.orchestra.run.smithi167.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-20T00:53:49.850 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftpmirror.your.org/pub/CentOS-Stream-All/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 163.237.219.7) 2024-08-20T00:53:49.851 INFO:teuthology.orchestra.run.smithi167.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-08-20T00:53:49.877 INFO:teuthology.orchestra.run.smithi167.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-08-20T00:53:49.902 INFO:teuthology.orchestra.run.smithi167.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-08-20T00:53:49.927 INFO:teuthology.orchestra.run.smithi167.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-20T00:53:49.953 INFO:teuthology.orchestra.run.smithi167.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-08-20T00:53:49.978 INFO:teuthology.orchestra.run.smithi167.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-20T00:53:50.011 INFO:teuthology.orchestra.run.smithi167.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-08-20T00:53:50.054 INFO:teuthology.orchestra.run.smithi167.stdout:(133/150): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-08-20T00:53:50.079 INFO:teuthology.orchestra.run.smithi167.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-08-20T00:53:50.104 INFO:teuthology.orchestra.run.smithi167.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-20T00:53:50.130 INFO:teuthology.orchestra.run.smithi167.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-08-20T00:53:50.155 INFO:teuthology.orchestra.run.smithi167.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 608 kB/s | 31 kB 00:00 2024-08-20T00:53:50.188 INFO:teuthology.orchestra.run.smithi167.stdout:(138/150): python3-kubernetes-26.1.0-2.el9.noar 3.3 MB/s | 1.0 MB 00:00 2024-08-20T00:53:50.190 INFO:teuthology.orchestra.run.smithi138.stdout:(68/150): openblas-openmp-0.3.26-1.el9.x86_64.r 1.7 MB/s | 4.9 MB 00:02 2024-08-20T00:53:50.191 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:50.212 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:50.213 INFO:teuthology.orchestra.run.smithi167.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-08-20T00:53:50.234 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:50.239 INFO:teuthology.orchestra.run.smithi167.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-08-20T00:53:50.264 INFO:teuthology.orchestra.run.smithi167.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 710 kB/s | 36 kB 00:00 2024-08-20T00:53:50.289 INFO:teuthology.orchestra.run.smithi167.stdout:(142/150): python3-typing-extensions-4.12.2-2.e 986 kB/s | 74 kB 00:00 2024-08-20T00:53:50.315 INFO:teuthology.orchestra.run.smithi167.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-08-20T00:53:50.340 INFO:teuthology.orchestra.run.smithi167.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-08-20T00:53:50.353 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:50.365 INFO:teuthology.orchestra.run.smithi167.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-08-20T00:53:50.390 INFO:teuthology.orchestra.run.smithi167.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-08-20T00:53:50.396 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 140.211.166.134) 2024-08-20T00:53:50.416 INFO:teuthology.orchestra.run.smithi167.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-08-20T00:53:50.466 INFO:teuthology.orchestra.run.smithi167.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-08-20T00:53:50.475 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:50.486 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:50.567 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:50.643 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:50.675 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-private-libs-13.16-1.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-private-libs-13.16-1.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:50.737 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-contrib-13.16-1.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/AppStream/x86_64/os/Packages/postgresql-contrib-13.16-1.el9.x86_64.rpm (IP: 172.65.90.22) 2024-08-20T00:53:51.048 INFO:teuthology.orchestra.run.smithi138.stdout:(69/150): postgresql-13.16-1.el9.x86_64.rpm 489 kB/s | 1.6 MB 00:03 2024-08-20T00:53:51.071 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:51.092 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] postgresql-server-13.16-1.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/postgresql-server-13.16-1.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:51.117 INFO:teuthology.orchestra.run.smithi142.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-08-20T00:53:51.150 INFO:teuthology.orchestra.run.smithi142.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-08-20T00:53:51.166 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:51.181 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:51.540 INFO:teuthology.orchestra.run.smithi138.stdout:(70/150): postgresql-private-libs-13.16-1.el9.x 102 kB/s | 137 kB 00:01 2024-08-20T00:53:51.708 INFO:teuthology.orchestra.run.smithi167.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 2.6 MB/s | 3.5 MB 00:01 2024-08-20T00:53:51.732 INFO:teuthology.orchestra.run.smithi138.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 5.3 MB/s | 1.0 MB 00:00 2024-08-20T00:53:51.766 INFO:teuthology.orchestra.run.smithi167.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 2.4 MB/s | 3.2 MB 00:01 2024-08-20T00:53:51.775 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2024-08-20T00:53:51.775 INFO:teuthology.orchestra.run.smithi167.stdout:Total 13 MB/s | 232 MB 00:17 2024-08-20T00:53:51.801 INFO:teuthology.orchestra.run.smithi138.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-08-20T00:53:51.974 INFO:teuthology.orchestra.run.smithi138.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 35 MB/s | 6.0 MB 00:00 2024-08-20T00:53:52.024 INFO:teuthology.orchestra.run.smithi138.stdout:(74/150): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-08-20T00:53:52.046 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:52.067 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] python3-devel-3.9.19-7.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/python3-devel-3.9.19-7.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:52.084 INFO:teuthology.orchestra.run.smithi142.stdout:(75/150): python3-devel-3.9.19-7.el9.x86_64.rpm 262 kB/s | 245 kB 00:00 2024-08-20T00:53:52.159 INFO:teuthology.orchestra.run.smithi142.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 3.3 MB/s | 248 kB 00:00 2024-08-20T00:53:52.201 INFO:teuthology.orchestra.run.smithi142.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 653 kB/s | 26 kB 00:00 2024-08-20T00:53:52.243 INFO:teuthology.orchestra.run.smithi142.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 461 kB/s | 19 kB 00:00 2024-08-20T00:53:52.285 INFO:teuthology.orchestra.run.smithi142.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-08-20T00:53:52.327 INFO:teuthology.orchestra.run.smithi142.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 836 kB/s | 35 kB 00:00 2024-08-20T00:53:52.419 INFO:teuthology.orchestra.run.smithi142.stdout:(81/150): openblas-openmp-0.3.26-1.el9.x86_64.r 690 kB/s | 4.9 MB 00:07 2024-08-20T00:53:52.517 INFO:teuthology.orchestra.run.smithi138.stdout:(75/150): postgresql-contrib-13.16-1.el9.x86_64 276 kB/s | 910 kB 00:03 2024-08-20T00:53:52.536 INFO:teuthology.orchestra.run.smithi142.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-08-20T00:53:52.575 INFO:teuthology.orchestra.run.smithi138.stdout:(76/150): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-08-20T00:53:52.594 INFO:teuthology.orchestra.run.smithi142.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-08-20T00:53:52.617 INFO:teuthology.orchestra.run.smithi138.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 615 kB/s | 26 kB 00:00 2024-08-20T00:53:52.636 INFO:teuthology.orchestra.run.smithi142.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-08-20T00:53:52.678 INFO:teuthology.orchestra.run.smithi142.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-08-20T00:53:52.684 INFO:teuthology.orchestra.run.smithi138.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 288 kB/s | 19 kB 00:00 2024-08-20T00:53:52.720 INFO:teuthology.orchestra.run.smithi142.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-08-20T00:53:52.734 INFO:teuthology.orchestra.run.smithi138.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-08-20T00:53:52.762 INFO:teuthology.orchestra.run.smithi142.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.7 MB/s | 159 kB 00:00 2024-08-20T00:53:52.776 INFO:teuthology.orchestra.run.smithi138.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 838 kB/s | 35 kB 00:00 2024-08-20T00:53:52.862 INFO:teuthology.orchestra.run.smithi142.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-08-20T00:53:52.890 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2024-08-20T00:53:52.893 INFO:teuthology.orchestra.run.smithi138.stdout:(81/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 44 MB/s | 5.1 MB 00:00 2024-08-20T00:53:52.904 INFO:teuthology.orchestra.run.smithi142.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-08-20T00:53:52.918 INFO:teuthology.orchestra.run.smithi138.stdout:(82/150): python3-devel-3.9.19-7.el9.x86_64.rpm 274 kB/s | 245 kB 00:00 2024-08-20T00:53:52.944 INFO:teuthology.orchestra.run.smithi138.stdout:(83/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-08-20T00:53:52.946 INFO:teuthology.orchestra.run.smithi142.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.3 MB/s | 54 kB 00:00 2024-08-20T00:53:52.994 INFO:teuthology.orchestra.run.smithi138.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-08-20T00:53:53.019 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2024-08-20T00:53:53.019 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2024-08-20T00:53:53.044 INFO:teuthology.orchestra.run.smithi138.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 850 kB/s | 42 kB 00:00 2024-08-20T00:53:53.103 INFO:teuthology.orchestra.run.smithi138.stdout:(86/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-08-20T00:53:53.137 INFO:teuthology.orchestra.run.smithi138.stdout:(87/150): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-08-20T00:53:53.162 INFO:teuthology.orchestra.run.smithi138.stdout:(88/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-08-20T00:53:53.187 INFO:teuthology.orchestra.run.smithi138.stdout:(89/150): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-08-20T00:53:53.212 INFO:teuthology.orchestra.run.smithi138.stdout:(90/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-08-20T00:53:53.238 INFO:teuthology.orchestra.run.smithi138.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-08-20T00:53:53.280 INFO:teuthology.orchestra.run.smithi138.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-20T00:53:53.322 INFO:teuthology.orchestra.run.smithi138.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-20T00:53:53.372 INFO:teuthology.orchestra.run.smithi138.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-08-20T00:53:53.414 INFO:teuthology.orchestra.run.smithi138.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 870 kB/s | 36 kB 00:00 2024-08-20T00:53:53.447 INFO:teuthology.orchestra.run.smithi138.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-08-20T00:53:53.498 INFO:teuthology.orchestra.run.smithi138.stdout:(97/150): socat-1.7.4.1-6.el9.x86_64.rpm 6.0 MB/s | 304 kB 00:00 2024-08-20T00:53:53.673 INFO:teuthology.orchestra.run.smithi138.stdout:(98/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-08-20T00:53:53.698 INFO:teuthology.orchestra.run.smithi138.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 289 kB/s | 58 kB 00:00 2024-08-20T00:53:53.699 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:53.720 INFO:teuthology.orchestra.run.smithi138.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 206.82.17.213) 2024-08-20T00:53:53.740 INFO:teuthology.orchestra.run.smithi138.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-08-20T00:53:53.746 INFO:teuthology.orchestra.run.smithi142.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.6 MB/s | 5.1 MB 00:01 2024-08-20T00:53:53.788 INFO:teuthology.orchestra.run.smithi142.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-08-20T00:53:53.791 INFO:teuthology.orchestra.run.smithi138.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-08-20T00:53:53.830 INFO:teuthology.orchestra.run.smithi142.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-08-20T00:53:53.922 INFO:teuthology.orchestra.run.smithi142.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-08-20T00:53:53.997 INFO:teuthology.orchestra.run.smithi142.stdout:(95/150): postgresql-server-13.16-1.el9.x86_64. 1.1 MB/s | 5.8 MB 00:05 2024-08-20T00:53:54.022 INFO:teuthology.orchestra.run.smithi142.stdout:(96/150): qatlib-service-24.02.0-1.el9.x86_64.r 358 kB/s | 36 kB 00:00 2024-08-20T00:53:54.089 INFO:teuthology.orchestra.run.smithi142.stdout:(97/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 520 kB/s | 47 kB 00:00 2024-08-20T00:53:54.116 INFO:teuthology.orchestra.run.smithi138.stdout:(102/150): postgresql-server-13.16-1.el9.x86_64 1.9 MB/s | 5.8 MB 00:03 2024-08-20T00:53:54.141 INFO:teuthology.orchestra.run.smithi138.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 64 kB/s | 22 kB 00:00 2024-08-20T00:53:54.148 INFO:teuthology.orchestra.run.smithi142.stdout:(98/150): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-08-20T00:53:54.173 INFO:teuthology.orchestra.run.smithi142.stdout:(99/150): uuid-1.6.2-55.el9.x86_64.rpm 693 kB/s | 58 kB 00:00 2024-08-20T00:53:54.193 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for http://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:54.200 INFO:teuthology.orchestra.run.smithi138.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-08-20T00:53:54.215 INFO:teuthology.orchestra.run.smithi142.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-08-20T00:53:54.233 INFO:teuthology.orchestra.run.smithi138.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-20T00:53:54.238 INFO:teuthology.orchestra.run.smithi142.stdout:[MIRROR] xfsprogs-devel-6.4.0-4.el9.x86_64.rpm: Status code: 404 for https://ftp-nyc.osuosl.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/xfsprogs-devel-6.4.0-4.el9.x86_64.rpm (IP: 64.50.233.100) 2024-08-20T00:53:54.258 INFO:teuthology.orchestra.run.smithi138.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-08-20T00:53:54.265 INFO:teuthology.orchestra.run.smithi142.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-08-20T00:53:54.307 INFO:teuthology.orchestra.run.smithi142.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 534 kB/s | 22 kB 00:00 2024-08-20T00:53:54.405 INFO:teuthology.orchestra.run.smithi138.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 30 MB/s | 4.4 MB 00:00 2024-08-20T00:53:54.434 INFO:teuthology.orchestra.run.smithi138.stdout:(108/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 126 kB/s | 96 kB 00:00 2024-08-20T00:53:54.459 INFO:teuthology.orchestra.run.smithi138.stdout:(109/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 479 kB/s | 26 kB 00:00 2024-08-20T00:53:54.493 INFO:teuthology.orchestra.run.smithi138.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-08-20T00:53:54.518 INFO:teuthology.orchestra.run.smithi138.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 6.4 MB/s | 151 kB 00:00 2024-08-20T00:53:54.543 INFO:teuthology.orchestra.run.smithi138.stdout:(112/150): liboath-2.6.7-2.el9.x86_64.rpm 444 kB/s | 49 kB 00:00 2024-08-20T00:53:54.582 INFO:teuthology.orchestra.run.smithi142.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-08-20T00:53:54.594 INFO:teuthology.orchestra.run.smithi138.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-08-20T00:53:54.619 INFO:teuthology.orchestra.run.smithi138.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-08-20T00:53:54.633 INFO:teuthology.orchestra.run.smithi142.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-08-20T00:53:54.644 INFO:teuthology.orchestra.run.smithi138.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T00:53:54.666 INFO:teuthology.orchestra.run.smithi142.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-08-20T00:53:54.670 INFO:teuthology.orchestra.run.smithi138.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 562 kB/s | 14 kB 00:00 2024-08-20T00:53:54.692 INFO:teuthology.orchestra.run.smithi142.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-08-20T00:53:54.695 INFO:teuthology.orchestra.run.smithi138.stdout:(117/150): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-08-20T00:53:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-08-20T00:53:54.771 INFO:teuthology.orchestra.run.smithi138.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-08-20T00:53:54.800 INFO:teuthology.orchestra.run.smithi142.stdout:(107/150): libarrow-9.0.0-11.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-08-20T00:53:54.804 INFO:teuthology.orchestra.run.smithi138.stdout:(120/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.2 MB/s | 863 kB 00:00 2024-08-20T00:53:54.825 INFO:teuthology.orchestra.run.smithi142.stdout:(108/150): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-08-20T00:53:54.830 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2024-08-20T00:53:54.830 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2024-08-20T00:53:54.851 INFO:teuthology.orchestra.run.smithi142.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-08-20T00:53:54.855 INFO:teuthology.orchestra.run.smithi138.stdout:(121/150): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-08-20T00:53:54.868 INFO:teuthology.orchestra.run.smithi142.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 3.9 MB/s | 67 kB 00:00 2024-08-20T00:53:54.888 INFO:teuthology.orchestra.run.smithi138.stdout:(122/150): python3-google-auth-2.33.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-08-20T00:53:54.893 INFO:teuthology.orchestra.run.smithi142.stdout:(111/150): luarocks-3.9.2-5.el9.noarch.rpm 5.9 MB/s | 151 kB 00:00 2024-08-20T00:53:54.913 INFO:teuthology.orchestra.run.smithi138.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.5 MB/s | 144 kB 00:00 2024-08-20T00:53:54.918 INFO:teuthology.orchestra.run.smithi142.stdout:(112/150): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 129 kB/s | 96 kB 00:00 2024-08-20T00:53:54.939 INFO:teuthology.orchestra.run.smithi138.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-08-20T00:53:54.960 INFO:teuthology.orchestra.run.smithi142.stdout:(113/150): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-08-20T00:53:54.964 INFO:teuthology.orchestra.run.smithi138.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-08-20T00:53:54.985 INFO:teuthology.orchestra.run.smithi142.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-08-20T00:53:54.989 INFO:teuthology.orchestra.run.smithi138.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-08-20T00:53:55.011 INFO:teuthology.orchestra.run.smithi142.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-08-20T00:53:55.015 INFO:teuthology.orchestra.run.smithi138.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-08-20T00:53:55.036 INFO:teuthology.orchestra.run.smithi142.stdout:(116/150): python3-certifi-2023.05.07-4.el9.noa 568 kB/s | 14 kB 00:00 2024-08-20T00:53:55.048 INFO:teuthology.orchestra.run.smithi138.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-08-20T00:53:55.061 INFO:teuthology.orchestra.run.smithi142.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-08-20T00:53:55.073 INFO:teuthology.orchestra.run.smithi138.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-08-20T00:53:55.086 INFO:teuthology.orchestra.run.smithi142.stdout:(118/150): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-08-20T00:53:55.090 INFO:teuthology.orchestra.run.smithi138.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 4.6 MB/s | 79 kB 00:00 2024-08-20T00:53:55.112 INFO:teuthology.orchestra.run.smithi142.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-08-20T00:53:55.116 INFO:teuthology.orchestra.run.smithi138.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-08-20T00:53:55.137 INFO:teuthology.orchestra.run.smithi142.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-08-20T00:53:55.149 INFO:teuthology.orchestra.run.smithi138.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 8.1 MB/s | 272 kB 00:00 2024-08-20T00:53:55.162 INFO:teuthology.orchestra.run.smithi142.stdout:(121/150): python3-google-auth-2.33.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-08-20T00:53:55.175 INFO:teuthology.orchestra.run.smithi138.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-08-20T00:53:55.188 INFO:teuthology.orchestra.run.smithi142.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-08-20T00:53:55.200 INFO:teuthology.orchestra.run.smithi138.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-08-20T00:53:55.204 INFO:teuthology.orchestra.run.smithi142.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 647 kB/s | 11 kB 00:00 2024-08-20T00:53:55.225 INFO:teuthology.orchestra.run.smithi138.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-08-20T00:53:55.230 INFO:teuthology.orchestra.run.smithi142.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-08-20T00:53:55.255 INFO:teuthology.orchestra.run.smithi142.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-08-20T00:53:55.277 INFO:teuthology.orchestra.run.smithi138.stdout:(136/150): python3-kubernetes-26.1.0-2.el9.noar 3.9 MB/s | 1.0 MB 00:00 2024-08-20T00:53:55.280 INFO:teuthology.orchestra.run.smithi142.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-08-20T00:53:55.301 INFO:teuthology.orchestra.run.smithi138.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-08-20T00:53:55.306 INFO:teuthology.orchestra.run.smithi142.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 791 kB/s | 20 kB 00:00 2024-08-20T00:53:55.326 INFO:teuthology.orchestra.run.smithi138.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-20T00:53:55.343 INFO:teuthology.orchestra.run.smithi138.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-08-20T00:53:55.356 INFO:teuthology.orchestra.run.smithi142.stdout:(128/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-08-20T00:53:55.368 INFO:teuthology.orchestra.run.smithi138.stdout:(140/150): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-08-20T00:53:55.389 INFO:teuthology.orchestra.run.smithi142.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-08-20T00:53:55.394 INFO:teuthology.orchestra.run.smithi138.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-20T00:53:55.415 INFO:teuthology.orchestra.run.smithi142.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-08-20T00:53:55.419 INFO:teuthology.orchestra.run.smithi138.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-20T00:53:55.440 INFO:teuthology.orchestra.run.smithi142.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-08-20T00:53:55.444 INFO:teuthology.orchestra.run.smithi138.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-20T00:53:55.465 INFO:teuthology.orchestra.run.smithi142.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-08-20T00:53:55.470 INFO:teuthology.orchestra.run.smithi138.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 797 kB/s | 20 kB 00:00 2024-08-20T00:53:55.491 INFO:teuthology.orchestra.run.smithi142.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 651 kB/s | 16 kB 00:00 2024-08-20T00:53:55.512 INFO:teuthology.orchestra.run.smithi138.stdout:(145/150): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-08-20T00:53:55.516 INFO:teuthology.orchestra.run.smithi142.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-08-20T00:53:55.537 INFO:teuthology.orchestra.run.smithi138.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-08-20T00:53:55.541 INFO:teuthology.orchestra.run.smithi142.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-08-20T00:53:55.567 INFO:teuthology.orchestra.run.smithi142.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-08-20T00:53:55.570 INFO:teuthology.orchestra.run.smithi138.stdout:(147/150): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-08-20T00:53:55.592 INFO:teuthology.orchestra.run.smithi142.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-08-20T00:53:55.617 INFO:teuthology.orchestra.run.smithi142.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-08-20T00:53:55.638 INFO:teuthology.orchestra.run.smithi138.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-08-20T00:53:55.642 INFO:teuthology.orchestra.run.smithi142.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-08-20T00:53:55.668 INFO:teuthology.orchestra.run.smithi142.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-08-20T00:53:55.693 INFO:teuthology.orchestra.run.smithi142.stdout:(141/150): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-08-20T00:53:55.718 INFO:teuthology.orchestra.run.smithi142.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-08-20T00:53:55.743 INFO:teuthology.orchestra.run.smithi142.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-08-20T00:53:55.769 INFO:teuthology.orchestra.run.smithi142.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 803 kB/s | 20 kB 00:00 2024-08-20T00:53:55.794 INFO:teuthology.orchestra.run.smithi142.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 7.5 MB/s | 192 kB 00:00 2024-08-20T00:53:55.819 INFO:teuthology.orchestra.run.smithi142.stdout:(146/150): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-08-20T00:53:55.870 INFO:teuthology.orchestra.run.smithi142.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-08-20T00:53:55.938 INFO:teuthology.orchestra.run.smithi138.stdout:(149/150): librados2-19.0.0-5872.gef427541.el9. 8.7 MB/s | 3.5 MB 00:00 2024-08-20T00:53:56.005 INFO:teuthology.orchestra.run.smithi138.stdout:(150/150): librbd1-19.0.0-5872.gef427541.el9.x8 7.6 MB/s | 3.2 MB 00:00 2024-08-20T00:53:56.014 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-08-20T00:53:56.014 INFO:teuthology.orchestra.run.smithi138.stdout:Total 12 MB/s | 232 MB 00:19 2024-08-20T00:53:56.220 INFO:teuthology.orchestra.run.smithi142.stdout:(148/150): librados2-19.0.0-5872.gef427541.el9. 8.7 MB/s | 3.5 MB 00:00 2024-08-20T00:53:56.278 INFO:teuthology.orchestra.run.smithi142.stdout:(149/150): librbd1-19.0.0-5872.gef427541.el9.x8 8.1 MB/s | 3.2 MB 00:00 2024-08-20T00:53:57.903 INFO:teuthology.orchestra.run.smithi142.stdout:(150/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.3 MB/s | 16 MB 00:04 2024-08-20T00:53:57.911 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-08-20T00:53:57.911 INFO:teuthology.orchestra.run.smithi142.stdout:Total 10 MB/s | 232 MB 00:22 2024-08-20T00:53:57.938 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-08-20T00:53:58.091 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-08-20T00:53:58.091 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-08-20T00:53:59.055 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-08-20T00:53:59.191 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-08-20T00:53:59.191 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-08-20T00:54:00.545 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-08-20T00:54:00.546 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-08-20T00:54:00.915 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2024-08-20T00:54:01.003 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T00:54:01.019 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-08-20T00:54:01.020 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-08-20T00:54:01.027 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T00:54:01.070 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T00:54:01.416 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T00:54:01.528 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:01.651 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:01.688 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:01.783 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:01.822 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:54:01.850 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T00:54:01.876 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T00:54:02.029 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T00:54:02.082 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T00:54:02.109 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:02.214 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:02.244 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:02.307 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:02.337 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:02.362 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:02.672 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:02.704 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:02.811 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T00:54:02.843 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T00:54:02.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T00:54:02.952 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T00:54:03.013 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T00:54:03.052 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T00:54:03.095 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T00:54:03.128 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T00:54:03.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T00:54:03.235 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T00:54:03.338 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T00:54:03.367 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T00:54:03.403 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T00:54:03.490 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T00:54:03.522 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T00:54:03.567 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T00:54:03.594 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T00:54:03.666 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T00:54:03.708 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T00:54:03.785 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T00:54:03.930 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T00:54:03.996 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T00:54:04.534 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T00:54:04.625 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T00:54:04.661 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T00:54:04.703 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T00:54:04.742 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T00:54:04.783 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T00:54:04.877 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T00:54:04.941 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T00:54:05.015 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T00:54:05.061 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T00:54:05.097 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T00:54:05.145 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T00:54:05.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T00:54:05.223 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T00:54:05.266 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T00:54:05.406 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T00:54:05.459 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T00:54:05.500 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T00:54:05.607 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T00:54:05.658 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T00:54:05.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T00:54:05.724 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T00:54:05.749 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T00:54:05.825 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T00:54:05.953 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T00:54:05.989 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T00:54:06.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T00:54:06.051 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T00:54:06.096 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T00:54:06.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T00:54:06.182 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T00:54:06.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T00:54:06.246 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T00:54:06.287 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T00:54:06.329 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T00:54:06.548 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T00:54:06.642 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T00:54:06.687 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T00:54:06.726 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T00:54:06.841 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T00:54:06.893 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-08-20T00:54:06.902 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T00:54:06.975 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-08-20T00:54:07.005 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T00:54:07.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T00:54:07.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-08-20T00:54:07.072 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-08-20T00:54:07.123 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T00:54:07.126 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-08-20T00:54:07.458 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T00:54:07.500 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-08-20T00:54:07.565 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:07.634 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:07.648 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T00:54:07.687 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:07.699 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T00:54:07.716 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:07.732 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T00:54:07.779 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:19.0.0-5872.gef427541.el9.x86_64 5/152 2024-08-20T00:54:07.810 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:07.811 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:07.842 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T00:54:07.857 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:54:07.886 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T00:54:07.896 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T00:54:07.912 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T00:54:07.941 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T00:54:07.945 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:08.002 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:54:08.030 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T00:54:08.041 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-08-20T00:54:08.068 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T00:54:08.075 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-08-20T00:54:08.077 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T00:54:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T00:54:08.270 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T00:54:08.308 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T00:54:08.317 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 10/152 2024-08-20T00:54:08.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:08.380 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-13.16-1.el9.x86_64 11/152 2024-08-20T00:54:08.385 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T00:54:08.407 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:08.442 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:08.471 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:08.518 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:08.535 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:08.548 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:08.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:08.568 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:08.634 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 13/152 2024-08-20T00:54:08.651 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:08.695 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-08-20T00:54:08.822 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:08.855 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:08.962 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T00:54:08.995 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T00:54:09.030 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T00:54:09.050 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T00:54:09.065 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:09.103 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-08-20T00:54:09.105 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T00:54:09.156 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T00:54:09.177 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T00:54:09.204 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T00:54:09.247 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T00:54:09.264 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-08-20T00:54:09.280 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T00:54:09.300 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/152 2024-08-20T00:54:09.308 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T00:54:09.346 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2024-08-20T00:54:09.379 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T00:54:09.430 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2024-08-20T00:54:09.432 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T00:54:09.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T00:54:09.490 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/152 2024-08-20T00:54:09.505 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T00:54:09.529 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 21/152 2024-08-20T00:54:09.583 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-08-20T00:54:09.598 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T00:54:09.632 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T00:54:09.635 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-08-20T00:54:09.672 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-08-20T00:54:09.678 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T00:54:09.704 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T00:54:09.763 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-08-20T00:54:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T00:54:09.827 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T00:54:09.834 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 26/152 2024-08-20T00:54:09.868 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-08-20T00:54:09.904 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T00:54:09.914 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 28/152 2024-08-20T00:54:10.007 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 29/152 2024-08-20T00:54:10.041 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T00:54:10.055 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 30/152 2024-08-20T00:54:10.102 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 31/152 2024-08-20T00:54:10.113 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T00:54:10.135 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 32/152 2024-08-20T00:54:10.228 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-08-20T00:54:10.287 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 34/152 2024-08-20T00:54:10.380 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-08-20T00:54:10.467 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T00:54:10.545 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-08-20T00:54:10.554 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:54:10.583 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T00:54:10.608 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-08-20T00:54:10.615 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T00:54:10.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T00:54:10.761 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T00:54:10.796 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T00:54:10.839 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T00:54:10.869 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T00:54:10.902 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T00:54:10.936 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T00:54:10.986 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:10.995 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T00:54:11.059 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T00:54:11.082 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:11.116 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T00:54:11.132 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T00:54:11.152 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T00:54:11.170 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T00:54:11.206 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T00:54:11.249 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-08-20T00:54:11.253 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T00:54:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T00:54:11.340 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T00:54:11.382 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T00:54:11.390 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-08-20T00:54:11.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 40/152 2024-08-20T00:54:11.478 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-08-20T00:54:11.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 42/152 2024-08-20T00:54:11.520 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T00:54:11.567 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-08-20T00:54:11.568 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T00:54:11.609 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T00:54:11.652 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T00:54:11.672 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-08-20T00:54:11.690 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:11.712 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T00:54:11.738 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 45/152 2024-08-20T00:54:11.763 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:11.775 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T00:54:11.791 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T00:54:11.815 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T00:54:11.841 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T00:54:11.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 46/152 2024-08-20T00:54:11.865 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T00:54:11.885 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 47/152 2024-08-20T00:54:11.922 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 48/152 2024-08-20T00:54:11.941 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T00:54:11.980 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 49/152 2024-08-20T00:54:12.021 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 50/152 2024-08-20T00:54:12.066 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 51/152 2024-08-20T00:54:12.076 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T00:54:12.114 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T00:54:12.116 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 52/152 2024-08-20T00:54:12.142 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T00:54:12.176 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T00:54:12.221 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T00:54:12.258 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T00:54:12.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 53/152 2024-08-20T00:54:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T00:54:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T00:54:12.333 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 54/152 2024-08-20T00:54:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T00:54:12.380 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 55/152 2024-08-20T00:54:12.387 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T00:54:12.438 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T00:54:12.472 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T00:54:12.510 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-08-20T00:54:12.558 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.33.0-1.el9.noarch 57/152 2024-08-20T00:54:12.561 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T00:54:12.602 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-08-20T00:54:12.611 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T00:54:12.641 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T00:54:12.672 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-08-20T00:54:12.696 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 60/152 2024-08-20T00:54:12.748 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T00:54:12.801 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T00:54:12.839 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-08-20T00:54:13.007 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-08-20T00:54:13.075 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-08-20T00:54:13.108 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-08-20T00:54:13.147 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-08-20T00:54:13.198 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-08-20T00:54:13.243 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 67/152 2024-08-20T00:54:13.281 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-08-20T00:54:13.319 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-08-20T00:54:13.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-08-20T00:54:13.403 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-08-20T00:54:13.446 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-08-20T00:54:13.506 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-08-20T00:54:13.539 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T00:54:13.581 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T00:54:13.604 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-08-20T00:54:13.614 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T00:54:13.654 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-08-20T00:54:13.699 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-08-20T00:54:13.715 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T00:54:13.774 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T00:54:13.822 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T00:54:13.823 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-08-20T00:54:13.895 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 78/152 2024-08-20T00:54:13.904 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T00:54:13.975 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T00:54:14.159 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T00:54:14.189 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T00:54:14.267 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T00:54:14.365 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:14.413 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:14.815 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 79/152 2024-08-20T00:54:14.873 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 80/152 2024-08-20T00:54:14.907 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 81/152 2024-08-20T00:54:14.931 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T00:54:14.976 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:15.016 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:15.022 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 82/152 2024-08-20T00:54:15.042 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T00:54:15.082 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.19-7.el9.x86_64 83/152 2024-08-20T00:54:15.104 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:15.104 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T00:54:15.104 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T00:54:15.104 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:15.134 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 84/152 2024-08-20T00:54:15.142 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:15.231 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-08-20T00:54:15.275 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-08-20T00:54:15.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.26-1.el9.x86_64 87/152 2024-08-20T00:54:15.547 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.26-1.el9.x86_64 88/152 2024-08-20T00:54:15.645 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-08-20T00:54:16.299 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T00:54:16.378 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:54:16.405 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T00:54:16.439 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T00:54:16.481 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-08-20T00:54:16.705 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-08-20T00:54:16.759 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T00:54:16.810 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:16.897 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:16.923 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T00:54:16.958 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T00:54:17.451 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T00:54:17.479 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:17.536 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:17.556 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T00:54:18.294 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-08-20T00:54:18.427 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:54:18.486 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-08-20T00:54:18.538 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-08-20T00:54:18.916 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-08-20T00:54:18.991 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:19.109 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 97/152 2024-08-20T00:54:19.180 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 98/152 2024-08-20T00:54:19.246 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-08-20T00:54:19.802 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 100/152 2024-08-20T00:54:19.859 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:19.860 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:19.894 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:19.953 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 101/152 2024-08-20T00:54:20.026 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 102/152 2024-08-20T00:54:20.448 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:20.480 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:20.562 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:20.563 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T00:54:20.563 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T00:54:20.563 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:20.598 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 103/152 2024-08-20T00:54:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 104/152 2024-08-20T00:54:23.651 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-08-20T00:54:23.651 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-08-20T00:54:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /root 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2024-08-20T00:54:26.799 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:26.971 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:26.989 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:26.989 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:26.989 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T00:54:26.989 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:28.508 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:28.534 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:28.534 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:28.534 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T00:54:28.535 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:28.535 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:28.535 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:28.759 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T00:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:28.782 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:28.783 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:29.225 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:29.249 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:29.364 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T00:54:29.412 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T00:54:29.467 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T00:54:29.660 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T00:54:29.710 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T00:54:29.796 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T00:54:29.884 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T00:54:30.024 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T00:54:30.068 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T00:54:30.100 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T00:54:30.128 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T00:54:30.473 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:30.502 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:30.534 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T00:54:30.599 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T00:54:30.633 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-08-20T00:54:32.129 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /root 2024-08-20T00:54:32.130 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-08-20T00:54:32.130 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:32.301 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:32.317 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:32.317 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T00:54:32.317 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:33.804 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:33.805 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:33.830 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:33.831 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:33.831 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T00:54:33.831 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:33.831 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:33.831 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:33.874 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:34.064 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:34.087 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:34.087 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:34.087 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T00:54:34.087 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:34.088 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:34.088 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:34.530 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:34.554 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:34.554 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:34.554 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T00:54:34.554 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:34.555 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:34.555 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:34.660 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T00:54:34.717 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T00:54:34.773 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T00:54:34.833 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:34.866 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:34.959 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T00:54:34.980 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:35.015 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T00:54:35.091 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T00:54:35.154 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T00:54:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T00:54:35.185 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:35.210 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:35.210 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:35.210 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T00:54:35.210 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:35.210 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:35.211 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:35.261 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:35.273 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:35.329 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T00:54:35.365 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T00:54:35.371 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T00:54:35.397 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T00:54:35.418 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T00:54:35.426 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T00:54:35.846 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:35.874 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:35.906 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T00:54:35.971 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T00:54:35.997 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 105/152 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-08-20T00:54:36.183 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-08-20T00:54:36.184 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:36.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T00:54:36.382 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:36.405 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:36.419 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 106/152 2024-08-20T00:54:36.419 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:36.419 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-08-20T00:54:36.419 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:36.429 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:36.429 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:36.429 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T00:54:36.429 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:36.430 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:36.430 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:36.472 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:36.496 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:36.496 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:36.496 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T00:54:36.496 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:36.549 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:36.572 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:36.891 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:36.914 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:36.915 INFO:teuthology.orchestra.run.smithi167.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:36.915 INFO:teuthology.orchestra.run.smithi167.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T00:54:36.915 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:36.915 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:36.915 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:38.078 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 107/152 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-08-20T00:54:38.137 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:38.379 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:38.430 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 108/152 2024-08-20T00:54:38.430 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:38.431 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-08-20T00:54:38.431 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:38.431 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-08-20T00:54:38.431 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:38.921 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:38.972 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 109/152 2024-08-20T00:54:38.973 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:38.973 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-08-20T00:54:38.973 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:38.973 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-08-20T00:54:38.973 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:39.104 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-08-20T00:54:39.170 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:39.190 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-08-20T00:54:39.237 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:39.253 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-08-20T00:54:39.479 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-08-20T00:54:39.541 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-08-20T00:54:39.644 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-08-20T00:54:39.751 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-08-20T00:54:39.901 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-08-20T00:54:39.952 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-08-20T00:54:40.014 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-08-20T00:54:40.040 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-16.el9.x86_64 120/152 2024-08-20T00:54:40.204 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:40.237 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:40.352 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:40.516 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T00:54:40.547 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:40.572 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:40.572 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:40.572 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T00:54:40.572 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:40.573 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:40.573 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:40.614 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:40.623 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:40.636 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:40.647 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:19.0.0-5872.gef427541.el9.noarch 121/152 2024-08-20T00:54:40.690 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 122/152 2024-08-20T00:54:40.733 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T00:54:40.768 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 123/152 2024-08-20T00:54:40.780 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T00:54:40.803 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:41.623 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T00:54:41.664 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T00:54:41.713 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T00:54:41.737 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T00:54:41.754 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T00:54:41.767 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:41.781 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:41.791 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:41.835 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:41.858 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:41.858 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:41.858 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T00:54:41.858 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:41.910 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:41.934 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:41.964 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T00:54:42.003 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T00:54:42.033 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T00:54:42.244 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:42.267 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:42.267 INFO:teuthology.orchestra.run.smithi142.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:42.268 INFO:teuthology.orchestra.run.smithi142.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T00:54:42.268 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:42.268 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:42.268 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:42.604 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:42.649 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:42.669 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T00:54:43.613 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T00:54:43.654 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T00:54:43.743 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T00:54:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T00:54:43.787 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:43.868 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:43.868 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:44.399 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 124/152 2024-08-20T00:54:44.476 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:45.554 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 125/152 2024-08-20T00:54:45.584 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:45.734 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 126/152 2024-08-20T00:54:45.915 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 127/152 2024-08-20T00:54:45.953 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 128/152 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-08-20T00:54:46.005 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:46.078 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:46.106 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 129/152 2024-08-20T00:54:46.204 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:19.0.0-5872.gef427541.el9.x86_64 130/152 2024-08-20T00:54:46.277 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 131/152 2024-08-20T00:54:46.947 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T00:54:46.997 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T00:54:47.038 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T00:54:47.070 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T00:54:47.097 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T00:54:47.276 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T00:54:47.307 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libtool-2.4.6-46.el9.x86_64 132/152 2024-08-20T00:54:47.319 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T00:54:47.349 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T00:54:47.361 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:47.413 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 133/152 2024-08-20T00:54:47.414 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:47.414 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-08-20T00:54:47.414 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:47.414 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-08-20T00:54:47.414 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:47.465 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:47.514 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-5872.gef427 134/152 2024-08-20T00:54:47.514 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:47.515 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-08-20T00:54:47.515 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:47.565 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 135/152 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-08-20T00:54:47.617 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:47.914 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:47.955 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:47.966 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:47.975 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 136/152 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-08-20T00:54:48.019 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:48.792 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T00:54:48.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T00:54:48.925 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T00:54:49.006 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T00:54:49.047 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T00:54:49.047 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:49.130 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:49.131 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:50.815 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:50.815 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T00:54:50.816 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T00:54:50.817 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T00:54:50.818 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T00:54:50.819 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T00:54:50.820 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T00:54:50.821 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T00:54:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T00:54:50.823 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T00:54:50.824 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T00:54:50.825 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T00:54:50.826 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T00:54:50.827 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T00:54:53.066 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.067 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T00:54:53.068 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.069 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T00:54:53.070 INFO:teuthology.orchestra.run.smithi167.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T00:54:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T00:54:53.072 INFO:teuthology.orchestra.run.smithi167.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T00:54:53.073 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T00:54:53.074 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.075 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T00:54:53.076 INFO:teuthology.orchestra.run.smithi167.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T00:54:53.077 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T00:54:53.078 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:54:53.078 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2024-08-20T00:54:53.098 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 137/152 2024-08-20T00:54:53.148 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 138/152 2024-08-20T00:54:53.217 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 139/152 2024-08-20T00:54:53.254 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 140/152 2024-08-20T00:54:53.290 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 141/152 2024-08-20T00:54:53.410 DEBUG:teuthology.parallel:result is None 2024-08-20T00:54:53.528 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 142/152 2024-08-20T00:54:53.597 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 143/152 2024-08-20T00:54:53.625 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 144/152 2024-08-20T00:54:54.390 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:54.440 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:54.483 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 145/152 2024-08-20T00:54:55.509 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:55.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T00:54:55.509 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T00:54:55.510 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T00:54:55.511 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T00:54:55.512 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T00:54:55.522 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T00:54:55.523 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T00:54:55.524 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T00:54:55.525 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T00:54:55.526 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T00:54:55.527 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T00:54:55.528 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T00:54:55.529 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T00:54:55.530 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T00:54:55.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T00:54:55.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T00:54:55.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T00:54:55.531 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T00:54:55.740 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 146/152 2024-08-20T00:54:55.838 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openssl-devel-1:3.2.2-2.el9.x86_64 147/152 2024-08-20T00:54:55.899 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-08-20T00:54:56.037 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-08-20T00:54:56.085 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-08-20T00:54:56.085 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:56.200 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-08-20T00:54:56.201 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:56.597 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:56.598 INFO:teuthology.orchestra.run.smithi138.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-08-20T00:54:56.598 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:54:57.732 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:54:57.732 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:57.732 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.733 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.734 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T00:54:57.735 INFO:teuthology.orchestra.run.smithi142.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.736 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T00:54:57.737 INFO:teuthology.orchestra.run.smithi142.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T00:54:57.738 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T00:54:57.739 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T00:54:57.740 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T00:54:57.741 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T00:54:57.742 INFO:teuthology.orchestra.run.smithi142.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.743 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:54:57.744 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-08-20T00:54:58.106 DEBUG:teuthology.parallel:result is None 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:19.0.0-5872.gef427541.el9.x86_64 1/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 3/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 4/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-5872.gef427 5/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 6/152 2024-08-20T00:55:03.223 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 7/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 8/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 9/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 10/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 11/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 12/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 13/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 14/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_ 15/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 16/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 17/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:19.0.0-5872.gef427541.el9.x86_6 18/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:19.0.0-5872.gef427541.el9.x86 19/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:19.0.0-5872.gef427541.el9.x86_64 20/152 2024-08-20T00:55:03.224 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:19.0.0-5872.gef427541.el 21/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:19.0.0-5872.gef427541.el9. 22/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_6 23/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 24/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 25/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 26/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 27/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 28/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 29/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-5872.gef427541. 30/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noa 31/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.n 32/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-5872.gef4 33/152 2024-08-20T00:55:03.225 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el 34/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 35/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-5872.gef427541.e 36/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 37/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:19.0.0-5872.gef427541.el9.noarch 38/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 39/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-08-20T00:55:03.226 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-08-20T00:55:03.227 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.26-1.el9.x86_64 66/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.26-1.el9.x86_64 67/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openssl-devel-1:3.2.2-2.el9.x86_64 68/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-08-20T00:55:03.228 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.19-7.el9.x86_64 77/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/152 2024-08-20T00:55:03.229 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/152 2024-08-20T00:55:03.230 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/152 2024-08-20T00:55:03.231 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.33.0-1.el9.noarch 122/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-08-20T00:55:03.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-08-20T00:55:03.233 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:19.0.0-5872.gef427541.el9.x86_64 149/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-08-20T00:55:03.234 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:19.0.0-5872.gef427541.el9.x86_64 151/152 2024-08-20T00:55:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-08-20T00:55:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:55:05.786 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-08-20T00:55:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.786 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: automake-1.16.2-8.el9.noarch 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: bison-3.7.4-5.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.787 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.788 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-volume-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-shell-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: cephfs-top-2:19.0.0-5872.gef427541.el9.noarch 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: flex-2.6.4-9.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: fuse-2.9.9-16.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-08-20T00:55:05.789 INFO:teuthology.orchestra.run.smithi138.stdout: grpc-data-1.46.7-10.el9.noarch 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-9.0.0-11.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-08-20T00:55:05.790 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: libtool-2.4.6-46.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: lua-5.4.4-4.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: luarocks-3.9.2-5.el9.noarch 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.26-1.el9.x86_64 2024-08-20T00:55:05.791 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.26-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: openssl-devel-1:3.2.2-2.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-13.16-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: postgresql-server-13.16-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-3.14.0-13.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-08-20T00:55:05.792 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.19-7.el9.x86_64 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.33.0-1.el9.noarch 2024-08-20T00:55:05.793 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-08-20T00:55:05.794 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-packaging-20.9-5.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-08-20T00:55:05.795 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-08-20T00:55:05.796 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-24.02.0-1.el9.x86_64 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-08-20T00:55:05.797 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:19.0.0-5872.gef427541.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: re2-1:20211101-3.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-6.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: uuid-1.6.2-55.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-08-20T00:55:05.798 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-08-20T00:55:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-08-20T00:55:05.799 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:55:05.799 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-08-20T00:55:06.357 DEBUG:teuthology.parallel:result is None 2024-08-20T00:55:06.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:06.510 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T00:55:06.557 INFO:teuthology.orchestra.run.smithi138.stdout:19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.558 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.558 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T00:55:06.560 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:06.702 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T00:55:06.728 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.729 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.729 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T00:55:06.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:06.878 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-08-20T00:55:06.905 INFO:teuthology.orchestra.run.smithi167.stdout:19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.905 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-5872.gef427541.el9 2024-08-20T00:55:06.905 INFO:teuthology.task.install:The correct ceph version 19.0.0-5872.gef427541 is installed. 2024-08-20T00:55:06.907 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-08-20T00:55:06.907 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:55:06.907 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T00:55:06.967 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:55:06.967 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T00:55:07.000 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:55:07.000 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-08-20T00:55:07.033 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-08-20T00:55:07.033 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:55:07.033 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T00:55:07.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T00:55:07.186 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:55:07.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T00:55:07.216 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T00:55:07.286 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:55:07.287 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-08-20T00:55:07.316 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-08-20T00:55:07.384 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-08-20T00:55:07.385 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:55:07.385 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T00:55:07.440 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T00:55:07.538 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:55:07.538 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T00:55:07.568 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T00:55:07.637 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:55:07.637 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-08-20T00:55:07.662 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-08-20T00:55:07.730 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-08-20T00:55:07.731 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:55:07.731 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T00:55:07.787 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T00:55:07.884 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:55:07.884 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T00:55:07.913 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T00:55:07.983 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:55:07.983 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-08-20T00:55:08.008 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-08-20T00:55:08.073 INFO:teuthology.run_tasks:Running task cephadm... 2024-08-20T00:55:08.180 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug cephsqlite': 20, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug'}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'mgr-modules': ['snap_schedule'], 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T00:55:08.180 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:08.180 INFO:tasks.cephadm:Cluster fsid is d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:55:08.180 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-08-20T00:55:08.181 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.138', 'mon.b': '172.21.15.142', 'mon.c': '172.21.15.167'} 2024-08-20T00:55:08.181 INFO:tasks.cephadm:First mon is mon.a on smithi138 2024-08-20T00:55:08.181 INFO:tasks.cephadm:First mgr is x 2024-08-20T00:55:08.181 INFO:tasks.cephadm:Normalizing hostnames... 2024-08-20T00:55:08.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-08-20T00:55:08.239 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-08-20T00:55:08.270 DEBUG:teuthology.orchestra.run.smithi167:> sudo hostname $(hostname -s) 2024-08-20T00:55:08.295 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-08-20T00:55:08.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:08.433 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'ef4275410d8483698016fe484b2a3c664080b4c1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82210/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-5872-gef427541', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-5872.gef427541'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-08-19 17:15:28.176472', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-08-19-1357', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-08-20T00:55:08.573 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-hemanth-testing-2024-08-19-1357, sha1 ef4275410d8483698016fe484b2a3c664080b4c1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:55:08.574 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T00:55:08.574 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm 2024-08-20T00:55:08.575 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:08.967 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 00:55 /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:08.967 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:09.343 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 00:55 /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:09.344 DEBUG:teuthology.orchestra.run.smithi167:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-08-19-1357/ef4275410d8483698016fe484b2a3c664080b4c1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:10.047 INFO:teuthology.orchestra.run.smithi167.stdout:-rw-r--r--. 1 ubuntu ubuntu 792014 Aug 20 00:55 /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:10.048 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:10.085 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:10.105 DEBUG:teuthology.orchestra.run.smithi167:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-08-20T00:55:10.139 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 on all hosts... 2024-08-20T00:55:10.140 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T00:55:10.144 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T00:55:10.147 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 pull 2024-08-20T00:55:11.781 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T00:55:11.893 INFO:teuthology.orchestra.run.smithi167.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T00:55:12.806 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T00:56:02.535 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T00:56:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T00:56:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T00:56:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-08-20T00:56:02.535 INFO:teuthology.orchestra.run.smithi138.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T00:56:02.536 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T00:56:02.536 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T00:56:07.268 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-20T00:56:07.268 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T00:56:07.269 INFO:teuthology.orchestra.run.smithi167.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T00:56:07.269 INFO:teuthology.orchestra.run.smithi167.stdout: "repo_digests": [ 2024-08-20T00:56:07.269 INFO:teuthology.orchestra.run.smithi167.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T00:56:07.269 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-08-20T00:56:07.269 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-20T00:56:17.982 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a", 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8" 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-08-20T00:56:17.983 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-08-20T00:56:18.009 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-08-20T00:56:18.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-08-20T00:56:18.102 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph 2024-08-20T00:56:18.135 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-08-20T00:56:18.192 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-08-20T00:56:18.217 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /etc/ceph 2024-08-20T00:56:18.244 INFO:tasks.cephadm:Writing seed config... 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-08-20T00:56:18.246 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-08-20T00:56:18.247 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-08-20T00:56:18.248 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:56:18.248 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-08-20T00:56:18.283 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d876cd00-5e8e-11ef-bccf-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-08-20T00:56:18.284 DEBUG:teuthology.orchestra.run.smithi138:mon.a> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service 2024-08-20T00:56:18.328 DEBUG:teuthology.orchestra.run.smithi138:mgr.x> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x.service 2024-08-20T00:56:18.372 INFO:tasks.cephadm:Bootstrapping... 2024-08-20T00:56:18.373 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 -v bootstrap --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.138 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-08-20T00:56:18.628 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-08-20T00:56:18.628 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', '-v', 'bootstrap', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.138', '--skip-admin-label'] 2024-08-20T00:56:18.628 INFO:teuthology.orchestra.run.smithi138.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-08-20T00:56:18.628 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying podman|docker is present... 2024-08-20T00:56:18.672 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stdout 5.2.1 2024-08-20T00:56:18.672 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying lvm2 is present... 2024-08-20T00:56:18.673 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying time synchronization is in place... 2024-08-20T00:56:18.686 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T00:56:18.687 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T00:56:18.699 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T00:56:18.699 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:56:18.712 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout enabled 2024-08-20T00:56:18.725 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout active 2024-08-20T00:56:18.725 INFO:teuthology.orchestra.run.smithi138.stdout:Unit chronyd.service is enabled and running 2024-08-20T00:56:18.725 INFO:teuthology.orchestra.run.smithi138.stdout:Repeating the final host check... 2024-08-20T00:56:18.769 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stdout 5.2.1 2024-08-20T00:56:18.769 INFO:teuthology.orchestra.run.smithi138.stdout:podman (/bin/podman) version 5.2.1 is present 2024-08-20T00:56:18.769 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl is present 2024-08-20T00:56:18.770 INFO:teuthology.orchestra.run.smithi138.stdout:lvcreate is present 2024-08-20T00:56:18.782 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-08-20T00:56:18.783 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-08-20T00:56:18.795 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-08-20T00:56:18.795 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:56:18.808 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout enabled 2024-08-20T00:56:18.821 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout active 2024-08-20T00:56:18.821 INFO:teuthology.orchestra.run.smithi138.stdout:Unit chronyd.service is enabled and running 2024-08-20T00:56:18.821 INFO:teuthology.orchestra.run.smithi138.stdout:Host looks OK 2024-08-20T00:56:18.821 INFO:teuthology.orchestra.run.smithi138.stdout:Cluster fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:18.821 INFO:teuthology.orchestra.run.smithi138.stdout:Acquiring lock 140000366795888 on /run/cephadm/d876cd00-5e8e-11ef-bccf-c7b262605968.lock 2024-08-20T00:56:18.822 INFO:teuthology.orchestra.run.smithi138.stdout:Lock 140000366795888 acquired on /run/cephadm/d876cd00-5e8e-11ef-bccf-c7b262605968.lock 2024-08-20T00:56:18.822 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying IP 172.21.15.138 port 3300 ... 2024-08-20T00:56:18.822 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying IP 172.21.15.138 port 6789 ... 2024-08-20T00:56:18.822 INFO:teuthology.orchestra.run.smithi138.stdout:Base mon IP(s) is [172.21.15.138:3300, 172.21.15.138:6789], mon addrv is [v2:172.21.15.138:3300,v1:172.21.15.138:6789] 2024-08-20T00:56:18.829 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.138 metric 100 2024-08-20T00:56:18.829 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.138 metric 100 2024-08-20T00:56:18.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-08-20T00:56:18.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-08-20T00:56:18.835 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1796sec hoplimit 64 pref medium 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-08-20T00:56:18.841 INFO:teuthology.orchestra.run.smithi138.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-08-20T00:56:18.842 INFO:teuthology.orchestra.run.smithi138.stdout:Mon IP `172.21.15.138` is in CIDR network `172.21.0.0/20` 2024-08-20T00:56:18.842 INFO:teuthology.orchestra.run.smithi138.stdout:Mon IP `172.21.15.138` is in CIDR network `172.21.0.0/20` 2024-08-20T00:56:18.842 INFO:teuthology.orchestra.run.smithi138.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-08-20T00:56:18.843 INFO:teuthology.orchestra.run.smithi138.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-08-20T00:56:18.844 INFO:teuthology.orchestra.run.smithi138.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T00:56:19.329 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stdout 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1... 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Getting image source signatures 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Copying blob sha256:0beb2e0cf0f87d34a70e3776822a6ff39fafdc5a4d91bc4481cb39d566d52b0d 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Copying config sha256:9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a 2024-08-20T00:56:19.330 INFO:teuthology.orchestra.run.smithi138.stdout:/bin/podman: stderr Writing manifest to image destination 2024-08-20T00:56:20.442 INFO:teuthology.orchestra.run.smithi138.stdout:ceph: stdout ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T00:56:20.442 INFO:teuthology.orchestra.run.smithi138.stdout:Ceph version: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev) 2024-08-20T00:56:20.442 INFO:teuthology.orchestra.run.smithi138.stdout:Extracting ceph user uid/gid from container image... 2024-08-20T00:56:21.516 INFO:teuthology.orchestra.run.smithi138.stdout:stat: stdout 167 167 2024-08-20T00:56:21.516 INFO:teuthology.orchestra.run.smithi138.stdout:Creating initial keys... 2024-08-20T00:56:22.590 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph-authtool: stdout AQC16cNmpis8NhAAjZmG8fqnsTfDx2+hYPplsQ== 2024-08-20T00:56:32.772 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph-authtool: stdout AQC56cNmoVByFBAAqMhwpVGhuQoRnzjvM7uJfA== 2024-08-20T00:56:33.891 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph-authtool: stdout AQDB6cNmwtsWDhAAj48kWGTM3ryhBmNMlVJUVQ== 2024-08-20T00:56:33.892 INFO:teuthology.orchestra.run.smithi138.stdout:Creating initial monmap... 2024-08-20T00:56:34.939 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T00:56:34.939 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-08-20T00:56:34.939 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:34.939 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout:monmaptool for a [v2:172.21.15.138:3300,v1:172.21.15.138:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout:setting min_mon_release = quincy 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: set fsid to d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:56:34.940 INFO:teuthology.orchestra.run.smithi138.stdout:Creating mon... 2024-08-20T00:56:35.973 INFO:teuthology.orchestra.run.smithi138.stdout:create mon.a on 2024-08-20T00:56:36.263 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-08-20T00:56:36.480 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-08-20T00:56:36.735 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target. 2024-08-20T00:56:36.735 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target → /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target. 2024-08-20T00:56:37.495 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a 2024-08-20T00:56:37.496 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service: Unit ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service not loaded. 2024-08-20T00:56:37.774 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target.wants/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@.service. 2024-08-20T00:56:38.414 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 podman[17696]: 2024-08-20 00:56:38.146245211 +0000 UTC m=+0.145343436 container create e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T00:56:38.414 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 podman[17696]: 2024-08-20 00:56:38.415993936 +0000 UTC m=+0.415092152 container init e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.build-date=20240731) 2024-08-20T00:56:38.523 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T00:56:38.523 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout disabled 2024-08-20T00:56:38.536 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T00:56:38.536 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:56:38.536 INFO:teuthology.orchestra.run.smithi138.stdout:firewalld.service is not enabled 2024-08-20T00:56:38.536 INFO:teuthology.orchestra.run.smithi138.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T00:56:38.537 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mon to start... 2024-08-20T00:56:38.537 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mon... 2024-08-20T00:56:38.678 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 podman[17696]: 2024-08-20 00:56:38.422559113 +0000 UTC m=+0.421657312 container start e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0) 2024-08-20T00:56:38.678 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 ceph-mon[17715]: mkfs d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:38.678 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 ceph-mon[17715]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T00:56:38.678 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 bash[17696]: e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 2024-08-20T00:56:38.678 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:38 smithi138 systemd[1]: Started Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:56:39.705 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: monmap epoch 1 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: last_changed 2024-08-20T00:56:34.289119+0000 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: min_mon_release 19 (squid) 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: election_strategy: 1 2024-08-20T00:56:39.706 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:56:39.707 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: fsmap 2024-08-20T00:56:39.707 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T00:56:39.707 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: mgrmap e1: no daemons active 2024-08-20T00:56:39.707 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:39 smithi138 ceph-mon[17715]: from='client.? 172.21.15.138:0/3255136613' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-08-20T00:56:39.712 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout cluster: 2024-08-20T00:56:39.712 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout id: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:39.712 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-08-20T00:56:39.712 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:39.712 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout services: 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.626429s) 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout data: 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout pgs: 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:39.713 INFO:teuthology.orchestra.run.smithi138.stdout:mon is available 2024-08-20T00:56:39.714 INFO:teuthology.orchestra.run.smithi138.stdout:Assimilating anything we can from ceph.conf... 2024-08-20T00:56:40.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:40 smithi138 ceph-mon[17715]: from='client.? 172.21.15.138:0/429705968' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T00:56:40.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:40 smithi138 ceph-mon[17715]: from='client.? 172.21.15.138:0/429705968' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-08-20T00:56:40.878 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:40.878 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout fsid = d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.138:3300,v1:172.21.15.138:6789] 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T00:56:40.879 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T00:56:40.880 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:40.880 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T00:56:40.880 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T00:56:40.880 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T00:56:40.880 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T00:56:40.883 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T00:56:40.883 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T00:56:40.884 INFO:teuthology.orchestra.run.smithi138.stdout:Generating new minimal ceph.conf... 2024-08-20T00:56:41.750 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:41 smithi138 ceph-mon[17715]: from='client.? 172.21.15.138:0/109316652' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:56:42.037 INFO:teuthology.orchestra.run.smithi138.stdout:Restarting the monitor... 2024-08-20T00:56:42.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:42 smithi138 systemd[1]: Stopping Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:56:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:42 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a[17711]: 2024-08-20T00:56:42.387+0000 7f7025400640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T00:56:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:42 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a[17711]: 2024-08-20T00:56:42.387+0000 7f7025400640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-08-20T00:56:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:42 smithi138 podman[17956]: 2024-08-20 00:56:42.625631363 +0000 UTC m=+0.354328847 container died e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=) 2024-08-20T00:56:43.258 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 podman[17956]: 2024-08-20 00:56:43.026085907 +0000 UTC m=+0.754783373 container cleanup e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2) 2024-08-20T00:56:43.259 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 bash[17956]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a 2024-08-20T00:56:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 podman[17968]: 2024-08-20 00:56:43.260819662 +0000 UTC m=+0.622765244 container remove e7a45800ba49e91785e6d91198d66cfb1c38cf198e48bd4422818d184b6393a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-08-20T00:56:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-20T00:56:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 systemd[1]: Stopped Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:56:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service: Consumed 1.221s CPU time. 2024-08-20T00:56:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 systemd[1]: Starting Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:56:43.893 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 podman[18057]: 2024-08-20 00:56:43.65812383 +0000 UTC m=+0.027176464 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:56:43.893 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 podman[18057]: 2024-08-20 00:56:43.778552789 +0000 UTC m=+0.147605413 container create 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-20T00:56:44.097 INFO:teuthology.orchestra.run.smithi138.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-08-20T00:56:44.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:43 smithi138 podman[18057]: 2024-08-20 00:56:43.998765747 +0000 UTC m=+0.367818370 container init 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-08-20T00:56:44.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 podman[18057]: 2024-08-20 00:56:44.005353765 +0000 UTC m=+0.374406389 container start 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-20T00:56:44.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T00:56:44.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T00:56:44.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: pidfile_write: ignore empty --pid-file 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: load: jerasure load: lrc 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: RocksDB version: 7.9.2 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Git sha 0 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: DB SUMMARY 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: DB Session ID: CWVAYN62IFWU3O9VE2PT 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: CURRENT file: CURRENT 2024-08-20T00:56:44.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94455 ; 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.error_if_exists: 0 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.create_if_missing: 0 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.paranoid_checks: 1 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T00:56:44.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.env: 0x5596dd250be0 2024-08-20T00:56:44.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T00:56:44.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.info_log: 0x5596f64d3740 2024-08-20T00:56:44.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T00:56:44.295 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.statistics: (nil) 2024-08-20T00:56:44.295 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.use_fsync: 0 2024-08-20T00:56:44.295 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_log_file_size: 0 2024-08-20T00:56:44.295 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_fallocate: 1 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.use_direct_reads: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.db_log_dir: 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.wal_dir: 2024-08-20T00:56:44.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.write_buffer_manager: 0x5596f64d6e60 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T00:56:44.297 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.unordered_write: 0 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.row_cache: None 2024-08-20T00:56:44.298 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.wal_filter: None 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.two_write_queues: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.wal_compression: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.atomic_flush: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.log_readahead_size: 0 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T00:56:44.299 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_background_jobs: 2 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_background_compactions: -1 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_subcompactions: 1 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T00:56:44.300 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_open_files: -1 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_background_flushes: -1 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Compression algorithms supported: 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kZSTD supported: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kXpressCompression supported: 0 2024-08-20T00:56:44.301 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kBZip2Compression supported: 0 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kLZ4Compression supported: 1 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kZlibCompression supported: 1 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: kSnappyCompression supported: 1 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T00:56:44.302 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.merge_operator: 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_filter: None 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5596f64d3340) 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: index_type: 0 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: data_block_index_type: 0 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: index_shortening: 1 2024-08-20T00:56:44.303 INFO:journalctl@ceph.mon.a.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: checksum: 4 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: no_block_cache: 0 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_cache: 0x5596f64c7610 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_cache_options: 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: capacity : 536870912 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: num_shard_bits : 4 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: strict_capacity_limit : 0 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_cache_compressed: (nil) 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: persistent_cache: (nil) 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_size: 4096 2024-08-20T00:56:44.304 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_size_deviation: 10 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_restart_interval: 16 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: index_block_restart_interval: 1 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: metadata_block_size: 4096 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: partition_filters: 0 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: use_delta_encoding: 1 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: filter_policy: bloomfilter 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: whole_key_filtering: 1 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: verify_compression: 0 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: format_version: 5 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: enable_index_compression: 1 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: block_align: 0 2024-08-20T00:56:44.305 INFO:journalctl@ceph.mon.a.smithi138.stdout: max_auto_readahead_size: 262144 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout: prepopulate_block_cache: 0 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout: initial_auto_readahead_size: 8192 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression: NoCompression 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.num_levels: 7 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T00:56:44.306 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T00:56:44.307 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T00:56:44.308 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T00:56:44.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.inplace_update_support: 0 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T00:56:44.310 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.bloom_locality: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.max_successive_merges: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.ttl: 2592000 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T00:56:44.311 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enable_blob_files: false 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.min_blob_size: 0 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5eaea488-347b-4284-84d3-a9f8d5c74a21 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115404047445, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-08-20T00:56:44.312 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115404049102, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90685, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 280, "table_properties": {"data_size": 88726, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11486, "raw_average_key_size": 48, "raw_value_size": 82251, "raw_average_value_size": 344, "num_data_blocks": 12, "num_entries": 239, "num_filter_entries": 239, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724115404, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5eaea488-347b-4284-84d3-a9f8d5c74a21", "db_session_id": "CWVAYN62IFWU3O9VE2PT", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115404049277, "job": 1, "event": "recovery_finished"} 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5596f64f8e00 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: DB pointer 0x5596f65f6000 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout: ** DB Stats ** 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:56:44.313 INFO:journalctl@ceph.mon.a.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: ** Compaction Stats [default] ** 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: L0 2/0 90.42 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Sum 2/0 90.42 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 77.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: ** Compaction Stats [default] ** 2024-08-20T00:56:44.314 INFO:journalctl@ceph.mon.a.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 77.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: Cumulative compaction: 0.00 GB write, 15.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:56:44.315 INFO:journalctl@ceph.mon.a.smithi138.stdout: Interval compaction: 0.00 GB write, 15.73 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: Block cache BinnedLRUCache@0x5596f64c7610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: starting mon.a rank 0 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???) e1 preinit fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).mds e1 new map 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).mds e1 print_map 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: e1 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: btime 2024-08-20T00:56:38:482793+0000 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T00:56:44.316 INFO:journalctl@ceph.mon.a.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout: legacy client fscid: -1 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout: 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout: No filesystems configured 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 bash[18057]: 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 systemd[1]: Started Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mon.a is new leader, mons a in quorum (ranks 0) 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: monmap epoch 1 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:44.317 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: last_changed 2024-08-20T00:56:34.289119+0000 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: min_mon_release 19 (squid) 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: election_strategy: 1 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: fsmap 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: osdmap e1: 0 total, 0 up, 0 in 2024-08-20T00:56:44.318 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:44 smithi138 ceph-mon[18076]: mgrmap e1: no daemons active 2024-08-20T00:56:45.292 INFO:teuthology.orchestra.run.smithi138.stdout:Wrote config to /etc/ceph/ceph.conf 2024-08-20T00:56:45.320 INFO:teuthology.orchestra.run.smithi138.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-08-20T00:56:45.320 INFO:teuthology.orchestra.run.smithi138.stdout:Creating mgr... 2024-08-20T00:56:45.321 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying port 0.0.0.0:9283 ... 2024-08-20T00:56:45.321 INFO:teuthology.orchestra.run.smithi138.stdout:Verifying port 0.0.0.0:8765 ... 2024-08-20T00:56:45.666 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x 2024-08-20T00:56:45.666 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x.service: Unit ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x.service not loaded. 2024-08-20T00:56:45.899 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968.target.wants/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@.service. 2024-08-20T00:56:45.929 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1137480094' entity='client.admin' 2024-08-20T00:56:46.241 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:45 smithi138 systemd[1]: Starting Ceph mgr.x for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:56:46.491 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 podman[18285]: 2024-08-20 00:56:46.146309795 +0000 UTC m=+0.026821061 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T00:56:46.491 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 podman[18285]: 2024-08-20 00:56:46.267819983 +0000 UTC m=+0.148331249 container create 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T00:56:46.491 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 podman[18285]: 2024-08-20 00:56:46.487092638 +0000 UTC m=+0.367603904 container init 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T00:56:46.491 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 podman[18285]: 2024-08-20 00:56:46.493112399 +0000 UTC m=+0.373623673 container start 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T00:56:46.629 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T00:56:46.630 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout disabled 2024-08-20T00:56:46.644 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T00:56:46.644 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:56:46.644 INFO:teuthology.orchestra.run.smithi138.stdout:firewalld.service is not enabled 2024-08-20T00:56:46.644 INFO:teuthology.orchestra.run.smithi138.stdout:Not possible to enable service . firewalld.service is not available 2024-08-20T00:56:46.661 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T00:56:46.661 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout disabled 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:firewalld.service is not enabled 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mgr to start... 2024-08-20T00:56:46.675 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mgr... 2024-08-20T00:56:46.790 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 bash[18285]: 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c 2024-08-20T00:56:46.790 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:46 smithi138 systemd[1]: Started Ceph mgr.x for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:56:47.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/217354964' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T00:56:47.985 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 0 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T00:56:47.986 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T00:56:47.987 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T00:56:47.990 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.990 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T00:56:38:482793+0000", 2024-08-20T00:56:47.991 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T00:56:47.992 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T00:56:38.483583+0000", 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:56:47.993 INFO:teuthology.orchestra.run.smithi138.stdout:mgr not available, waiting (1/15)... 2024-08-20T00:56:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1468431660' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T00:56:51.300 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:51.300 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 0 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T00:56:51.301 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T00:56:51.302 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.303 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T00:56:38:482793+0000", 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "available": false, 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T00:56:51.304 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T00:56:38.483583+0000", 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:56:51.305 INFO:teuthology.orchestra.run.smithi138.stdout:mgr not available, waiting (2/15)... 2024-08-20T00:56:53.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: Activating manager daemon x 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: mgrmap e2: x(active, starting, since 0.00419086s) 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: Manager daemon x is now available 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' 2024-08-20T00:56:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:52 smithi138 ceph-mon[18076]: from='mgr.14100 172.21.15.138:0/315214521' entity='mgr.x' 2024-08-20T00:56:54.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:54 smithi138 ceph-mon[18076]: mgrmap e3: x(active, since 1.00942s) 2024-08-20T00:56:54.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:54 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2868796391' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "health": { 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.723 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 0 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "a" 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "monmap": { 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-08-20T00:56:54.724 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-08-20T00:56:54.725 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-08-20T00:56:54.728 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.728 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-08-20T00:56:54.728 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-08-20T00:56:54.728 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-08-20T00:56:54.728 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.729 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "btime": "2024-08-20T00:56:38:482793+0000", 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.730 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modules": [ 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "iostat", 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "nfs", 2024-08-20T00:56:54.731 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "restful" 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ], 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "modified": "2024-08-20T00:56:38.483583+0000", 2024-08-20T00:56:54.732 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "services": {} 2024-08-20T00:56:54.733 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout }, 2024-08-20T00:56:54.733 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-08-20T00:56:54.733 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:56:54.733 INFO:teuthology.orchestra.run.smithi138.stdout:mgr is available 2024-08-20T00:56:55.773 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:55 smithi138 ceph-mon[18076]: mgrmap e4: x(active, since 2s) 2024-08-20T00:56:55.773 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:55 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1426905113' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-08-20T00:56:56.064 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:56.064 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [global] 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout fsid = d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.138:3300,v1:172.21.15.138:6789] 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [mgr] 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout [osd] 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-08-20T00:56:56.065 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-08-20T00:56:56.066 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-08-20T00:56:56.066 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-08-20T00:56:56.066 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-08-20T00:56:56.066 INFO:teuthology.orchestra.run.smithi138.stdout:Enabling cephadm module... 2024-08-20T00:56:57.219 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:56 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1923310056' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-08-20T00:56:57.219 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:56 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setuser ceph since I am not root 2024-08-20T00:56:57.220 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:56:56 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setgroup ceph since I am not root 2024-08-20T00:56:58.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:57 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1923310056' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-08-20T00:56:58.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:57 smithi138 ceph-mon[18076]: mgrmap e5: x(active, since 4s) 2024-08-20T00:56:58.998 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:56:58 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1342630827' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for the mgr to restart... 2024-08-20T00:56:59.006 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mgr epoch 5... 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: Active manager daemon x restarted 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: Activating manager daemon x 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: osdmap e2: 0 total, 0 up, 0 in 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: mgrmap e6: x(active, starting, since 0.00375277s) 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:57:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:02 smithi138 ceph-mon[18076]: Manager daemon x is now available 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:mgr epoch 5 is available 2024-08-20T00:57:04.561 INFO:teuthology.orchestra.run.smithi138.stdout:Setting orchestrator backend to cephadm... 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: Found migration_current of "None". Setting to last migration. 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:04.862 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: mgrmap e7: x(active, since 1.00938s) 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-20T00:57:04.863 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-mon[18076]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-08-20T00:57:05.133 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:57:04.863+0000 7f1437a00640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:00:57:04] ENGINE Error in HTTPServer.serve 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Traceback (most recent call last): 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._connections.run(self.expiration_interval) 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._run(expiration_interval) 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:57:05.134 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s = self.context.wrap_socket( 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: return self.sslsocket_class._create( 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self.do_handshake() 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._sslobj.do_handshake() 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:57:05.135 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:04] ENGINE Bus STARTING 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:04] ENGINE Serving on http://172.21.15.138:8765 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:04] ENGINE Serving on https://172.21.15.138:7150 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:04] ENGINE Bus STARTED 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:04] ENGINE Error in HTTPServer.serve 2024-08-20T00:57:05.963 INFO:journalctl@ceph.mon.a.smithi138.stdout: Traceback (most recent call last): 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._run(expiration_interval) 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: s = self.context.wrap_socket( 2024-08-20T00:57:05.964 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: return self.sslsocket_class._create( 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: self.do_handshake() 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._sslobj.do_handshake() 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:05.965 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:05 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:06.976 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:06 smithi138 ceph-mon[18076]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:06.976 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:06 smithi138 ceph-mon[18076]: mgrmap e8: x(active, since 2s) 2024-08-20T00:57:07.242 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout value unchanged 2024-08-20T00:57:07.243 INFO:teuthology.orchestra.run.smithi138.stdout:Generating ssh key... 2024-08-20T00:57:08.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:07 smithi138 ceph-mon[18076]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:08.520 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Generating public/private rsa key pair. 2024-08-20T00:57:08.520 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Your identification has been saved in /tmp/tmp7b7fwu85/key 2024-08-20T00:57:08.520 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Your public key has been saved in /tmp/tmp7b7fwu85/key.pub 2024-08-20T00:57:08.520 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: The key fingerprint is: 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: SHA256:QxhoIgYsHaGYkPrTIc4n2F3skmIGXAFp+dLmtTITU3c ceph-d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: The key's randomart image is: 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: +---[RSA 3072]----+ 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: |=+*+... | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: |*Oo.o. + E | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: |X.+o..o o | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: |oo.*..o. | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: | B+++=. S | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: |. @=*.. . | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: | o =+. | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: | | 2024-08-20T00:57:08.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: | | 2024-08-20T00:57:08.522 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: +----[SHA256]-----+ 2024-08-20T00:57:09.164 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:09 smithi138 ceph-mon[18076]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:09.164 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:09 smithi138 ceph-mon[18076]: Generating ssh key... 2024-08-20T00:57:09.165 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:09 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:09.165 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:09 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:10.228 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:10 smithi138 ceph-mon[18076]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:10.233 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:57:10.233 INFO:teuthology.orchestra.run.smithi138.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-08-20T00:57:10.234 INFO:teuthology.orchestra.run.smithi138.stdout:Adding key to root@localhost authorized_keys... 2024-08-20T00:57:10.245 INFO:teuthology.orchestra.run.smithi138.stdout:Adding host smithi138... 2024-08-20T00:57:11.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:11 smithi138 ceph-mon[18076]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "addr": "172.21.15.138", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:13.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:13 smithi138 ceph-mon[18076]: Deploying cephadm binary to smithi138 2024-08-20T00:57:15.468 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout Added host 'smithi138' with addr '172.21.15.138' 2024-08-20T00:57:15.468 INFO:teuthology.orchestra.run.smithi138.stdout:Deploying unmanaged mon service... 2024-08-20T00:57:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:15 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:16.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:15 smithi138 ceph-mon[18076]: Added host smithi138 2024-08-20T00:57:16.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:15 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:16.898 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-08-20T00:57:16.898 INFO:teuthology.orchestra.run.smithi138.stdout:Deploying unmanaged mgr service... 2024-08-20T00:57:17.471 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:17 smithi138 ceph-mon[18076]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:17.471 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:17 smithi138 ceph-mon[18076]: Saving service mon spec with placement count:5 2024-08-20T00:57:17.471 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:17 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:19.439 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:19 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:19.439 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:19 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:19.439 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:19 smithi138 ceph-mon[18076]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:19.439 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:19 smithi138 ceph-mon[18076]: Saving service mgr spec with placement count:2 2024-08-20T00:57:19.440 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:19 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:19.444 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-08-20T00:57:21.381 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:21 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/72736925' entity='client.admin' 2024-08-20T00:57:21.382 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:21 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:22.767 INFO:teuthology.orchestra.run.smithi138.stdout:Enabling the dashboard module... 2024-08-20T00:57:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:23 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2635789895' entity='client.admin' 2024-08-20T00:57:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:23 smithi138 ceph-mon[18076]: from='mgr.14118 172.21.15.138:0/3872177437' entity='mgr.x' 2024-08-20T00:57:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:23 smithi138 ceph-mon[18076]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:24.521 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setuser ceph since I am not root 2024-08-20T00:57:24.522 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setgroup ceph since I am not root 2024-08-20T00:57:24.522 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:24 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/260677809' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-08-20T00:57:25.174 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:25 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/260677809' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-08-20T00:57:25.175 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:25 smithi138 ceph-mon[18076]: mgrmap e9: x(active, since 21s) 2024-08-20T00:57:26.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:26 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2390754819' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "available": true, 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for the mgr to restart... 2024-08-20T00:57:26.679 INFO:teuthology.orchestra.run.smithi138.stdout:Waiting for mgr epoch 9... 2024-08-20T00:57:30.658 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: Active manager daemon x restarted 2024-08-20T00:57:30.658 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: Activating manager daemon x 2024-08-20T00:57:30.658 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: osdmap e3: 0 total, 0 up, 0 in 2024-08-20T00:57:30.658 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: mgrmap e10: x(active, starting, since 0.00459999s) 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:57:30.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:30 smithi138 ceph-mon[18076]: Manager daemon x is now available 2024-08-20T00:57:31.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:31.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:57:31.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:57:31.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-mon[18076]: mgrmap e11: x(active, since 1.00908s) 2024-08-20T00:57:31.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-mon[18076]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:57:31.644+0000 7f59bbc00640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:00:57:31] ENGINE Error in HTTPServer.serve 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Traceback (most recent call last): 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._connections.run(self.expiration_interval) 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._run(expiration_interval) 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:57:32.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s = self.context.wrap_socket( 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: return self.sslsocket_class._create( 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self.do_handshake() 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._sslobj.do_handshake() 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:57:32.042 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:57:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout { 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout "initialized": true 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout } 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:mgr epoch 9 is available 2024-08-20T00:57:32.135 INFO:teuthology.orchestra.run.smithi138.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-08-20T00:57:32.381 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:32 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:31] ENGINE Bus STARTING 2024-08-20T00:57:32.381 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:32 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:31] ENGINE Serving on https://172.21.15.138:7150 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:32 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:31] ENGINE Error in HTTPServer.serve 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: Traceback (most recent call last): 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._run(expiration_interval) 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:57:32.382 INFO:journalctl@ceph.mon.a.smithi138.stdout: s = self.context.wrap_socket( 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: return self.sslsocket_class._create( 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: self.do_handshake() 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._sslobj.do_handshake() 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:32 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:31] ENGINE Serving on http://172.21.15.138:8765 2024-08-20T00:57:32.383 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:32 smithi138 ceph-mon[18076]: [20/Aug/2024:00:57:31] ENGINE Bus STARTED 2024-08-20T00:57:33.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:33 smithi138 ceph-mon[18076]: mgrmap e12: x(active, since 2s) 2024-08-20T00:57:33.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:33 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:33.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:33 smithi138 ceph-mon[18076]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:34.427 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIFETCCAvmgAwIBAgIUEwr0h0++6kJK8qhiEThDXi/MffYwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDgyMDAwNTczM1oXDTM0\nMDgyMTAwNTczM1owGDEWMBQGA1UEAwwNMTcyLjIxLjE1LjEzODCCAiIwDQYJKoZI\nhvcNAQEBBQADggIPADCCAgoCggIBAOCt07OzA8VAC+CMTn/fbWuXl4EN8Nf2iT1M\niszIyTsKyrolKZa14xinOUCcshuZSCpxnK0AYWvVUXiCtj6ucOxNanRP9632G1qM\nvnz6G/skDVHM2rooJA28rNA2UBtN5bd98OJs+dC1U9eR5gd4tlKgjCqXhBBLOS7d\nGQYMxE50tDt0sc+LM5z6hBUOjFX5EaLiYN2RtehCRfQ6EGShacTiEWmAbWUiB8Bd\nWXGIa3pzwL51Ew48WvY1m4QRsj0NGpEGe+2gAvA+bnERtmsVObp5EeeTmuqU1suk\ncyJU3T1Wy8Zi/HTtz+IeoKPA8W6BqSLQIkUex8ml5lJ87O9o3o6ZtYmYRq5Q+jtX\nUN7BRU3wmXshZOV3KtISgqMWNUD5luLT2GyQVj8hAqI0avmO5RC4fbhK7eAz2YYO\nPw5feYcVXIYmTQi85ZCtWQgs4XeNXsIln64T1SfVfBj9iw7xEgswFHUN9c/DNQsv\nbzmIX4DVNYWd1bVwtVKTUWqsxY1qWRlkomuJYEoovLqj5KFfAyYckXT69fBlUcCV\nlotKtvn56OYGHJzCAjFLUDfxeh29/iZUuLTl+bXGC7RWPmCKOpKJR2RpbDrAHXTh\nVVZVQ5lmmP+wa+cG/e8tNbz/6RUvN/hHNF269dIhq9oB8bjpdUgJCGAn4JOm9VWC\nWawiNCyLAgMBAAGjVDBSMEIGA1UdEQQ7MDmCHnNtaXRoaTEzOC5mcm9udC5zZXBp\nYS5jZXBoLmNvbYIRZGFzaGJvYXJkX3NlcnZlcnOHBKwVD4owDAYDVR0TAQH/BAIw\nADANBgkqhkiG9w0BAQsFAAOCAgEAeyL9uduUQdIiF6nMLXKlop0uNivueiWRsPvi\n2Z1X+5RFgG6ghqLoEG4g0KkX7mknz/Q2UKaLHPIfBrAjAGOO+HJH4uVpzHh8T8sW\nERnwHGpSrNYzexMhaxRnETql99C7l3y+bvObyNUiOqGx4C9S1eRdqBolWTdwJE2o\n0HW99W0lP2/j1MYBE2R+pfZCNvCC9dIEFi71R51N2wXQLP5nS4iGXQDspDLgs5YR\n/3OeY2WhazErH//qw7g9fUyGYVhf9i5zem6SkxTdNS9hpbpzbUJ2B/6hIDNGFbQQ\nHyuvoEz97609zuO6tfpwvxBHLJYSYpDtUh9DUBUatG4pWYrGmfWrK1o7ehIMeKWw\nosSXlVIqFC+Eq0bGsclz0UDc//f6MSDSeH25co2ru/tGnWsqsAiwuiawCrZrmsZB\nMuroCQst8IQSq3GeVRRjKT+1iiLwKLb5rG/g39tNu0V5dclvP5R5kzwzYfgXSRgs\n3zKLPqgjw0xC+HsuJILjthcO0ztoKJRACLWO7plGZRZfgS6ewN8i3iVUcfzLUFn8\n2czjGgh81Yb6o7Fh5Jf126PXF5N32ssEqgHQV8mZiYmg2sHEqN8bNCzu7LOCLoRn\nvwa0V19TNz0Qqx4jzajgD/I+JjfjbQe2QE2ZXUTn/BwRrWyYsLRZmROKzSb7rCq+\n6w8Msok=\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEA4K3Ts7MDxUAL4IxOf99ta5eXgQ3w1/aJPUyKzMjJOwrKuiUp\nlrXjGKc5QJyyG5lIKnGcrQBha9VReIK2Pq5w7E1qdE/3rfYbWoy+fPob+yQNUcza\nuigkDbys0DZQG03lt33w4mz50LVT15HmB3i2UqCMKpeEEEs5Lt0ZBgzETnS0O3Sx\nz4sznPqEFQ6MVfkRouJg3ZG16EJF9DoQZKFpxOIRaYBtZSIHwF1ZcYhrenPAvnUT\nDjxa9jWbhBGyPQ0akQZ77aAC8D5ucRG2axU5unkR55Oa6pTWy6RzIlTdPVbLxmL8\ndO3P4h6go8DxboGpItAiRR7HyaXmUnzs72jejpm1iZhGrlD6O1dQ3sFFTfCZeyFk\n5Xcq0hKCoxY1QPmW4tPYbJBWPyECojRq+Y7lELh9uErt4DPZhg4/Dl95hxVchiZN\nCLzlkK1ZCCzhd41ewiWfrhPVJ9V8GP2LDvESCzAUdQ31z8M1Cy9vOYhfgNU1hZ3V\ntXC1UpNRaqzFjWpZGWSia4lgSii8uqPkoV8DJhyRdPr18GVRwJWWi0q2+fno5gYc\nnMICMUtQN/F6Hb3+JlS4tOX5tcYLtFY+YIo6kolHZGlsOsAddOFVVlVDmWaY/7Br\n5wb97y01vP/pFS83+Ec0Xbr10iGr2gHxuOl1SAkIYCfgk6b1VYJZrCI0LIsCAwEA\nAQKCAgAhW7hRN49OwP9CcUfSaoIyIElpQCr1wEUzabg4nB3B0LdgGIEVPvTJxezq\nhUQDHv3890NTgvlJQQcd77ej6ihv8oqPXF7MtaaYCh+E1L7vdF8vOaLGqDZk0meM\nvmj5tNYaKUO+NSneF/NWep3+6/2TRQJVQeEPaOvDjnuBpMbx/cJ+CrRqY84092js\nQMMLR2Cb9iUloIBeoesFo31S7jg9oscCYAQIqPB3r3nGhoJWpaBaEW3T25gxyAHK\nrl9FUJBqNIk/3asHWm2cB2I11z/r9L/OHzc4BFiOgbW/xNPYS0O+tRcAkV0fwyZc\nnxqkKpzJw2yN0KUfG6ztAF6j/nD/JIzFwoHE6hL1E8Q7abSlCzj/fIDNIDYHs1TG\ne7KGNvBvRH892b71ExljXgjjZnv+YIbTf1fYVI9PpfddxWhbV3s5DZeBFSgPBWbv\nOmrLbov+J/CeKm9TS0C1gfCuckeB+X6NSc3YTp5r+2DnzNn6z04PrKfZeCJek01x\nIE+kndQ2D0ql8Vwv7YGUv4EIZcR1eqsGnAwETECPOS8jQR5HQR5zn9P1KAoeo/ZI\nrZcFwXuMj9+VZS6AJSt5G7Tiuy2MK39vTLWPKYINyCrnsozg32AtT6wmzaoc/twS\nPZ51tlp5BrU+2yj31/WZ4/y3GP1hshX8yZtmihYOaCg93nC+eQKCAQEA9Y4tlykq\nPyrzb5yA/7GL1fzFteWI0dV0RCtj7doQnIZA6iNJ6434eeDWulOBZqKu1/TYC+ZT\nKUdt3uDz65OC6Wm4F1lupCuLMSubAwUYEir4viHjN8piLmIsNMJhQKv5g8PRirJZ\nNqca4fUa1V8ZAdyIKAQBMPb45PWE+08cVIf+NrCOqcx+a+HxO9RlKQkACfwLsyzp\nsqrSshf7o6y8D9WcHkLK9z7wWfqzetXgS78Ob9sq1OrGpauGoNTU1FUUSu3yF5AO\nasRXG/NJKP3rmTfpm1qdTPPGXsgHpR/VjYxp/6C4Z99C1KvnCpnhUbn2R3rsEf6i\n1k2brGee2DUKlQKCAQEA6jxUI6VOOhoFqkOHZ9w/Vy8p5XXvcZHhbD8d9c6F8eFv\nxNNojF0I1TbLiTRCDh+mEPuMsQ32JVG+qgNi6jw0N4/48p5/1MP6seBStFg09wVk\ntfy93Hw1SUOYwJaBQN2B5Emnpsld28nG5ZM5JYqTBurnKn+2MkKCrg0z/OpheKYF\n34PfzWBjaUGLfUHQSS2WBlzdaJXhVxBSP57kiaWmohdxjjL7lJESh9gDXbv/Rb/F\nljs2J4MHeuzozuOIAgvaGzXYrWl3nJ+itzybm/eSjnaUoEa+v1JBne39pMj3zYOv\n2sDrGGwwMaTO6B7NDOSLutfcKK4drW28Uhv0FYuynwKCAQEA8Dsstxn6ztnCV4sR\nWRXl7G/jLVVmSYM+wAKDPQt9/OQLVgU4KE84rGQfLyLLEhJOY9CSHPI0ocnT35zM\nhchG5vwash04dTx0bKJwgL/XEM8hk0Gr/cTt0NsSzHrukkqtTQ+iTMK3LRTG5BDb\nM6xZ1sxeGGfykmjMj5LzirmWycxSFEcfC1eYxwrA6vucDE1V0G7ZM23ZJIaiosF7\n2mD/i647kg9eag0VL7JpLN9ZFEHSafRNLwiRVzs79tTBcwnZVIT9t7ejoyh64Elf\nZc8lUzy0Otynz8UeaVPgxtciId0hRomCX3y48T9MRb7OBRqdqsjDc8den7JFfqHR\ncUxTEQKCAQBXcaIvlHy5sBFBWOhp241B7PeFFBwJl95B3OZAZ5MVk4V4Qzwaenjb\nsBMX64xlXWdPm7kQOBgWIog9S/pXW5Zaz8KuQIGUWn1yfj4h2XQca8UHM41ljHEQ\nIs4Zx5YFMyxUnzKZTz9c190Kw4VoAG2W/YbDFN5dsPq8I22vE4jD06cJpF+eekdi\neferETDCwOPrCCwX6SwwpoUt/xFuWYr1pC7eWSYWdnomgHvxOEe6MDeBOCWlZoMa\nXusioUqbCPWphtaXFnJy5B2IrnWhKjFqQdaGZyK6peidHsW8CNKJxImX7sFVYr4z\nKkWb3jPKhl+rpv4nqaimn/rX5Fug14AdAoIBAQDvMFjI2LQCufv3qRKwnaVnmFKX\nILs3HlLEiRe39yD0Tz5J+CxbbE8RhYBSNH/gL7OJ8ZmCERbU4cd44AuyhPZ3E95Z\nPIzDMMKfhcKFb87SBL6CqFpCWEY/SAnO47Er04/7ct7/WxcfmNe7T42UZdFRC5FM\nWL5N2vdldRDnM1EeqsbHRjcwPvOc+bsoLuVXUs+c67r2DIA/LILF0ONohcj/SXd6\nHRLz4pjNAoj75j7me6Z5X9IDh3MemcH+pOeuZO+N2GclbbN2tW8yRhjWLcgMxIH6\nnrAPIJ5h8BaV51fUiHoyYdji1N8RFQn8mdAuWMQIWR5ri56/Pp8TSnxwLz6R\n-----END RSA PRIVATE KEY-----\n"} 2024-08-20T00:57:35.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:35 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:36.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:36 smithi138 ceph-mon[18076]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:36.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:36 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:36.353 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-08-20T00:57:37.947 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-08-20T00:57:37.947 INFO:teuthology.orchestra.run.smithi138.stdout:Creating initial admin user... 2024-08-20T00:57:38.273 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:38 smithi138 ceph-mon[18076]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:38.273 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:38 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:39.554 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$H1s/g1KpL4GDl.uyMcFqHOm4zAzpP.CLYzwsDyeDwDtNlA9jEre0W", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1724115458, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-08-20T00:57:39.554 INFO:teuthology.orchestra.run.smithi138.stdout:Fetching dashboard port number... 2024-08-20T00:57:40.237 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:39 smithi138 ceph-mon[18076]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:40.237 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:39 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:40.895 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stdout 8443 2024-08-20T00:57:40.914 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-08-20T00:57:40.914 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout disabled 2024-08-20T00:57:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-08-20T00:57:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:systemctl: stdout inactive 2024-08-20T00:57:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:firewalld.service is not enabled 2024-08-20T00:57:40.928 INFO:teuthology.orchestra.run.smithi138.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout:Ceph Dashboard is now available at: 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout: URL: https://smithi138.front.sepia.ceph.com:8443/ 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout: User: admin 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout: Password: j36m8534cl 2024-08-20T00:57:40.931 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:40.932 INFO:teuthology.orchestra.run.smithi138.stdout:Saving cluster configuration to /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config directory 2024-08-20T00:57:41.236 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:40 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2145556204' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout:Or, if you are only running a single cluster on this host: 2024-08-20T00:57:42.441 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout:Please consider enabling telemetry to help improve Ceph: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: ceph telemetry on 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout:For more information see: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:42.442 INFO:teuthology.orchestra.run.smithi138.stdout:Bootstrap complete. 2024-08-20T00:57:42.499 INFO:tasks.cephadm:Fetching config... 2024-08-20T00:57:42.500 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:57:42.500 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-08-20T00:57:42.535 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-08-20T00:57:42.536 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:57:42.536 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-08-20T00:57:42.613 INFO:tasks.cephadm:Fetching mon keyring... 2024-08-20T00:57:42.614 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:57:42.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/keyring of=/dev/stdout 2024-08-20T00:57:42.709 INFO:tasks.cephadm:Fetching pub ssh key... 2024-08-20T00:57:42.709 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:57:42.709 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-08-20T00:57:42.785 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-08-20T00:57:42.785 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T00:57:42.909 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:57:42.935 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T00:57:42.977 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:57:42.989 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-08-20T00:57:43.012 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:42 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/423929418' entity='client.admin' 2024-08-20T00:57:43.030 INFO:teuthology.orchestra.run.smithi167.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UsBZiP/ZQc0W0Ao8Ujcawr96eXhZGQc6+Acem40VsRfom8cKCcuPye1xXCCk1UrcdFl7HfwaLzJSvxkWBbFjKPozWPlnBzKsxx4Qa+m2+dMrkQAPfclMYpWicpJV98ySs9X03pJGP2ISwSqSUAFuuuuY9potJiSHpJrOTkhNHXEZbJedUNbOJ0voE6rVSyQD9KToASsE+be9/mQdGyLLBfgv5/eRKqz5qfCXM6OYGS+6g4jkVPY0XdizBktAiZmGJr2AmVm7VHyx1sXBYngbp5P9+y7fmI0sQ/tmcRq0WTIaMDn/8RUoXmBIPemd1Ap/JmDNAk22C6YQ/3P7+mOj8fj0MTD+JlBtYrZkJBUGVQSUN1o+c/1+9RlCiDELWBx9Q8DJPfHkmGIyYlZFmsL+rXqGDgPfqloOuD6CDwj/I47f+E5CpXTFyTGAKETXp74JEzPT9tANrDz0nK/kQgn9DclfwYd5ME6YOtHJNlFzbCiDgZT5JMEqP2NZL15hsaE= ceph-d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:57:43.043 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-08-20T00:57:43.412 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:57:43.841 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:43 smithi138 ceph-mon[18076]: mgrmap e13: x(active, since 12s) 2024-08-20T00:57:45.025 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-08-20T00:57:45.025 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-08-20T00:57:45.243 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3443879746' entity='client.admin' 2024-08-20T00:57:45.511 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:57:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:57:46.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:46.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:46.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:46 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:47.243 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-08-20T00:57:47.243 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:57:47.243 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-08-20T00:57:47.263 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:57:47.263 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:57:47.320 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-08-20T00:57:47.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch host add smithi142 2024-08-20T00:57:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:57:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:57:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:57:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:47 smithi138 ceph-mon[18076]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:57:48.198 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:57:48.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:48 smithi138 ceph-mon[18076]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:57:48.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:48 smithi138 ceph-mon[18076]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:57:48.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:48 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:48.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:48 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:48.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:48 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:49 smithi138 ceph-mon[18076]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:50 smithi138 ceph-mon[18076]: Deploying cephadm binary to smithi142 2024-08-20T00:57:50.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:50 smithi138 ceph-mon[18076]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:52.143 INFO:teuthology.orchestra.run.smithi138.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-08-20T00:57:52.869 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T00:57:53.229 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:57:53.476 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:53.476 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: Added host smithi142 2024-08-20T00:57:53.476 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:53.476 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:53.476 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:53.477 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:53 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:54.001 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:57:54.001 INFO:teuthology.orchestra.run.smithi138.stdout:[{"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-08-20T00:57:54.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:54 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:54.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:54 smithi138 ceph-mon[18076]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T00:57:54.620 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:54 smithi138 ceph-mon[18076]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:54.657 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi167 2024-08-20T00:57:54.658 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:57:54.658 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.conf 2024-08-20T00:57:54.678 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:57:54.678 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:57:54.736 INFO:tasks.cephadm:Adding host smithi167 to orchestrator... 2024-08-20T00:57:54.736 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch host add smithi167 2024-08-20T00:57:55.089 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:57:57.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:57.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:57 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:58.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:58 smithi138 ceph-mon[18076]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:57:58.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:58 smithi138 ceph-mon[18076]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:57:58.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:58 smithi138 ceph-mon[18076]: Deploying cephadm binary to smithi167 2024-08-20T00:57:58.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:58 smithi138 ceph-mon[18076]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:57:59.002 INFO:teuthology.orchestra.run.smithi138.stdout:Added host 'smithi167' with addr '172.21.15.167' 2024-08-20T00:57:59.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:59 smithi138 ceph-mon[18076]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:57:59.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:59 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:57:59.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:57:59 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:57:59.700 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch host ls --format=json 2024-08-20T00:58:00.055 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:00.479 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:00 smithi138 ceph-mon[18076]: Added host smithi167 2024-08-20T00:58:00.479 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:00 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:00.479 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:00 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:00.786 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:58:00.786 INFO:teuthology.orchestra.run.smithi138.stdout:[{"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}, {"addr": "172.21.15.167", "hostname": "smithi167", "labels": [], "status": ""}] 2024-08-20T00:58:01.325 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:01 smithi138 ceph-mon[18076]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:01.325 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:01 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:01.326 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:01 smithi138 ceph-mon[18076]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T00:58:01.362 INFO:tasks.cephadm:Setting crush tunables to default 2024-08-20T00:58:01.362 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd crush tunables default 2024-08-20T00:58:01.714 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:02.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:02 smithi138 ceph-mon[18076]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:03.291 INFO:teuthology.orchestra.run.smithi138.stderr:adjusted tunables profile to default 2024-08-20T00:58:03.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T00:58:03.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:03.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:03.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:03.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:03.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:03.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:03.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:03.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:03 smithi138 ceph-mon[18076]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:04.006 INFO:tasks.cephadm:Adding mon.a on smithi138 2024-08-20T00:58:04.007 INFO:tasks.cephadm:Adding mon.b on smithi142 2024-08-20T00:58:04.007 INFO:tasks.cephadm:Adding mon.c on smithi167 2024-08-20T00:58:04.007 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch apply mon '3;smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c' 2024-08-20T00:58:04.245 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:04 smithi138 ceph-mon[18076]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:04.871 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mon update... 2024-08-20T00:58:05.454 DEBUG:teuthology.orchestra.run.smithi142:mon.b> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.b.service 2024-08-20T00:58:05.456 DEBUG:teuthology.orchestra.run.smithi167:mon.c> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.c.service 2024-08-20T00:58:05.459 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T00:58:05.459 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: Saving service mon spec with placement smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c;count:3 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:06.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:06.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:06.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:05 smithi138 ceph-mon[18076]: Deploying daemon mon.c on smithi167 2024-08-20T00:58:06.361 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:07.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:06 smithi138 ceph-mon[18076]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:08.193 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:58:08.193 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":1,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","modified":"2024-08-20T00:56:34.289119Z","created":"2024-08-20T00:56:34.289119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-08-20T00:58:08.194 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 1 2024-08-20T00:58:08.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:08 smithi167 systemd[1]: Starting Ceph mon.c for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:58:08.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:08 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/180774585' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:09.036 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:08 smithi167 podman[21946]: 2024-08-20 00:58:08.833382326 +0000 UTC m=+0.015834476 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T00:58:09.037 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:08 smithi167 podman[21946]: 2024-08-20 00:58:08.934059029 +0000 UTC m=+0.116511170 container create f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-08-20T00:58:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 podman[21946]: 2024-08-20 00:58:09.136919997 +0000 UTC m=+0.319372133 container init f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-20T00:58:09.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 podman[21946]: 2024-08-20 00:58:09.140074327 +0000 UTC m=+0.322526469 container start f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: set uid:gid to 167:167 (ceph:ceph) 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev), process ceph-mon, pid 2 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pidfile_write: ignore empty --pid-file 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: load: jerasure load: lrc 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: RocksDB version: 7.9.2 2024-08-20T00:58:09.468 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Git sha 0 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: DB SUMMARY 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: DB Session ID: F626J69MWT0W69G3BPQ0 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: CURRENT file: CURRENT 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-20T00:58:09.469 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.error_if_exists: 0 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.create_if_missing: 0 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.paranoid_checks: 1 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T00:58:09.470 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.env: 0x559e7b396be0 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.info_log: 0x559e7d0e9700 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.statistics: (nil) 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.use_fsync: 0 2024-08-20T00:58:09.471 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_log_file_size: 0 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_fallocate: 1 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T00:58:09.472 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.use_direct_reads: 0 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.db_log_dir: 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.wal_dir: 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T00:58:09.473 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T00:58:09.474 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.write_buffer_manager: 0x559e7d0ece60 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T00:58:09.475 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.unordered_write: 0 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T00:58:09.476 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.row_cache: None 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.wal_filter: None 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.two_write_queues: 0 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T00:58:09.477 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.wal_compression: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.atomic_flush: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.log_readahead_size: 0 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T00:58:09.478 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T00:58:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T00:58:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T00:58:09.479 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T00:58:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T00:58:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T00:58:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_background_jobs: 2 2024-08-20T00:58:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_background_compactions: -1 2024-08-20T00:58:09.480 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_subcompactions: 1 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T00:58:09.481 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_open_files: -1 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T00:58:09.482 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_background_flushes: -1 2024-08-20T00:58:09.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Compression algorithms supported: 2024-08-20T00:58:09.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kZSTD supported: 0 2024-08-20T00:58:09.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kXpressCompression supported: 0 2024-08-20T00:58:09.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kBZip2Compression supported: 0 2024-08-20T00:58:09.483 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kLZ4Compression supported: 1 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kZlibCompression supported: 1 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: kSnappyCompression supported: 1 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-08-20T00:58:09.484 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.merge_operator: 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_filter: None 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T00:58:09.485 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559e7d0e9380) 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks: 1 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: pin_top_level_index_and_filter: 1 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_type: 0 2024-08-20T00:58:09.486 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_index_type: 0 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_shortening: 1 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: checksum: 4 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: no_block_cache: 0 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache: 0x559e7d0dd610 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_name: BinnedLRUCache 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_options: 2024-08-20T00:58:09.487 INFO:journalctl@ceph.mon.c.smithi167.stdout: capacity : 536870912 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_shard_bits : 4 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: strict_capacity_limit : 0 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: high_pri_pool_ratio: 0.000 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_cache_compressed: (nil) 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: persistent_cache: (nil) 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size: 4096 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_size_deviation: 10 2024-08-20T00:58:09.488 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_restart_interval: 16 2024-08-20T00:58:09.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: index_block_restart_interval: 1 2024-08-20T00:58:09.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: metadata_block_size: 4096 2024-08-20T00:58:09.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: partition_filters: 0 2024-08-20T00:58:09.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: use_delta_encoding: 1 2024-08-20T00:58:09.489 INFO:journalctl@ceph.mon.c.smithi167.stdout: filter_policy: bloomfilter 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: whole_key_filtering: 1 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: verify_compression: 0 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: read_amp_bytes_per_bit: 0 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: format_version: 5 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_index_compression: 1 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: block_align: 0 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: max_auto_readahead_size: 262144 2024-08-20T00:58:09.490 INFO:journalctl@ceph.mon.c.smithi167.stdout: prepopulate_block_cache: 0 2024-08-20T00:58:09.491 INFO:journalctl@ceph.mon.c.smithi167.stdout: initial_auto_readahead_size: 8192 2024-08-20T00:58:09.491 INFO:journalctl@ceph.mon.c.smithi167.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T00:58:09.491 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T00:58:09.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T00:58:09.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression: NoCompression 2024-08-20T00:58:09.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T00:58:09.492 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.num_levels: 7 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:58:09.493 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T00:58:09.494 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T00:58:09.495 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T00:58:09.496 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.inplace_update_support: 0 2024-08-20T00:58:09.497 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.bloom_locality: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.max_successive_merges: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.ttl: 2592000 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T00:58:09.498 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enable_blob_files: false 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.min_blob_size: 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e403a352-e98c-4fef-a998-cb178f42fc33 2024-08-20T00:58:09.499 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115489169117, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115489169539, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724115489, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e403a352-e98c-4fef-a998-cb178f42fc33", "db_session_id": "F626J69MWT0W69G3BPQ0", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115489169633, "job": 1, "event": "recovery_finished"} 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559e7d10ce00 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: DB pointer 0x559e7d20a000 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T00:58:09.500 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** DB Stats ** 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:09.501 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T00:58:09.502 INFO:journalctl@ceph.mon.c.smithi167.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** Compaction Stats [default] ** 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-08-20T00:58:09.503 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T00:58:09.504 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache BinnedLRUCache@0x559e7d0dd610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-08-20T00:58:09.505 INFO:journalctl@ceph.mon.c.smithi167.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: using public_addr v2:172.21.15.167:0/0 -> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: starting mon.c rank -1 at public addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] at bind addrs [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(???) e0 preinit fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:09.506 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).mds e1 new map 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).mds e1 print_map 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: e1 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: btime 2024-08-20T00:56:38:482793+0000 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: legacy client fscid: -1 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: 2024-08-20T00:58:09.507 INFO:journalctl@ceph.mon.c.smithi167.stdout: No filesystems configured 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:09.508 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.509 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Deploying cephadm binary to smithi167 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:09.510 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Added host smithi167 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.511 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:09.512 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.513 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Saving service mon spec with placement smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c;count:3 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: Deploying daemon mon.c on smithi167 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/180774585' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 ceph-mon[21965]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T00:58:09.514 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 bash[21946]: f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 2024-08-20T00:58:09.515 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:09 smithi167 systemd[1]: Started Ceph mon.c for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:58:09.751 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T00:58:09.752 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T00:58:10.006 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T00:58:14.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:14.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: mon.a calling monitor election 2024-08-20T00:58:14.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: mon.c calling monitor election 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: monmap epoch 2 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: last_changed 2024-08-20T00:58:09.198178+0000 2024-08-20T00:58:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: min_mon_release 19 (squid) 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: election_strategy: 1 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: fsmap 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: mgrmap e13: x(active, since 43s) 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: overall HEALTH_OK 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:14.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:14 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: mon.a calling monitor election 2024-08-20T00:58:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: mon.c calling monitor election 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: monmap epoch 2 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: last_changed 2024-08-20T00:58:09.198178+0000 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: min_mon_release 19 (squid) 2024-08-20T00:58:14.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: election_strategy: 1 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: fsmap 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: mgrmap e13: x(active, since 43s) 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: overall HEALTH_OK 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:14.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:14 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:15.066 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:58:15.066 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","modified":"2024-08-20T00:58:09.198178Z","created":"2024-08-20T00:56:34.289119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T00:58:15.066 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-08-20T00:58:15.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:15 smithi167 ceph-mon[21965]: Deploying daemon mon.b on smithi142 2024-08-20T00:58:15.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:15 smithi167 ceph-mon[21965]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:15.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:15 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/3286123419' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:15.357 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:15 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:15.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:15 smithi138 ceph-mon[18076]: Deploying daemon mon.b on smithi142 2024-08-20T00:58:15.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:15 smithi138 ceph-mon[18076]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:15.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:15 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/3286123419' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:15.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:15 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:16.540 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:16 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:58:16.197+0000 7f59fc200640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-08-20T00:58:16.664 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T00:58:16.665 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T00:58:16.920 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T00:58:17.362 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:17 smithi167 ceph-mon[21965]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:17.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:17 smithi138 ceph-mon[18076]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:17.602 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:58:17.603 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":2,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","modified":"2024-08-20T00:58:09.198178Z","created":"2024-08-20T00:56:34.289119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-08-20T00:58:17.603 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 2 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Git sha 0 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Compile date 2024-08-19 15:50:59 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: DB SUMMARY 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: DB Session ID: BCO7FBFU7GLYOO4DLJ5P 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: CURRENT file: CURRENT 2024-08-20T00:58:17.763 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: IDENTITY file: IDENTITY 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.error_if_exists: 0 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.create_if_missing: 0 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.paranoid_checks: 1 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.flush_verify_memtable_count: 1 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.env: 0x55738ce91be0 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.fs: PosixFileSystem 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.info_log: 0x55738dd7d700 2024-08-20T00:58:17.764 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_file_opening_threads: 16 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.statistics: (nil) 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.use_fsync: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_log_file_size: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.log_file_time_to_roll: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.keep_log_file_num: 1000 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.recycle_log_file_num: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_fallocate: 1 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_mmap_reads: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_mmap_writes: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.use_direct_reads: 0 2024-08-20T00:58:17.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.create_missing_column_families: 0 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.db_log_dir: 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.wal_dir: 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.table_cache_numshardbits: 6 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.WAL_ttl_seconds: 0 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.WAL_size_limit_MB: 0 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.is_fd_close_on_exec: 1 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.advise_random_on_open: 1 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.db_write_buffer_size: 0 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.write_buffer_manager: 0x55738dd80e60 2024-08-20T00:58:17.766 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.use_adaptive_mutex: 0 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.rate_limiter: (nil) 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.wal_recovery_mode: 2 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enable_thread_tracking: 0 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enable_pipelined_write: 0 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.unordered_write: 0 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.row_cache: None 2024-08-20T00:58:17.767 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.wal_filter: None 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_ingest_behind: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.two_write_queues: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.manual_wal_flush: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.wal_compression: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.atomic_flush: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.persist_stats_to_disk: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.write_dbid_to_manifest: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.log_readahead_size: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.best_efforts_recovery: 0 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-08-20T00:58:17.768 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.allow_data_in_errors: 0 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.db_host_id: __hostname__ 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enforce_single_del_contracts: true 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_background_jobs: 2 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_background_compactions: -1 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_subcompactions: 1 2024-08-20T00:58:17.769 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.delayed_write_rate : 16777216 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_total_wal_size: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.stats_dump_period_sec: 600 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.stats_persist_period_sec: 600 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_open_files: -1 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bytes_per_sync: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.wal_bytes_per_sync: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.strict_bytes_per_sync: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_readahead_size: 0 2024-08-20T00:58:17.771 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_background_flushes: -1 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Compression algorithms supported: 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kZSTD supported: 0 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kXpressCompression supported: 0 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kBZip2Compression supported: 0 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kLZ4Compression supported: 1 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kZlibCompression supported: 1 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kLZ4HCCompression supported: 1 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: kSnappyCompression supported: 1 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Fast CRC32 supported: Supported on x86 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: DMutex implementation: pthread_mutex_t 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-08-20T00:58:17.772 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.merge_operator: 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_filter: None 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_filter_factory: None 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.sst_partitioner_factory: None 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.memtable_factory: SkipListFactory 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.table_factory: BlockBasedTable 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55738dd7d380) 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks: 1 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: pin_top_level_index_and_filter: 1 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_type: 0 2024-08-20T00:58:17.773 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_index_type: 0 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_shortening: 1 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: data_block_hash_table_util_ratio: 0.750000 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: checksum: 4 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: no_block_cache: 0 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache: 0x55738dd71610 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_name: BinnedLRUCache 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_options: 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: capacity : 536870912 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_shard_bits : 4 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: strict_capacity_limit : 0 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: high_pri_pool_ratio: 0.000 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_cache_compressed: (nil) 2024-08-20T00:58:17.774 INFO:journalctl@ceph.mon.b.smithi142.stdout: persistent_cache: (nil) 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size: 4096 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_size_deviation: 10 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_restart_interval: 16 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: index_block_restart_interval: 1 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: metadata_block_size: 4096 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: partition_filters: 0 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: use_delta_encoding: 1 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: filter_policy: bloomfilter 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: whole_key_filtering: 1 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: verify_compression: 0 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: read_amp_bytes_per_bit: 0 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: format_version: 5 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_index_compression: 1 2024-08-20T00:58:17.775 INFO:journalctl@ceph.mon.b.smithi142.stdout: block_align: 0 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout: max_auto_readahead_size: 262144 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout: prepopulate_block_cache: 0 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout: initial_auto_readahead_size: 8192 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout: num_file_reads_for_auto_readahead: 2 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.write_buffer_size: 33554432 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_write_buffer_number: 2 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression: NoCompression 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression: Disabled 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.prefix_extractor: nullptr 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.num_levels: 7 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-08-20T00:58:17.776 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.window_bits: -14 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.level: 32767 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.strategy: 0 2024-08-20T00:58:17.777 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.enabled: false 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.target_file_size_base: 67108864 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.target_file_size_multiplier: 1 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-08-20T00:58:17.778 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.arena_block_size: 1048576 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.disable_auto_compactions: 0 2024-08-20T00:58:17.779 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-08-20T00:58:17.780 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.inplace_update_support: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.inplace_update_num_locks: 10000 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.memtable_huge_page_size: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.bloom_locality: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.max_successive_merges: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.optimize_filters_for_hits: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.paranoid_file_checks: 0 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.force_consistency_checks: 1 2024-08-20T00:58:17.781 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.report_bg_io_stats: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.ttl: 2592000 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.periodic_compaction_seconds: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enable_blob_files: false 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.min_blob_size: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_file_size: 268435456 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_compression_type: NoCompression 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.enable_blob_garbage_collection: false 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.blob_file_starting_level: 0 2024-08-20T00:58:17.782 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9caac514-15bb-450d-97f7-24fa74a10dee 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115497765011, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115497765562, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1724115497, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9caac514-15bb-450d-97f7-24fa74a10dee", "db_session_id": "BCO7FBFU7GLYOO4DLJ5P", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: EVENT_LOG_v1 {"time_micros": 1724115497765686, "job": 1, "event": "recovery_finished"} 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55738dda0e00 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: DB pointer 0x55738de9e000 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-08-20T00:58:17.783 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** DB Stats ** 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:17.784 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** Compaction Stats [default] ** 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:17.785 INFO:journalctl@ceph.mon.b.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Interval compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache BinnedLRUCache@0x55738dd71610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-08-20T00:58:17.786 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-08-20T00:58:17.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-08-20T00:58:17.787 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: starting mon.b rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:18.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(???) e0 preinit fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:18.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).mds e1 new map 2024-08-20T00:58:18.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).mds e1 print_map 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: e1 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: btime 2024-08-20T00:56:38:482793+0000 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: legacy client fscid: -1 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout: No filesystems configured 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-08-20T00:58:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi167", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:18.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Deploying cephadm binary to smithi167 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Added host smithi167 2024-08-20T00:58:18.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:18.245 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2659267847' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:58:18.246 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Saving service mon spec with placement smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c;count:3 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Deploying daemon mon.c on smithi167 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.247 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/180774585' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.a calling monitor election 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.c calling monitor election 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-08-20T00:58:18.248 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: monmap epoch 2 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: last_changed 2024-08-20T00:58:09.198178+0000 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: min_mon_release 19 (squid) 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: election_strategy: 1 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: fsmap 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mgrmap e13: x(active, since 43s) 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: overall HEALTH_OK 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:18.249 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: Deploying daemon mon.b on smithi142 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/3286123419' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 ceph-mon[21799]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 bash[21780]: df50ac551d24d80ad817891ebf354539ab62708048f5e924bc47f8e9463d8037 2024-08-20T00:58:18.250 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:17 smithi142 systemd[1]: Started Ceph mon.b for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:58:19.185 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-08-20T00:58:19.185 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph mon dump -f json 2024-08-20T00:58:19.438 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T00:58:23.154 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T00:58:23.154 INFO:teuthology.orchestra.run.smithi167.stdout:{"epoch":3,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","modified":"2024-08-20T00:58:17.791629Z","created":"2024-08-20T00:56:34.289119Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:3300","nonce":0},{"type":"v1","addr":"172.21.15.167:6789","nonce":0}]},"addr":"172.21.15.167:6789/0","public_addr":"172.21.15.167:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-08-20T00:58:23.155 INFO:teuthology.orchestra.run.smithi167.stderr:dumped monmap epoch 3 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: mon.a calling monitor election 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: mon.c calling monitor election 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:23.155 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: mon.b calling monitor election 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: monmap epoch 3 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: last_changed 2024-08-20T00:58:17.791629+0000 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:23.156 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: min_mon_release 19 (squid) 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: election_strategy: 1 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: fsmap 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: mgrmap e13: x(active, since 52s) 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: overall HEALTH_OK 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.157 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:22 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: mon.a calling monitor election 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: mon.c calling monitor election 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: mon.b calling monitor election 2024-08-20T00:58:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: monmap epoch 3 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: last_changed 2024-08-20T00:58:17.791629+0000 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: min_mon_release 19 (squid) 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: election_strategy: 1 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-08-20T00:58:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: fsmap 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: mgrmap e13: x(active, since 52s) 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: overall HEALTH_OK 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.244 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:22 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: mon.a calling monitor election 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: mon.c calling monitor election 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: mon.b calling monitor election 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: monmap epoch 3 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: last_changed 2024-08-20T00:58:17.791629+0000 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: created 2024-08-20T00:56:34.289119+0000 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: min_mon_release 19 (squid) 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: election_strategy: 1 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: 0: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.a 2024-08-20T00:58:23.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: 1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.c 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: 2: [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon.b 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: fsmap 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: osdmap e4: 0 total, 0 up, 0 in 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: mgrmap e13: x(active, since 52s) 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: overall HEALTH_OK 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:22 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:23.732 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-08-20T00:58:23.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config generate-minimal-conf 2024-08-20T00:58:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:23 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:23 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/803250185' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:23 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:23 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:24.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:23 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/803250185' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:24.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:23 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:24.104 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:24.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:23 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:24.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:23 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/803250185' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-08-20T00:58:24.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:23 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:24.910 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:58:24.791+0000 7f59fc200640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-08-20T00:58:24.911 INFO:teuthology.orchestra.run.smithi138.stdout:# minimal ceph.conf for d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:24.911 INFO:teuthology.orchestra.run.smithi138.stdout:[global] 2024-08-20T00:58:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: fsid = d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T00:58:24.911 INFO:teuthology.orchestra.run.smithi138.stdout: mon_host = [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] 2024-08-20T00:58:25.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:25 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2173694869' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:25 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2173694869' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:25.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:25.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:25 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2173694869' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:25.644 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-08-20T00:58:25.644 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:58:25.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T00:58:25.704 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:58:25.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:25.799 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:58:25.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T00:58:25.828 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:58:25.828 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:25.895 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:58:25.895 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.conf 2024-08-20T00:58:25.924 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:58:25.924 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:25.991 INFO:tasks.cephadm:enabling module snap_schedule 2024-08-20T00:58:25.991 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:26.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:26.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:26 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:26 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.347 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:26.348 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:26 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:26.883 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:27.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:27 smithi138 ceph-mon[18076]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T00:58:27.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:27 smithi138 ceph-mon[18076]: Reconfiguring daemon mon.a on smithi138 2024-08-20T00:58:27.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:27 smithi138 ceph-mon[18076]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:27.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:27 smithi167 ceph-mon[21965]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T00:58:27.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:27 smithi167 ceph-mon[21965]: Reconfiguring daemon mon.a on smithi138 2024-08-20T00:58:27.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:27 smithi167 ceph-mon[21965]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:27.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:27 smithi142 ceph-mon[21799]: Reconfiguring mon.a (unknown last config time)... 2024-08-20T00:58:27.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:27 smithi142 ceph-mon[21799]: Reconfiguring daemon mon.a on smithi138 2024-08-20T00:58:27.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:27 smithi142 ceph-mon[21799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T00:58:28.684 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:28 smithi142 ceph-mon[21799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:58:28.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T00:58:28.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:28 smithi167 ceph-mon[21965]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setuser ceph since I am not root 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ignoring --setgroup ceph since I am not root 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='mgr.14150 172.21.15.138:0/994184274' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:28.751 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:58:28.752 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T00:58:28.752 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:28 smithi138 ceph-mon[18076]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:29.090 INFO:tasks.cephadm:Adding mgr.x on smithi138 2024-08-20T00:58:29.090 INFO:tasks.cephadm:Adding mgr.y on smithi142 2024-08-20T00:58:29.090 INFO:tasks.cephadm:Adding mgr.z on smithi167 2024-08-20T00:58:29.090 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch apply mgr '3;smithi138=x;smithi142=y;smithi167=z' 2024-08-20T00:58:29.330 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T00:58:29.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:29 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T00:58:29.680 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:29 smithi167 ceph-mon[21965]: mgrmap e14: x(active, since 58s) 2024-08-20T00:58:29.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:29 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T00:58:29.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:29 smithi142 ceph-mon[21799]: mgrmap e14: x(active, since 58s) 2024-08-20T00:58:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:29 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/313338443' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T00:58:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:29 smithi138 ceph-mon[18076]: mgrmap e14: x(active, since 58s) 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: Active manager daemon x restarted 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: Activating manager daemon x 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: mgrmap e15: x(active, starting, since 0.00694643s) 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:58:34.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:34 smithi167 ceph-mon[21965]: Manager daemon x is now available 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: Active manager daemon x restarted 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: Activating manager daemon x 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: mgrmap e15: x(active, starting, since 0.00694643s) 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:58:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:58:34.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:58:34.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:58:34.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:34 smithi142 ceph-mon[21799]: Manager daemon x is now available 2024-08-20T00:58:34.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: Active manager daemon x restarted 2024-08-20T00:58:34.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: Activating manager daemon x 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: osdmap e5: 0 total, 0 up, 0 in 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: mgrmap e15: x(active, starting, since 0.00694643s) 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-08-20T00:58:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:34 smithi138 ceph-mon[18076]: Manager daemon x is now available 2024-08-20T00:58:35.394 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mgr update... 2024-08-20T00:58:35.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:35.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: mgrmap e16: x(active, since 1.00885s) 2024-08-20T00:58:35.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:35.737 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:35.737 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.737 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.737 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.738 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.738 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: mgrmap e16: x(active, since 1.00885s) 2024-08-20T00:58:35.738 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-08-20T00:58:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: mgrmap e16: x(active, since 1.00885s) 2024-08-20T00:58:35.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.079 DEBUG:teuthology.orchestra.run.smithi142:mgr.y> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.y.service 2024-08-20T00:58:36.082 DEBUG:teuthology.orchestra.run.smithi167:mgr.z> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z.service 2024-08-20T00:58:36.084 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-08-20T00:58:36.085 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-08-20T00:58:36.090 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-08-20T00:58:36.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set client debug_client 20 2024-08-20T00:58:36.137 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-08-20T00:58:36.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set client debug_ms 1 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:58:35.843+0000 7f5726400640 -1 log_channel(cephadm) log [ERR] : [20/Aug/2024:00:58:35] ENGINE Error in HTTPServer.serve 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: Traceback (most recent call last): 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._connections.run(self.expiration_interval) 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:58:36.138 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._run(expiration_interval) 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: s = self.context.wrap_socket( 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:58:36.139 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: return self.sslsocket_class._create( 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self.do_handshake() 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: self._sslobj.do_handshake() 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:58:36.140 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 00:58:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T00:58:36.181 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-08-20T00:58:36.181 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-08-20T00:58:36.225 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-08-20T00:58:36.226 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-08-20T00:58:36.273 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-08-20T00:58:36.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds 20 2024-08-20T00:58:36.320 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-08-20T00:58:36.320 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-08-20T00:58:36.369 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-08-20T00:58:36.369 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds debug_ms 2 2024-08-20T00:58:36.419 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-08-20T00:58:36.419 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-08-20T00:58:36.468 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-08-20T00:58:36.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-08-20T00:58:36.521 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-08-20T00:58:36.521 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-08-20T00:58:36.576 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-08-20T00:58:36.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-08-20T00:58:36.629 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-08-20T00:58:36.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-08-20T00:58:36.686 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-08-20T00:58:36.717 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:36 smithi167 ceph-mon[21965]: Saving service mgr spec with placement smithi138=x;smithi142=y;smithi167=z;count:3 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.718 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:36 smithi138 ceph-mon[18076]: Saving service mgr spec with placement smithi138=x;smithi142=y;smithi167=z;count:3 2024-08-20T00:58:36.719 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.719 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.719 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.719 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:36 smithi142 ceph-mon[21799]: Saving service mgr spec with placement smithi138=x;smithi142=y;smithi167=z;count:3 2024-08-20T00:58:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:36.778 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-08-20T00:58:36.778 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-08-20T00:58:36.837 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-08-20T00:58:36.837 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-08-20T00:58:36.892 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-08-20T00:58:36.892 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-08-20T00:58:36.953 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-08-20T00:58:36.953 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-08-20T00:58:37.018 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-08-20T00:58:37.019 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-08-20T00:58:37.074 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-08-20T00:58:37.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mgr debug_cephsqlite 20 2024-08-20T00:58:37.108 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.123 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-08-20T00:58:37.124 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-08-20T00:58:37.124 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.167 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.184 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-08-20T00:58:37.184 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-08-20T00:58:37.242 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-08-20T00:58:37.242 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-08-20T00:58:37.243 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.304 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-08-20T00:58:37.304 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-08-20T00:58:37.305 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.360 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-08-20T00:58:37.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-08-20T00:58:37.430 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-08-20T00:58:37.430 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-08-20T00:58:37.486 DEBUG:tasks.cephadm:set 26 configs 2024-08-20T00:58:37.486 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi138') 2024-08-20T00:58:37.531 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:37.579 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: [20/Aug/2024:00:58:35] ENGINE Bus STARTING 2024-08-20T00:58:37.579 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: [20/Aug/2024:00:58:35] ENGINE Serving on https://172.21.15.138:7150 2024-08-20T00:58:37.579 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: [20/Aug/2024:00:58:35] ENGINE Error in HTTPServer.serve 2024-08-20T00:58:37.579 INFO:journalctl@ceph.mon.a.smithi138.stdout: Traceback (most recent call last): 2024-08-20T00:58:37.580 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:58:37.580 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._connections.run(self.expiration_interval) 2024-08-20T00:58:37.580 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:58:37.580 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._run(expiration_interval) 2024-08-20T00:58:37.580 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: s = self.context.wrap_socket( 2024-08-20T00:58:37.581 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: return self.sslsocket_class._create( 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: self.do_handshake() 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: self._sslobj.do_handshake() 2024-08-20T00:58:37.582 INFO:journalctl@ceph.mon.a.smithi138.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:58:37.583 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: [20/Aug/2024:00:58:35] ENGINE Serving on http://172.21.15.138:8765 2024-08-20T00:58:37.583 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: [20/Aug/2024:00:58:35] ENGINE Bus STARTED 2024-08-20T00:58:37.583 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.583 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.583 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.584 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.584 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: mgrmap e17: x(active, since 2s) 2024-08-20T00:58:37.584 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.585 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.585 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.585 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.585 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.585 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: [20/Aug/2024:00:58:35] ENGINE Bus STARTING 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: [20/Aug/2024:00:58:35] ENGINE Serving on https://172.21.15.138:7150 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: [20/Aug/2024:00:58:35] ENGINE Error in HTTPServer.serve 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout: Traceback (most recent call last): 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:58:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._connections.run(self.expiration_interval) 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._run(expiration_interval) 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: s = self.context.wrap_socket( 2024-08-20T00:58:37.714 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: return self.sslsocket_class._create( 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: self.do_handshake() 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: self._sslobj.do_handshake() 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: [20/Aug/2024:00:58:35] ENGINE Serving on http://172.21.15.138:8765 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: [20/Aug/2024:00:58:35] ENGINE Bus STARTED 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: mgrmap e17: x(active, since 2s) 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.716 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: [20/Aug/2024:00:58:35] ENGINE Bus STARTING 2024-08-20T00:58:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: [20/Aug/2024:00:58:35] ENGINE Serving on https://172.21.15.138:7150 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: [20/Aug/2024:00:58:35] ENGINE Error in HTTPServer.serve 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout: Traceback (most recent call last): 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._connections.run(self.expiration_interval) 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-08-20T00:58:37.742 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._run(expiration_interval) 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: s = self.context.wrap_socket( 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: return self.sslsocket_class._create( 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: self.do_handshake() 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: self._sslobj.do_handshake() 2024-08-20T00:58:37.743 INFO:journalctl@ceph.mon.b.smithi142.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: [20/Aug/2024:00:58:35] ENGINE Serving on http://172.21.15.138:8765 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: [20/Aug/2024:00:58:35] ENGINE Bus STARTED 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: mgrmap e17: x(active, since 2s) 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi167", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.744 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:37.808 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:40.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:39 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2596559028' entity='client.admin' 2024-08-20T00:58:40.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:40.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:40.234 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:39 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2596559028' entity='client.admin' 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"} : dispatch 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: Updating smithi138:/etc/ceph/ceph.conf 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: Updating smithi142:/etc/ceph/ceph.conf 2024-08-20T00:58:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: Updating smithi167:/etc/ceph/ceph.conf 2024-08-20T00:58:40.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:39 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2596559028' entity='client.admin' 2024-08-20T00:58:41.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:41 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3839000746' entity='client.admin' 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:41 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3839000746' entity='client.admin' 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi167:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.conf 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:41.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-08-20T00:58:41.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:41 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3839000746' entity='client.admin' 2024-08-20T00:58:42.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:42.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.655 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: Deploying daemon mgr.z on smithi167 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:42.656 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:42 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2823260332' entity='client.admin' 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: Deploying daemon mgr.z on smithi167 2024-08-20T00:58:42.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:42.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:42 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2823260332' entity='client.admin' 2024-08-20T00:58:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: Updating smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/config/ceph.client.admin.keyring 2024-08-20T00:58:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: Deploying daemon mgr.z on smithi167 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:42 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2823260332' entity='client.admin' 2024-08-20T00:58:43.213 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 systemd[1]: Starting Ceph mgr.z for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:58:43.585 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 podman[24596]: 2024-08-20 00:58:43.209598158 +0000 UTC m=+0.015802239 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T00:58:43.585 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 podman[24596]: 2024-08-20 00:58:43.30958313 +0000 UTC m=+0.115787213 container create e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-08-20T00:58:43.585 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 podman[24596]: 2024-08-20 00:58:43.512023377 +0000 UTC m=+0.318227457 container init e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-20T00:58:43.585 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 podman[24596]: 2024-08-20 00:58:43.514808309 +0000 UTC m=+0.321012394 container start e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-08-20T00:58:43.585 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 ceph-mgr[24615]: -- 172.21.15.167:0/1996811221 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5619a00430e0 con 0x5619a0021400 2024-08-20T00:58:43.586 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 bash[24596]: e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 2024-08-20T00:58:43.943 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi138') 2024-08-20T00:58:43.944 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi138') 2024-08-20T00:58:43.962 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 00:58:43 smithi167 systemd[1]: Started Ceph mgr.z for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:58:44.356 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/166422142' entity='client.admin' 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:44.357 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:44.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/166422142' entity='client.admin' 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:44.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/166422142' entity='client.admin' 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:44.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:45.402 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:45 smithi138 ceph-mon[18076]: Deploying daemon mgr.y on smithi142 2024-08-20T00:58:45.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.403+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faf6003ee60 msgr2=0x7faf60041320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:45.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.403+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faf6003ee60 0x7faf60041320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faf7c107df0 tx=0x7faf68005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.403+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf7c102920 msgr2=0x7faf7c19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.403+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf7c102920 0x7faf7c19ae80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7faf6c005d90 tx=0x7faf6c003c80 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 shutdown_connections 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faf6003ee60 0x7faf60041320 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf7c108cb0 0x7faf7c1a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faf7c108330 0x7faf7c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 --2- 172.21.15.138:0/4153222321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf7c102920 0x7faf7c19ae80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 >> 172.21.15.138:0/4153222321 conn(0x7faf7c0fe350 msgr2=0x7faf7c1048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 shutdown_connections 2024-08-20T00:58:45.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:45.404+0000 7faf7b200640 1 -- 172.21.15.138:0/4153222321 wait complete. 2024-08-20T00:58:45.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:45 smithi142 ceph-mon[21799]: Deploying daemon mgr.y on smithi142 2024-08-20T00:58:45.491 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 systemd[1]: Starting Ceph mgr.y for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T00:58:45.670 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:45 smithi167 ceph-mon[21965]: Deploying daemon mgr.y on smithi142 2024-08-20T00:58:45.791 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 podman[24113]: 2024-08-20 00:58:45.538952669 +0000 UTC m=+0.108955876 container create 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T00:58:45.791 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 podman[24113]: 2024-08-20 00:58:45.453872464 +0000 UTC m=+0.023875603 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T00:58:45.792 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 podman[24113]: 2024-08-20 00:58:45.766155755 +0000 UTC m=+0.336158880 container init 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True) 2024-08-20T00:58:45.792 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 podman[24113]: 2024-08-20 00:58:45.76957154 +0000 UTC m=+0.339574675 container start 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T00:58:45.848 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.848 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.856 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.861 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.865 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.870 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.873 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:45.879 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:46.064 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 ceph-mgr[24132]: -- 172.21.15.142:0/3381530999 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b737f350e0 con 0x55b737f13000 2024-08-20T00:58:46.064 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 bash[24113]: 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c 2024-08-20T00:58:46.064 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 00:58:45 smithi142 systemd[1]: Started Ceph mgr.y for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T00:58:46.449 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi138') 2024-08-20T00:58:46.449 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi138') 2024-08-20T00:58:46.449 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi138') 2024-08-20T00:58:46.450 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi138') 2024-08-20T00:58:46.450 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi138') 2024-08-20T00:58:46.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:46.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:46 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:46 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:46.729 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:46 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:58:46.831 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:47.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 -- 172.21.15.138:0/183563977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed4410cc70 msgr2=0x7fed4410f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:47.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 --2- 172.21.15.138:0/183563977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed4410cc70 0x7fed4410f0d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009ca0 tx=0x7fed3c02a8c0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 -- 172.21.15.138:0/183563977 shutdown_connections 2024-08-20T00:58:47.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 --2- 172.21.15.138:0/183563977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed4410cc70 0x7fed4410f0d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 --2- 172.21.15.138:0/183563977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44104130 0x7fed4410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 --2- 172.21.15.138:0/183563977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed44103810 0x7fed44103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 -- 172.21.15.138:0/183563977 >> 172.21.15.138:0/183563977 conn(0x7fed440fd2d0 msgr2=0x7fed440ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 -- 172.21.15.138:0/183563977 shutdown_connections 2024-08-20T00:58:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.651+0000 7fed4a800640 1 -- 172.21.15.138:0/183563977 wait complete. 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 Processor -- start 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 -- start start 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 0x7fed4419b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 0x7fed441a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed44112930 con 0x7fed44104130 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed441127b0 con 0x7fed4410cc70 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.652+0000 7fed4a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed44112ab0 con 0x7fed44103810 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:60548/0 (socket says 172.21.15.138:60548) 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 -- 172.21.15.138:0/4270715523 learned_addr learned my addr 172.21.15.138:0/4270715523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:47.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49e00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 0x7fed441a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:47.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed48a00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 0x7fed4419b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:47.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 -- 172.21.15.138:0/4270715523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 msgr2=0x7fed441a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 0x7fed441a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 -- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 msgr2=0x7fed4419b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 0x7fed4419b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 -- 172.21.15.138:0/4270715523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed441a0f50 con 0x7fed44103810 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49e00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 0x7fed441a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed48a00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 0x7fed4419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.653+0000 7fed49400640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fed3800e9f0 tx=0x7fed3800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.654+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3800cd80 con 0x7fed44103810 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.654+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed380108e0 con 0x7fed44103810 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.654+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38018980 con 0x7fed44103810 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.654+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a1240 con 0x7fed44103810 2024-08-20T00:58:47.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.654+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a7900 con 0x7fed44103810 2024-08-20T00:58:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.655+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed44103bf0 con 0x7fed44103810 2024-08-20T00:58:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.655+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 50765+0+0 (secure 0 0 0) 0x7fed38010430 con 0x7fed44103810 2024-08-20T00:58:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.656+0000 7fed42000640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 0x7fed2c041320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.656+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fed380538e0 con 0x7fed44103810 2024-08-20T00:58:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.656+0000 7fed48a00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 0x7fed2c041320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.657+0000 7fed48a00640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 0x7fed2c041320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed4410e1b0 tx=0x7fed30005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:47.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.660+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fed38010b40 con 0x7fed44103810 2024-08-20T00:58:47.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.857+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7fed4419bfd0 con 0x7fed44103810 2024-08-20T00:58:47.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.862+0000 7fed42000640 1 -- 172.21.15.138:0/4270715523 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v17)=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fed380071c0 con 0x7fed44103810 2024-08-20T00:58:47.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 msgr2=0x7fed2c041320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:47.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 0x7fed2c041320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fed4410e1b0 tx=0x7fed30005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 msgr2=0x7fed4419aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fed3800e9f0 tx=0x7fed3800eeb0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 shutdown_connections 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed2c03ee60 0x7fed2c041320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed4410cc70 0x7fed441a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed44104130 0x7fed4419b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 --2- 172.21.15.138:0/4270715523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed44103810 0x7fed4419aea0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:47.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 >> 172.21.15.138:0/4270715523 conn(0x7fed440fd2d0 msgr2=0x7fed440fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:47.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 shutdown_connections 2024-08-20T00:58:47.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.865+0000 7fed4a800640 1 -- 172.21.15.138:0/4270715523 wait complete. 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:48.197 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.197 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: Reconfiguring daemon mgr.x on smithi138 2024-08-20T00:58:48.197 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:48 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:48.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 -- 172.21.15.138:0/2690482229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010a6a0 msgr2=0x7f0fd010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 --2- 172.21.15.138:0/2690482229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010a6a0 0x7f0fd010aae0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009900 tx=0x7f0fb802a3f0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 -- 172.21.15.138:0/2690482229 shutdown_connections 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 --2- 172.21.15.138:0/2690482229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010b020 0x7f0fd0111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 --2- 172.21.15.138:0/2690482229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010a6a0 0x7f0fd010aae0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 --2- 172.21.15.138:0/2690482229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd0105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 -- 172.21.15.138:0/2690482229 >> 172.21.15.138:0/2690482229 conn(0x7f0fd01006c0 msgr2=0x7f0fd0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 -- 172.21.15.138:0/2690482229 shutdown_connections 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.326+0000 7f0fd5800640 1 -- 172.21.15.138:0/2690482229 wait complete. 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.327+0000 7f0fd5800640 1 Processor -- start 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.327+0000 7f0fd5800640 1 -- start start 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.327+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010a6a0 0x7f0fd019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fd5800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010b020 0x7f0fd01a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fd5800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0114d30 con 0x7f0fd010b020 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fd5800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0114bb0 con 0x7f0fd010a6a0 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fd5800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0114eb0 con 0x7f0fd0104c90 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42738/0 (socket says 172.21.15.138:42738) 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 -- 172.21.15.138:0/305987033 learned_addr learned my addr 172.21.15.138:0/305987033 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcf400640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010b020 0x7f0fd01a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fc7e00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010a6a0 0x7f0fd019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 -- 172.21.15.138:0/305987033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010a6a0 msgr2=0x7f0fd019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010a6a0 0x7f0fd019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 -- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010b020 msgr2=0x7f0fd01a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010b020 0x7f0fd01a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 -- 172.21.15.138:0/305987033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd01a1230 con 0x7f0fd0104c90 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.328+0000 7f0fcea00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000e9f0 tx=0x7f0fc000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.329+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000ccc0 con 0x7f0fd0104c90 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.329+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc00108e0 con 0x7f0fd0104c90 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.329+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd01a1520 con 0x7f0fd0104c90 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.329+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0018880 con 0x7f0fd0104c90 2024-08-20T00:58:48.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.329+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd01a7db0 con 0x7f0fd0104c90 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.330+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 50765+0+0 (secure 0 0 0) 0x7f0fc0018a20 con 0x7f0fd0104c90 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.330+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd0105f40 con 0x7f0fd0104c90 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.330+0000 7f0fc7400640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 0x7f0fb4041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.330+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f0fc0053870 con 0x7f0fd0104c90 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.331+0000 7f0fc7e00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 0x7f0fb4041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.331+0000 7f0fc7e00640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 0x7f0fb4041270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8005ab0 tx=0x7f0fb8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:48.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.335+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0fc0020360 con 0x7f0fd0104c90 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: Reconfiguring daemon mgr.x on smithi138 2024-08-20T00:58:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:48 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: Reconfiguring mgr.x (unknown last config time)... 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-08-20T00:58:48.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-08-20T00:58:48.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:58:48.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: Reconfiguring daemon mgr.x on smithi138 2024-08-20T00:58:48.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:48 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:48.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.530+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f0fd010ce00 con 0x7f0fd0104c90 2024-08-20T00:58:48.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.537+0000 7f0fc7400640 1 -- 172.21.15.138:0/305987033 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v19)=0 v19) ==== 116+0+0 (secure 0 0 0) 0x7f0fc0013190 con 0x7f0fd0104c90 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.539+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 msgr2=0x7f0fb4041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.539+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 0x7f0fb4041270 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8005ab0 tx=0x7f0fb8005a20 comp rx=0 tx=0).stop 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.539+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 msgr2=0x7f0fd019b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.539+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000e9f0 tx=0x7f0fc000edf0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 shutdown_connections 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0fb403edb0 0x7f0fb4041270 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fd010b020 0x7f0fd01a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fd010a6a0 0x7f0fd019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 --2- 172.21.15.138:0/305987033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0fd0104c90 0x7f0fd019b180 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 >> 172.21.15.138:0/305987033 conn(0x7f0fd01006c0 msgr2=0x7f0fd0102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:48.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 shutdown_connections 2024-08-20T00:58:48.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.540+0000 7f0fd5800640 1 -- 172.21.15.138:0/305987033 wait complete. 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.869+0000 7f5b41600640 1 -- 172.21.15.138:0/1985988268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c1090e0 msgr2=0x7f5b3c1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.869+0000 7f5b41600640 1 --2- 172.21.15.138:0/1985988268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c1094c0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009900 tx=0x7f5b2c02a3f0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 -- 172.21.15.138:0/1985988268 shutdown_connections 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 --2- 172.21.15.138:0/1985988268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 0x7f5b3c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 --2- 172.21.15.138:0/1985988268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 --2- 172.21.15.138:0/1985988268 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c1094c0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 -- 172.21.15.138:0/1985988268 >> 172.21.15.138:0/1985988268 conn(0x7f5b3c078760 msgr2=0x7f5b3c07ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 -- 172.21.15.138:0/1985988268 shutdown_connections 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 -- 172.21.15.138:0/1985988268 wait complete. 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.870+0000 7f5b41600640 1 Processor -- start 2024-08-20T00:58:48.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 -- start start 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 0x7f5b3c1a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c1190d0 con 0x7f5b3c1090e0 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c118f50 con 0x7f5b3c10f470 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b41600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3c119250 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b3aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b3a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42756/0 (socket says 172.21.15.138:42756) 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b3a000640 1 -- 172.21.15.138:0/180023974 learned_addr learned my addr 172.21.15.138:0/180023974 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:48.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.871+0000 7f5b3b400640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 0x7f5b3c1a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 -- 172.21.15.138:0/180023974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 msgr2=0x7f5b3c1a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 0x7f5b3c1a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 -- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c1090e0 msgr2=0x7f5b3c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 -- 172.21.15.138:0/180023974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3c1a57d0 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b3a000640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e9f0 tx=0x7f5b2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800cd80 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b280108e0 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28018980 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3c1a5ac0 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.872+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3c1ac180 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.873+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3c10a390 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.873+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 50765+0+0 (secure 0 0 0) 0x7f5b28010430 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.874+0000 7f5b33400640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 0x7f5b20041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:48.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.874+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5b280538f0 con 0x7f5b3c10eaf0 2024-08-20T00:58:48.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.874+0000 7f5b3aa00640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 0x7f5b20041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:48.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.874+0000 7f5b3aa00640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 0x7f5b20041270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005ab0 tx=0x7f5b2c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:48.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.878+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5b2800b2c0 con 0x7f5b3c10eaf0 2024-08-20T00:58:49.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.067+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f5b3c111250 con 0x7f5b3c10eaf0 2024-08-20T00:58:49.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.072+0000 7f5b33400640 1 -- 172.21.15.138:0/180023974 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f5b28010a90 con 0x7f5b3c10eaf0 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.074+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 msgr2=0x7f5b20041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.074+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 0x7f5b20041270 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c005ab0 tx=0x7f5b2c005a20 comp rx=0 tx=0).stop 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.074+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 msgr2=0x7f5b3c19fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.074+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e9f0 tx=0x7f5b2800eeb0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 shutdown_connections 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5b2003edb0 0x7f5b20041270 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5b3c10f470 0x7f5b3c1a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5b3c10eaf0 0x7f5b3c19fc60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 --2- 172.21.15.138:0/180023974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b3c1090e0 0x7f5b3c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 >> 172.21.15.138:0/180023974 conn(0x7f5b3c078760 msgr2=0x7f5b3c0790b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:49.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 shutdown_connections 2024-08-20T00:58:49.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.075+0000 7f5b41600640 1 -- 172.21.15.138:0/180023974 wait complete. 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: Standby manager daemon z started 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:49.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:49 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: Standby manager daemon z started 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:49.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:49 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.522 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: Standby manager daemon z started 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.167:0/1588164524' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:49.523 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:49 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:49.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 -- 172.21.15.138:0/3808241895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10ae90 msgr2=0x7f217c111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 --2- 172.21.15.138:0/3808241895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10ae90 0x7f217c111690 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2174009be0 tx=0x7f217402a6c0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 -- 172.21.15.138:0/3808241895 shutdown_connections 2024-08-20T00:58:49.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 --2- 172.21.15.138:0/3808241895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10ae90 0x7f217c111690 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 --2- 172.21.15.138:0/3808241895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10a510 0x7f217c10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 --2- 172.21.15.138:0/3808241895 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 -- 172.21.15.138:0/3808241895 >> 172.21.15.138:0/3808241895 conn(0x7f217c100550 msgr2=0x7f217c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 -- 172.21.15.138:0/3808241895 shutdown_connections 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:47.999+0000 7f2181600640 1 -- 172.21.15.138:0/3808241895 wait complete. 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 Processor -- start 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 -- start start 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 0x7f217c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10ae90 0x7f217c1a0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:49.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114b50 con 0x7f217c10a510 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f217c1149d0 con 0x7f217c10ae90 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2181600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f217c114cd0 con 0x7f217c104b00 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2173e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 0x7f217c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f217be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f217be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42726/0 (socket says 172.21.15.138:42726) 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f2173e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 0x7f217c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39506/0 (socket says 172.21.15.138:39506) 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.001+0000 7f217be00640 1 -- 172.21.15.138:0/1576293939 learned_addr learned my addr 172.21.15.138:0/1576293939 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f2180c00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10ae90 0x7f217c1a0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 -- 172.21.15.138:0/1576293939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10ae90 msgr2=0x7f217c1a0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10ae90 0x7f217c1a0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 -- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 msgr2=0x7f217c19b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 0x7f217c19b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 -- 172.21.15.138:0/1576293939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c1a0f50 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f217be00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e9f0 tx=0x7f216c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c00cd80 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f216c0108e0 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f216c018980 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.002+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f217c1a1240 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.003+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f217c1a7900 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.003+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 17) ==== 50765+0+0 (secure 0 0 0) 0x7f216c010430 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.003+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f217c104ee0 con 0x7f217c104b00 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.004+0000 7f2179600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 0x7f2160041320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:49.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.004+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f216c0538f0 con 0x7f217c104b00 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.004+0000 7f2173e00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 0x7f2160041320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.005+0000 7f2173e00640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 0x7f2160041320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f217c102bd0 tx=0x7f2164009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.009+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f216c010a90 con 0x7f217c104b00 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.196+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f217c1a7d50 con 0x7f217c104b00 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.203+0000 7f2179600640 1 -- 172.21.15.138:0/1576293939 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v18)=0 v18) ==== 115+0+0 (secure 0 0 0) 0x7f216c013dc0 con 0x7f217c104b00 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.205+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 msgr2=0x7f2160041320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.205+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 0x7f2160041320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f217c102bd0 tx=0x7f2164009260 comp rx=0 tx=0).stop 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.205+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 msgr2=0x7f217c19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.205+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f216c00e9f0 tx=0x7f216c00eeb0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 shutdown_connections 2024-08-20T00:58:49.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f216003ee60 0x7f2160041320 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f217c10ae90 0x7f217c1a0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f217c10a510 0x7f217c19b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 --2- 172.21.15.138:0/1576293939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f217c104b00 0x7f217c19aea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 >> 172.21.15.138:0/1576293939 conn(0x7f217c100550 msgr2=0x7f217c1039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 shutdown_connections 2024-08-20T00:58:49.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:48.206+0000 7f2181600640 1 -- 172.21.15.138:0/1576293939 wait complete. 2024-08-20T00:58:50.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 -- 172.21.15.138:0/2979091142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 msgr2=0x7f22981116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 --2- 172.21.15.138:0/2979091142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981116a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2290009ca0 tx=0x7f229002a8b0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 -- 172.21.15.138:0/2979091142 shutdown_connections 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 --2- 172.21.15.138:0/2979091142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981116a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 --2- 172.21.15.138:0/2979091142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f229810a520 0x7f229810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 --2- 172.21.15.138:0/2979091142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2298104b10 0x7f2298104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 -- 172.21.15.138:0/2979091142 >> 172.21.15.138:0/2979091142 conn(0x7f2298100540 msgr2=0x7f2298102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 -- 172.21.15.138:0/2979091142 shutdown_connections 2024-08-20T00:58:50.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.233+0000 7f229d400640 1 -- 172.21.15.138:0/2979091142 wait complete. 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.234+0000 7f229d400640 1 Processor -- start 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.234+0000 7f229d400640 1 -- start start 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 0x7f229819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2298114bb0 con 0x7f229810a520 2024-08-20T00:58:50.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2298114a30 con 0x7f2298104b10 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2298114d30 con 0x7f229810aea0 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 0x7f229819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:49918/0 (socket says 172.21.15.138:49918) 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297e00640 1 -- 172.21.15.138:0/235352042 learned_addr learned my addr 172.21.15.138:0/235352042 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:50.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f229ca00640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 -- 172.21.15.138:0/235352042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 msgr2=0x7f22981a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 -- 172.21.15.138:0/235352042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 msgr2=0x7f229819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297400640 1 -- 172.21.15.138:0/235352042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22981a0fd0 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.235+0000 7f2297e00640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f2297400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 0x7f229819b460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f228400ccc0 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2284005020 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22981a12c0 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2284004a00 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.236+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22981a7980 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.237+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2298104ef0 con 0x7f229810a520 2024-08-20T00:58:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.238+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 100893+0+0 (secure 0 0 0) 0x7f2284004ba0 con 0x7f229810a520 2024-08-20T00:58:50.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.240+0000 7f2294c00640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 0x7f227c07ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.240+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f228409aa90 con 0x7f229810a520 2024-08-20T00:58:50.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.240+0000 7f2297e00640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 0x7f227c07ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.241+0000 7f2297e00640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 0x7f227c07ba10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22880095f0 tx=0x7f2288005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:50.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.244+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2284063d60 con 0x7f229810a520 2024-08-20T00:58:50.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.444+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f229819c050 con 0x7f229810a520 2024-08-20T00:58:50.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.447+0000 7f2294c00640 1 -- 172.21.15.138:0/235352042 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v22)=0 v22) ==== 129+0+0 (secure 0 0 0) 0x7f2284063660 con 0x7f229810a520 2024-08-20T00:58:50.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.450+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 msgr2=0x7f227c07ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.450+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 0x7f227c07ba10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f22880095f0 tx=0x7f2288005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:50.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.450+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 msgr2=0x7f229819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.450+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 0x7f229819b460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f228400e9f0 tx=0x7f228400eeb0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 shutdown_connections 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f227c079550 0x7f227c07ba10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f229810aea0 0x7f22981a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f229810a520 0x7f229819b460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 --2- 172.21.15.138:0/235352042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2298104b10 0x7f229819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 >> 172.21.15.138:0/235352042 conn(0x7f2298100540 msgr2=0x7f2298103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 shutdown_connections 2024-08-20T00:58:50.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.451+0000 7f229d400640 1 -- 172.21.15.138:0/235352042 wait complete. 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.639+0000 7f2277600640 1 -- 172.21.15.138:0/2113635139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010b030 msgr2=0x7f2270111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.639+0000 7f2277600640 1 --2- 172.21.15.138:0/2113635139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010b030 0x7f2270111830 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2268009ca0 tx=0x7f226802a8c0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.639+0000 7f2277600640 1 -- 172.21.15.138:0/2113635139 shutdown_connections 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 --2- 172.21.15.138:0/2113635139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010b030 0x7f2270111830 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 --2- 172.21.15.138:0/2113635139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010a6b0 0x7f227010aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 --2- 172.21.15.138:0/2113635139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 0x7f2270105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 -- 172.21.15.138:0/2113635139 >> 172.21.15.138:0/2113635139 conn(0x7f2270100730 msgr2=0x7f2270102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 -- 172.21.15.138:0/2113635139 shutdown_connections 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 -- 172.21.15.138:0/2113635139 wait complete. 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.640+0000 7f2277600640 1 Processor -- start 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 -- start start 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 0x7f227019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010b030 0x7f22701a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2270114d40 con 0x7f227010a6b0 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2270114bc0 con 0x7f227010b030 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2277600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2270114ec0 con 0x7f2270104ca0 2024-08-20T00:58:50.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f226fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f226fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39612/0 (socket says 172.21.15.138:39612) 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f226fe00640 1 -- 172.21.15.138:0/2018683763 learned_addr learned my addr 172.21.15.138:0/2018683763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2275600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010b030 0x7f22701a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.641+0000 7f2274c00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 0x7f227019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 -- 172.21.15.138:0/2018683763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 msgr2=0x7f227019b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 0x7f227019b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 -- 172.21.15.138:0/2018683763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010b030 msgr2=0x7f22701a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010b030 0x7f22701a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 -- 172.21.15.138:0/2018683763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22701a11d0 con 0x7f227010a6b0 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226fe00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f225c00d950 tx=0x7f225c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:50.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c017030 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f225c004020 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22701a14c0 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225c0047a0 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.642+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22701a7d50 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.643+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2270105080 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.645+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) ==== 100893+0+0 (secure 0 0 0) 0x7f225c009a30 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.646+0000 7f226d600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 0x7f225407b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.646+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f225c09a340 con 0x7f227010a6b0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.646+0000 7f2274c00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 0x7f225407b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.647+0000 7f2274c00640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 0x7f225407b960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f226000bbe0 tx=0x7f226000b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:50.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.650+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f225c063040 con 0x7f227010a6b0 2024-08-20T00:58:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:50 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:50 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T00:58:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:50 smithi138 ceph-mon[18076]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-20T00:58:50.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:50 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:50.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/235352042' entity='client.admin' 2024-08-20T00:58:50.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.845+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f227019c250 con 0x7f227010a6b0 2024-08-20T00:58:50.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.850+0000 7f226d600640 1 -- 172.21.15.138:0/2018683763 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v23)=0 v23) ==== 120+0+0 (secure 0 0 0) 0x7f225c067590 con 0x7f227010a6b0 2024-08-20T00:58:50.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 msgr2=0x7f225407b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 0x7f225407b960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f226000bbe0 tx=0x7f226000b390 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 msgr2=0x7f227019b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f225c00d950 tx=0x7f225c00de10 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 shutdown_connections 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f22540794a0 0x7f225407b960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f227010b030 0x7f22701a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f227010a6b0 0x7f227019b660 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 --2- 172.21.15.138:0/2018683763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2270104ca0 0x7f227019b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 >> 172.21.15.138:0/2018683763 conn(0x7f2270100730 msgr2=0x7f2270102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 shutdown_connections 2024-08-20T00:58:50.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:50.852+0000 7f2277600640 1 -- 172.21.15.138:0/2018683763 wait complete. 2024-08-20T00:58:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:50 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:50 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T00:58:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:50 smithi167 ceph-mon[21965]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-20T00:58:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:50 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:50 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/235352042' entity='client.admin' 2024-08-20T00:58:50.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:50 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:50 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-08-20T00:58:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:50 smithi142 ceph-mon[21799]: mgrmap e18: x(active, since 15s), standbys: z 2024-08-20T00:58:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:50 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:58:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:50 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/235352042' entity='client.admin' 2024-08-20T00:58:51.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 -- 172.21.15.138:0/421235331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16010b030 msgr2=0x7fd160111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 --2- 172.21.15.138:0/421235331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16010b030 0x7fd160111830 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd158009ca0 tx=0x7fd15802a8c0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 -- 172.21.15.138:0/421235331 shutdown_connections 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 --2- 172.21.15.138:0/421235331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd16010b030 0x7fd160111830 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 --2- 172.21.15.138:0/421235331 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010a6b0 0x7fd16010aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 --2- 172.21.15.138:0/421235331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd160104ca0 0x7fd160105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 -- 172.21.15.138:0/421235331 >> 172.21.15.138:0/421235331 conn(0x7fd160100730 msgr2=0x7fd160102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 -- 172.21.15.138:0/421235331 shutdown_connections 2024-08-20T00:58:51.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.671+0000 7fd167200640 1 -- 172.21.15.138:0/421235331 wait complete. 2024-08-20T00:58:51.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.672+0000 7fd167200640 1 Processor -- start 2024-08-20T00:58:51.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.672+0000 7fd167200640 1 -- start start 2024-08-20T00:58:51.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd160104ca0 0x7fd16019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 0x7fd1601a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd160114d40 con 0x7fd160104ca0 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd160114bc0 con 0x7fd16010a6b0 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd167200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd160114ec0 con 0x7fd16010b030 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd160104ca0 0x7fd16019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:49956/0 (socket says 172.21.15.138:49956) 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 -- 172.21.15.138:0/3197551243 learned_addr learned my addr 172.21.15.138:0/3197551243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd165200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 0x7fd1601a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 -- 172.21.15.138:0/3197551243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 msgr2=0x7fd1601a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 0x7fd1601a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 -- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd160104ca0 msgr2=0x7fd16019b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd160104ca0 0x7fd16019b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd15f400640 1 -- 172.21.15.138:0/3197551243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1601a1200 con 0x7fd16010a6b0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.673+0000 7fd165200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 0x7fd1601a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd15f400640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9f0 tx=0x7fd14c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c00cd80 con 0x7fd16010a6b0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1601a14f0 con 0x7fd16010a6b0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd14c0054d0 con 0x7fd16010a6b0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c010690 con 0x7fd16010a6b0 2024-08-20T00:58:51.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.674+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1601a7bb0 con 0x7fd16010a6b0 2024-08-20T00:58:51.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.675+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd160105080 con 0x7fd16010a6b0 2024-08-20T00:58:51.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.676+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) ==== 100893+0+0 (secure 0 0 0) 0x7fd14c004e90 con 0x7fd16010a6b0 2024-08-20T00:58:51.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.677+0000 7fd15cc00640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 0x7fd14407b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.678+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd14c09ae80 con 0x7fd16010a6b0 2024-08-20T00:58:51.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.678+0000 7fd15fe00640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 0x7fd14407b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.678+0000 7fd15fe00640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 0x7fd14407b960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd150009bd0 tx=0x7fd150005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:51.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.681+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd14c064210 con 0x7fd16010a6b0 2024-08-20T00:58:51.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.323+0000 7faad3c00640 1 -- 172.21.15.138:0/2316762145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 msgr2=0x7faacc105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.323+0000 7faad3c00640 1 --2- 172.21.15.138:0/2316762145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc105080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faac0009900 tx=0x7faac002a410 comp rx=0 tx=0).stop 2024-08-20T00:58:51.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 -- 172.21.15.138:0/2316762145 shutdown_connections 2024-08-20T00:58:51.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 --2- 172.21.15.138:0/2316762145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 --2- 172.21.15.138:0/2316762145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 --2- 172.21.15.138:0/2316762145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc105080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 -- 172.21.15.138:0/2316762145 >> 172.21.15.138:0/2316762145 conn(0x7faacc100730 msgr2=0x7faacc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 -- 172.21.15.138:0/2316762145 shutdown_connections 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.324+0000 7faad3c00640 1 -- 172.21.15.138:0/2316762145 wait complete. 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 Processor -- start 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 -- start start 2024-08-20T00:58:51.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7faacc114d40 con 0x7faacc10a6b0 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7faacc114bc0 con 0x7faacc104ca0 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad3c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7faacc114ec0 con 0x7faacc10b030 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faacbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad1200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:49902/0 (socket says 172.21.15.138:49902) 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faacbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39576/0 (socket says 172.21.15.138:39576) 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad1200640 1 -- 172.21.15.138:0/1096871605 learned_addr learned my addr 172.21.15.138:0/1096871605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.325+0000 7faad1c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 -- 172.21.15.138:0/1096871605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 msgr2=0x7faacc1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 -- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 msgr2=0x7faacc19b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 -- 172.21.15.138:0/1096871605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc1a1280 con 0x7faacc104ca0 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faacbe00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad1200640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faac0005ab0 tx=0x7faac0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:51.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac003f070 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faac0031b60 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac0035030 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.326+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc1a1510 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.327+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc1a7c30 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.328+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc105080 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.328+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) ==== 50765+0+0 (secure 0 0 0) 0x7faac0042070 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.328+0000 7faac9600640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 0x7faab4041270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.328+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7faac0073af0 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.328+0000 7faacbe00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 0x7faab4041270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.329+0000 7faacbe00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 0x7faab4041270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faacc10a2f0 tx=0x7faab8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.333+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7faac004a260 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.523+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7faacc10ce70 con 0x7faacc104ca0 2024-08-20T00:58:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.528+0000 7faac9600640 1 -- 172.21.15.138:0/1096871605 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v21)=0 v21) ==== 119+0+0 (secure 0 0 0) 0x7faac0031d10 con 0x7faacc104ca0 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 msgr2=0x7faab4041270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 0x7faab4041270 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7faacc10a2f0 tx=0x7faab8005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 msgr2=0x7faacc19b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7faac0005ab0 tx=0x7faac0003c10 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 shutdown_connections 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7faab403edb0 0x7faab4041270 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7faacc10b030 0x7faacc1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc10a6b0 0x7faacc19b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 --2- 172.21.15.138:0/1096871605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faacc104ca0 0x7faacc19b1d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.530+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 >> 172.21.15.138:0/1096871605 conn(0x7faacc100730 msgr2=0x7faacc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.531+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 shutdown_connections 2024-08-20T00:58:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:49.531+0000 7faad3c00640 1 -- 172.21.15.138:0/1096871605 wait complete. 2024-08-20T00:58:51.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.862+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fd14c05fda0 con 0x7fd16010a6b0 2024-08-20T00:58:51.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.877+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd16019c280 con 0x7fd16010a6b0 2024-08-20T00:58:51.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.884+0000 7fd15cc00640 1 -- 172.21.15.138:0/3197551243 <== mon.2 v2:172.21.15.142:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7fd14c00f3c0 con 0x7fd16010a6b0 2024-08-20T00:58:51.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 msgr2=0x7fd14407b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 0x7fd14407b960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd150009bd0 tx=0x7fd150005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:51.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 msgr2=0x7fd16019b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:51.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00e9f0 tx=0x7fd14c00edf0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 shutdown_connections 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd1440794a0 0x7fd14407b960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd16010b030 0x7fd1601a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd16010a6b0 0x7fd16019b690 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 --2- 172.21.15.138:0/3197551243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd160104ca0 0x7fd16019b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 >> 172.21.15.138:0/3197551243 conn(0x7fd160100730 msgr2=0x7fd160102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 shutdown_connections 2024-08-20T00:58:51.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:51.887+0000 7fd167200640 1 -- 172.21.15.138:0/3197551243 wait complete. 2024-08-20T00:58:52.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2018683763' entity='client.admin' 2024-08-20T00:58:52.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: Standby manager daemon y started 2024-08-20T00:58:52.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T00:58:52.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:52.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T00:58:52.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:51 smithi138 ceph-mon[18076]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2018683763' entity='client.admin' 2024-08-20T00:58:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: Standby manager daemon y started 2024-08-20T00:58:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T00:58:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:52.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T00:58:52.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:51 smithi167 ceph-mon[21965]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2018683763' entity='client.admin' 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: Standby manager daemon y started 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-08-20T00:58:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:51 smithi142 ceph-mon[21799]: from='mgr.? 172.21.15.142:0/4224196990' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-08-20T00:58:53.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:52 smithi167 ceph-mon[21965]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-20T00:58:53.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:52 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T00:58:53.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:52 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:53.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:52 smithi142 ceph-mon[21799]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-20T00:58:53.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:52 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T00:58:53.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:52 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:53.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:52 smithi138 ceph-mon[18076]: mgrmap e19: x(active, since 17s), standbys: z, y 2024-08-20T00:58:53.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:52 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-08-20T00:58:53.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:52 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:53.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 -- 172.21.15.138:0/566603552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075210 msgr2=0x7fdeb80755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:53.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 --2- 172.21.15.138:0/566603552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075210 0x7fdeb80755f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009900 tx=0x7fdeac02a410 comp rx=0 tx=0).stop 2024-08-20T00:58:53.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 -- 172.21.15.138:0/566603552 shutdown_connections 2024-08-20T00:58:53.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 --2- 172.21.15.138:0/566603552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb810ef10 0x7fdeb8111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 --2- 172.21.15.138:0/566603552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb8075b30 0x7fdeb810e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 --2- 172.21.15.138:0/566603552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075210 0x7fdeb80755f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 -- 172.21.15.138:0/566603552 >> 172.21.15.138:0/566603552 conn(0x7fdeb80fe350 msgr2=0x7fdeb8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 -- 172.21.15.138:0/566603552 shutdown_connections 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.818+0000 7fdebe800640 1 -- 172.21.15.138:0/566603552 wait complete. 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.819+0000 7fdebe800640 1 Processor -- start 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 -- start start 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 0x7fdeb819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:53.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb810ef10 0x7fdeb81a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8114b40 con 0x7fdeb8075b30 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb81149c0 con 0x7fdeb8075210 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebe800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdeb8114cc0 con 0x7fdeb810ef10 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 0x7fdeb819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39654/0 (socket says 172.21.15.138:39654) 2024-08-20T00:58:53.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.820+0000 7fdebca00640 1 -- 172.21.15.138:0/1953688384 learned_addr learned my addr 172.21.15.138:0/1953688384 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebde00640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb810ef10 0x7fdeb81a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 -- 172.21.15.138:0/1953688384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb810ef10 msgr2=0x7fdeb81a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb810ef10 0x7fdeb81a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 -- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 msgr2=0x7fdeb819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 -- 172.21.15.138:0/1953688384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeb81a1020 con 0x7fdeb8075210 2024-08-20T00:58:53.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebca00640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.821+0000 7fdebd400640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 0x7fdeb819af70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdeac02aa00 tx=0x7fdeac003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.822+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac03f070 con 0x7fdeb8075210 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.822+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeb81a12b0 con 0x7fdeb8075210 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.822+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeb81a79d0 con 0x7fdeb8075210 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.822+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeac031b50 con 0x7fdeb8075210 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.822+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeac049060 con 0x7fdeb8075210 2024-08-20T00:58:53.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.823+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeb80755f0 con 0x7fdeb8075210 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.825+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fdeac042070 con 0x7fdeb8075210 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.826+0000 7fdeb6000640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 0x7fdea00b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.826+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fdeac102850 con 0x7fdeb8075210 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.827+0000 7fdebca00640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 0x7fdea00b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.827+0000 7fdebca00640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 0x7fdea00b6180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8100af0 tx=0x7fdea4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:53.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:53.829+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdeac0cb3b0 con 0x7fdeb8075210 2024-08-20T00:58:54.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.037+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fdeb81a7c70 con 0x7fdeb8075210 2024-08-20T00:58:54.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.043+0000 7fdeb6000640 1 -- 172.21.15.138:0/1953688384 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7fdeac0cf900 con 0x7fdeb8075210 2024-08-20T00:58:54.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 msgr2=0x7fdea00b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 0x7fdea00b6180 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8100af0 tx=0x7fdea4005f90 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 msgr2=0x7fdeb819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 0x7fdeb819af70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdeac02aa00 tx=0x7fdeac003b90 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 shutdown_connections 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdea00b3cc0 0x7fdea00b6180 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdeb810ef10 0x7fdeb81a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdeb8075b30 0x7fdeb819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 --2- 172.21.15.138:0/1953688384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdeb8075210 0x7fdeb819af70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 >> 172.21.15.138:0/1953688384 conn(0x7fdeb80fe350 msgr2=0x7fdeb8111090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 shutdown_connections 2024-08-20T00:58:54.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:54.045+0000 7fdebe800640 1 -- 172.21.15.138:0/1953688384 wait complete. 2024-08-20T00:58:54.709 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:54.710 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:54.711 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:55.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:55 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:55.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:55 smithi138 ceph-mon[18076]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:55.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:55 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:55.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:55 smithi167 ceph-mon[21965]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:55.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:55 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:55.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:55 smithi142 ceph-mon[21799]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:56.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 -- 172.21.15.138:0/784961326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0751f0 msgr2=0x7ff17c0755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 --2- 172.21.15.138:0/784961326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0751f0 0x7ff17c0755d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009900 tx=0x7ff16c02a410 comp rx=0 tx=0).stop 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 -- 172.21.15.138:0/784961326 shutdown_connections 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 --2- 172.21.15.138:0/784961326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c10ef30 0x7ff17c111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 --2- 172.21.15.138:0/784961326 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c075b10 0x7ff17c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 --2- 172.21.15.138:0/784961326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c0751f0 0x7ff17c0755d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 -- 172.21.15.138:0/784961326 >> 172.21.15.138:0/784961326 conn(0x7ff17c0fe350 msgr2=0x7ff17c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 -- 172.21.15.138:0/784961326 shutdown_connections 2024-08-20T00:58:56.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.581+0000 7ff180a00640 1 -- 172.21.15.138:0/784961326 wait complete. 2024-08-20T00:58:56.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 Processor -- start 2024-08-20T00:58:56.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 -- start start 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c075b10 0x7ff17c19b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 0x7ff17c1a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c114b60 con 0x7ff17c10ef30 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1149e0 con 0x7ff17c075b10 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.582+0000 7ff180a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c114ce0 con 0x7ff17c0751f0 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42852/0 (socket says 172.21.15.138:42852) 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 -- 172.21.15.138:0/138289363 learned_addr learned my addr 172.21.15.138:0/138289363 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:56.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17b200640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 0x7ff17c1a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff179e00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c075b10 0x7ff17c19b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 -- 172.21.15.138:0/138289363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c075b10 msgr2=0x7ff17c19b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c075b10 0x7ff17c19b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 -- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 msgr2=0x7ff17c1a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 0x7ff17c1a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17a800640 1 -- 172.21.15.138:0/138289363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1a1040 con 0x7ff17c0751f0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.583+0000 7ff17b200640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 0x7ff17c1a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff17a800640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005ab0 tx=0x7ff16c003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c03f070 con 0x7ff17c0751f0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c031d60 con 0x7ff17c0751f0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c035030 con 0x7ff17c0751f0 2024-08-20T00:58:56.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1a12d0 con 0x7ff17c0751f0 2024-08-20T00:58:56.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.584+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1a79f0 con 0x7ff17c0751f0 2024-08-20T00:58:56.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.585+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c0755d0 con 0x7ff17c0751f0 2024-08-20T00:58:56.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.587+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7ff16c042070 con 0x7ff17c0751f0 2024-08-20T00:58:56.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.589+0000 7ff173400640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 0x7ff1600b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:56.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.589+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff16c102790 con 0x7ff17c0751f0 2024-08-20T00:58:56.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.589+0000 7ff179e00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 0x7ff1600b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:56.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.589+0000 7ff179e00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 0x7ff1600b6180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff17c100af0 tx=0x7ff16800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:56.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.593+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff16c0cb2f0 con 0x7ff17c0751f0 2024-08-20T00:58:56.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.791+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7ff17c1a7d40 con 0x7ff17c0751f0 2024-08-20T00:58:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.797+0000 7ff173400640 1 -- 172.21.15.138:0/138289363 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v26)=0 v26) ==== 138+0+0 (secure 0 0 0) 0x7ff16c0cf840 con 0x7ff17c0751f0 2024-08-20T00:58:56.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.799+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 msgr2=0x7ff1600b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:56.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.799+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 0x7ff1600b6180 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff17c100af0 tx=0x7ff16800b410 comp rx=0 tx=0).stop 2024-08-20T00:58:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.799+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 msgr2=0x7ff17c19af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.799+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff16c005ab0 tx=0x7ff16c003c80 comp rx=0 tx=0).stop 2024-08-20T00:58:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 shutdown_connections 2024-08-20T00:58:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1600b3cc0 0x7ff1600b6180 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff17c10ef30 0x7ff17c1a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff17c075b10 0x7ff17c19b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 --2- 172.21.15.138:0/138289363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff17c0751f0 0x7ff17c19af90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.800+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 >> 172.21.15.138:0/138289363 conn(0x7ff17c0fe350 msgr2=0x7ff17c1110a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.801+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 shutdown_connections 2024-08-20T00:58:56.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:56.801+0000 7ff180a00640 1 -- 172.21.15.138:0/138289363 wait complete. 2024-08-20T00:58:57.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:57 smithi138 ceph-mon[18076]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:57.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:57 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:57.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:57 smithi167 ceph-mon[21965]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:57.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:57 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:57.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:57 smithi142 ceph-mon[21799]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:57.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:57 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:58.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 -- 172.21.15.138:0/2528747237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f400410a520 msgr2=0x7f400410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 --2- 172.21.15.138:0/2528747237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f400410a520 0x7f400410a960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009900 tx=0x7f3ff002a3f0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 -- 172.21.15.138:0/2528747237 shutdown_connections 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 --2- 172.21.15.138:0/2528747237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410aea0 0x7f40041116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 --2- 172.21.15.138:0/2528747237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f400410a520 0x7f400410a960 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 --2- 172.21.15.138:0/2528747237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4004104b10 0x7f4004104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 -- 172.21.15.138:0/2528747237 >> 172.21.15.138:0/2528747237 conn(0x7f4004100540 msgr2=0x7f4004102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 -- 172.21.15.138:0/2528747237 shutdown_connections 2024-08-20T00:58:58.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.175+0000 7f400b000640 1 -- 172.21.15.138:0/2528747237 wait complete. 2024-08-20T00:58:58.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 Processor -- start 2024-08-20T00:58:58.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 -- start start 2024-08-20T00:58:58.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410a520 0x7f400419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400410aea0 0x7f40041a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4004114b70 con 0x7f4004104b10 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f40041149f0 con 0x7f400410aea0 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.176+0000 7f400b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4004114cf0 con 0x7f400410a520 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60668/0 (socket says 172.21.15.138:60668) 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 -- 172.21.15.138:0/2544146509 learned_addr learned my addr 172.21.15.138:0/2544146509 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009200640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410a520 0x7f400419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:58.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f400a600640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400410aea0 0x7f40041a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 -- 172.21.15.138:0/2544146509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410a520 msgr2=0x7f400419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410a520 0x7f400419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 -- 172.21.15.138:0/2544146509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400410aea0 msgr2=0x7f40041a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400410aea0 0x7f40041a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 -- 172.21.15.138:0/2544146509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40041a0fd0 con 0x7f4004104b10 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.177+0000 7f4009c00640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009d50 tx=0x7f3ff800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.178+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8003b70 con 0x7f4004104b10 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.178+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8003d10 con 0x7f4004104b10 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.178+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40041a12c0 con 0x7f4004104b10 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.178+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80108a0 con 0x7f4004104b10 2024-08-20T00:58:58.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.178+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40041a7b50 con 0x7f4004104b10 2024-08-20T00:58:58.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.179+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4004104ef0 con 0x7f4004104b10 2024-08-20T00:58:58.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.182+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f3ff8004000 con 0x7f4004104b10 2024-08-20T00:58:58.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.183+0000 7f4002000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 0x7f3fec0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:58.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.183+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3ff80e2780 con 0x7f4004104b10 2024-08-20T00:58:58.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.184+0000 7f4009200640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 0x7f3fec0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:58.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.184+0000 7f4009200640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 0x7f3fec0b6180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005ab0 tx=0x7f3ff0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:58.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.185+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3ff80ab9c0 con 0x7f4004104b10 2024-08-20T00:58:58.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.374+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7f400419c050 con 0x7f4004104b10 2024-08-20T00:58:58.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.378+0000 7f4002000640 1 -- 172.21.15.138:0/2544146509 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v28)=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f400419c050 con 0x7f4004104b10 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.380+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 msgr2=0x7f3fec0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.380+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 0x7f3fec0b6180 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005ab0 tx=0x7f3ff0005a20 comp rx=0 tx=0).stop 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.380+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 msgr2=0x7f400419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.380+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8009d50 tx=0x7f3ff800c6a0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 shutdown_connections 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3fec0b3cc0 0x7f3fec0b6180 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f400410aea0 0x7f40041a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f400410a520 0x7f400419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 --2- 172.21.15.138:0/2544146509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4004104b10 0x7f400419af20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:58.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 >> 172.21.15.138:0/2544146509 conn(0x7f4004100540 msgr2=0x7f4004102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:58.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 shutdown_connections 2024-08-20T00:58:58.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:58.381+0000 7f400b000640 1 -- 172.21.15.138:0/2544146509 wait complete. 2024-08-20T00:58:58.423 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.424 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.643 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.643 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.643 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.644 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.644 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:58:58.957 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:58 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:58:58.957 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:58 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2544146509' entity='client.admin' 2024-08-20T00:58:58.957 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:58:58 smithi138 ceph-mon[18076]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:58 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:58:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:58 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2544146509' entity='client.admin' 2024-08-20T00:58:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:58:58 smithi167 ceph-mon[21965]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:59.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:58 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:58:59.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:58 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2544146509' entity='client.admin' 2024-08-20T00:58:59.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:58:58 smithi142 ceph-mon[21799]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 -- 172.21.15.138:0/2471563086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 msgr2=0x7f705c113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 --2- 172.21.15.138:0/2471563086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c113150 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7048009860 tx=0x7f704802a340 comp rx=0 tx=0).stop 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 -- 172.21.15.138:0/2471563086 shutdown_connections 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 --2- 172.21.15.138:0/2471563086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c113690 0x7f705c115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 --2- 172.21.15.138:0/2471563086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c113150 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 --2- 172.21.15.138:0/2471563086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c077400 0x7f705c0777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 -- 172.21.15.138:0/2471563086 >> 172.21.15.138:0/2471563086 conn(0x7f705c100730 msgr2=0x7f705c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 -- 172.21.15.138:0/2471563086 shutdown_connections 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.569+0000 7f7061c00640 1 -- 172.21.15.138:0/2471563086 wait complete. 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.570+0000 7f7061c00640 1 Processor -- start 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.570+0000 7f7061c00640 1 -- start start 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c077400 0x7f705c19f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 0x7f705c1a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f705c119070 con 0x7f705c113690 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f705c118ef0 con 0x7f705c077d20 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f7061c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1191f0 con 0x7f705c077400 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 0x7f705c1a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:59.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50012/0 (socket says 172.21.15.138:50012) 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 -- 172.21.15.138:0/3410849964 learned_addr learned my addr 172.21.15.138:0/3410849964 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705aa00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c077400 0x7f705c19f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 -- 172.21.15.138:0/3410849964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c077400 msgr2=0x7f705c19f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c077400 0x7f705c19f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 -- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 msgr2=0x7f705c1a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 0x7f705c1a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705a000640 1 -- 172.21.15.138:0/3410849964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c1a5630 con 0x7f705c077d20 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.571+0000 7f705b400640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 0x7f705c1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f705a000640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7048035040 tx=0x7f7048032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7048037040 con 0x7f705c077d20 2024-08-20T00:58:59.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7048041070 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c1a58c0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70480035f0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.572+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c1abfe0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.573+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c0777e0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.576+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f7048004050 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.578+0000 7f7053400640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 0x7f70400b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.578+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7048101ac0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.578+0000 7f705aa00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 0x7f70400b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.579+0000 7f705aa00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 0x7f70400b60d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f704c009530 tx=0x7f704c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.582+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f70480cacf0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.778+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f705c1ac2d0 con 0x7f705c077d20 2024-08-20T00:58:59.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.784+0000 7f7053400640 1 -- 172.21.15.138:0/3410849964 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v27)=0 v27) ==== 106+0+0 (secure 0 0 0) 0x7f70480ca5f0 con 0x7f705c077d20 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 msgr2=0x7f70400b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 0x7f70400b60d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f704c009530 tx=0x7f704c009260 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 msgr2=0x7f705c19fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7048035040 tx=0x7f7048032d50 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 shutdown_connections 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70400b3c10 0x7f70400b60d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f705c113690 0x7f705c1a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705c077d20 0x7f705c19fac0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 --2- 172.21.15.138:0/3410849964 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705c077400 0x7f705c19f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 >> 172.21.15.138:0/3410849964 conn(0x7f705c100730 msgr2=0x7f705c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 shutdown_connections 2024-08-20T00:58:59.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:57.786+0000 7f7061c00640 1 -- 172.21.15.138:0/3410849964 wait complete. 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 -- 172.21.15.138:0/3628796447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c101500 msgr2=0x7ff96c1018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 --2- 172.21.15.138:0/3628796447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c101500 0x7ff96c1018e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff95c009930 tx=0x7ff95c02a440 comp rx=0 tx=0).stop 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 -- 172.21.15.138:0/3628796447 shutdown_connections 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 --2- 172.21.15.138:0/3628796447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108f80 0x7ff96c10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 --2- 172.21.15.138:0/3628796447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c108600 0x7ff96c108a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 --2- 172.21.15.138:0/3628796447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c101500 0x7ff96c1018e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 -- 172.21.15.138:0/3628796447 >> 172.21.15.138:0/3628796447 conn(0x7ff96c076cc0 msgr2=0x7ff96c0770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 -- 172.21.15.138:0/3628796447 shutdown_connections 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.687+0000 7ff971000640 1 -- 172.21.15.138:0/3628796447 wait complete. 2024-08-20T00:59:00.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.688+0000 7ff971000640 1 Processor -- start 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.688+0000 7ff971000640 1 -- start start 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c101500 0x7ff96c19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108600 0x7ff96c19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c112b40 con 0x7ff96c108f80 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c1129c0 con 0x7ff96c108600 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff971000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c112cc0 con 0x7ff96c101500 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60708/0 (socket says 172.21.15.138:60708) 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 -- 172.21.15.138:0/438018529 learned_addr learned my addr 172.21.15.138:0/438018529 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff969e00640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c101500 0x7ff96c19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:00.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff969400640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108600 0x7ff96c19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 -- 172.21.15.138:0/438018529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c101500 msgr2=0x7ff96c19b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c101500 0x7ff96c19b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 -- 172.21.15.138:0/438018529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108600 msgr2=0x7ff96c19b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108600 0x7ff96c19b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.689+0000 7ff96a800640 1 -- 172.21.15.138:0/438018529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1a12b0 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff96a800640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eaa0 tx=0x7ff96400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96400cdb0 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff964005020 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c1a15a0 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9640116d0 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.690+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c1a7c60 con 0x7ff96c108f80 2024-08-20T00:59:00.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.691+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c1018e0 con 0x7ff96c108f80 2024-08-20T00:59:00.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.693+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7ff9640051d0 con 0x7ff96c108f80 2024-08-20T00:59:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.696+0000 7ff962a00640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 0x7ff9500b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.696+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7ff9640e25c0 con 0x7ff96c108f80 2024-08-20T00:59:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.696+0000 7ff969e00640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 0x7ff9500b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:00.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.697+0000 7ff969e00640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 0x7ff9500b6050 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff95c031440 tx=0x7ff95c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:00.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.698+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff964016030 con 0x7ff96c108f80 2024-08-20T00:59:00.783 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:00 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:59:00.783 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:00 smithi138 ceph-mon[18076]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:00.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.890+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7ff96c19c330 con 0x7ff96c108f80 2024-08-20T00:59:00.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.894+0000 7ff962a00640 1 -- 172.21.15.138:0/438018529 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v30)=0 v30) ==== 125+0+0 (secure 0 0 0) 0x7ff96c19c330 con 0x7ff96c108f80 2024-08-20T00:59:00.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 msgr2=0x7ff9500b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:00.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 0x7ff9500b6050 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff95c031440 tx=0x7ff95c005a20 comp rx=0 tx=0).stop 2024-08-20T00:59:00.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 msgr2=0x7ff96c1a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:00.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff96400eaa0 tx=0x7ff96400ef60 comp rx=0 tx=0).stop 2024-08-20T00:59:00.896 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 shutdown_connections 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9500b3b90 0x7ff9500b6050 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff96c108f80 0x7ff96c1a0ae0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff96c108600 0x7ff96c19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 --2- 172.21.15.138:0/438018529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff96c101500 0x7ff96c19b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 >> 172.21.15.138:0/438018529 conn(0x7ff96c076cc0 msgr2=0x7ff96c1076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 shutdown_connections 2024-08-20T00:59:00.897 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:00.897+0000 7ff971000640 1 -- 172.21.15.138:0/438018529 wait complete. 2024-08-20T00:59:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:00 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:59:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:00 smithi167 ceph-mon[21965]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:00 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:59:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:00 smithi142 ceph-mon[21799]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:01.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 -- 172.21.15.138:0/4209603630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 msgr2=0x7f1414115a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:01.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 --2- 172.21.15.138:0/4209603630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f1414115a70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f140c009ca0 tx=0x7f140c02a8c0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 -- 172.21.15.138:0/4209603630 shutdown_connections 2024-08-20T00:59:01.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 --2- 172.21.15.138:0/4209603630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f1414115a70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 --2- 172.21.15.138:0/4209603630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 0x7f141410baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 --2- 172.21.15.138:0/4209603630 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f14141092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 -- 172.21.15.138:0/4209603630 >> 172.21.15.138:0/4209603630 conn(0x7f1414078550 msgr2=0x7f141407a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 -- 172.21.15.138:0/4209603630 shutdown_connections 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.502+0000 7f1413200640 1 -- 172.21.15.138:0/4209603630 wait complete. 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 Processor -- start 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 -- start start 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f141419f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 0x7f141419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1414118ee0 con 0x7f141410c030 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1414118d60 con 0x7f141410b6b0 2024-08-20T00:59:01.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.503+0000 7f1413200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1414119060 con 0x7f1414108f10 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1412800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f141419f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1412800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60684/0 (socket says 172.21.15.138:60684) 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 0x7f141419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1412800640 1 -- 172.21.15.138:0/1492285850 learned_addr learned my addr 172.21.15.138:0/1492285850 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 -- 172.21.15.138:0/1492285850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 msgr2=0x7f141419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 0x7f141419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 -- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 msgr2=0x7f14141a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1411e00640 1 -- 172.21.15.138:0/1492285850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14141a5380 con 0x7f1414108f10 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.504+0000 7f1412800640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f1411e00640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f141419f2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f140400e9f0 tx=0x7f140400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:01.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140400ccc0 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14141a5670 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1404005020 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1404004a00 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.505+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14141abd30 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.506+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14141092f0 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.508+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f1404004ba0 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.509+0000 7f140aa00640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 0x7f13f80b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.510+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f14040e1fa0 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.510+0000 7f1411400640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 0x7f13f80b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.510+0000 7f1411400640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 0x7f13f80b6180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1414108970 tx=0x7f140000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.511+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f14040ab150 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.701+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f14141a0400 con 0x7f1414108f10 2024-08-20T00:59:01.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.708+0000 7f140aa00640 1 -- 172.21.15.138:0/1492285850 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v29)=0 v29) ==== 120+0+0 (secure 0 0 0) 0x7f14141a0400 con 0x7f1414108f10 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 msgr2=0x7f13f80b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 0x7f13f80b6180 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1414108970 tx=0x7f140000b410 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 msgr2=0x7f141419f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f141419f2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f140400e9f0 tx=0x7f140400eeb0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 shutdown_connections 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13f80b3cc0 0x7f13f80b6180 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f141410c030 0x7f14141a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f141410b6b0 0x7f141419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 --2- 172.21.15.138:0/1492285850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1414108f10 0x7f141419f2d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 >> 172.21.15.138:0/1492285850 conn(0x7f1414078550 msgr2=0x7f141407a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 shutdown_connections 2024-08-20T00:59:01.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:58:59.710+0000 7f1413200640 1 -- 172.21.15.138:0/1492285850 wait complete. 2024-08-20T00:59:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:01 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/438018529' entity='client.admin' 2024-08-20T00:59:02.173 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi138') 2024-08-20T00:59:02.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:01 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/438018529' entity='client.admin' 2024-08-20T00:59:02.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:01 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/438018529' entity='client.admin' 2024-08-20T00:59:02.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 -- 172.21.15.138:0/222813372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 msgr2=0x7fa244111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 --2- 172.21.15.138:0/222813372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa244111840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa23c009ca0 tx=0x7fa23c02a8c0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 -- 172.21.15.138:0/222813372 shutdown_connections 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 --2- 172.21.15.138:0/222813372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa244111840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 --2- 172.21.15.138:0/222813372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa24410a6c0 0x7fa24410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 --2- 172.21.15.138:0/222813372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa244104cb0 0x7fa244105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 -- 172.21.15.138:0/222813372 >> 172.21.15.138:0/222813372 conn(0x7fa244100720 msgr2=0x7fa244102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 -- 172.21.15.138:0/222813372 shutdown_connections 2024-08-20T00:59:02.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.377+0000 7fa24a400640 1 -- 172.21.15.138:0/222813372 wait complete. 2024-08-20T00:59:02.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 Processor -- start 2024-08-20T00:59:02.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 -- start start 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 0x7fa24419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa2441a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa244114d10 con 0x7fa24410b040 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa244114b90 con 0x7fa24410a6c0 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.378+0000 7fa24a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa244114e90 con 0x7fa244104cb0 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa2441a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa23aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 0x7fa24419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39364/0 (socket says 172.21.15.138:39364) 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 -- 172.21.15.138:0/4163914744 learned_addr learned my addr 172.21.15.138:0/4163914744 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 -- 172.21.15.138:0/4163914744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 msgr2=0x7fa24419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:02.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 0x7fa24419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 -- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 msgr2=0x7fa2441a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa2441a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 -- 172.21.15.138:0/4163914744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2441a11f0 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa23aa00640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 0x7fa24419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.379+0000 7fa243400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa23400ea60 tx=0x7fa23400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.380+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23400cde0 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.380+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2441a14e0 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.380+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2340035f0 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.380+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234010690 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.380+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2441a7d70 con 0x7fa244104cb0 2024-08-20T00:59:02.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.381+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa244105090 con 0x7fa244104cb0 2024-08-20T00:59:02.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.383+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fa2340037a0 con 0x7fa244104cb0 2024-08-20T00:59:02.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.384+0000 7fa240c00640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 0x7fa2280b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:02.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.384+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa2340e24c0 con 0x7fa244104cb0 2024-08-20T00:59:02.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.385+0000 7fa23aa00640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 0x7fa2280b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:02.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.385+0000 7fa23aa00640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 0x7fa2280b6050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa244106be0 tx=0x7fa22c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:02.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.386+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa2340ab670 con 0x7fa244104cb0 2024-08-20T00:59:02.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.581+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fa24419c270 con 0x7fa244104cb0 2024-08-20T00:59:02.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.588+0000 7fa240c00640 1 -- 172.21.15.138:0/4163914744 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v31)=0 v31) ==== 122+0+0 (secure 0 0 0) 0x7fa2340aaf70 con 0x7fa244104cb0 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 msgr2=0x7fa2280b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 0x7fa2280b6050 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa244106be0 tx=0x7fa22c005f90 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 msgr2=0x7fa24419b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa23400ea60 tx=0x7fa23400ef20 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 shutdown_connections 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2280b3b90 0x7fa2280b6050 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa24410b040 0x7fa2441a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa24410a6c0 0x7fa24419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 --2- 172.21.15.138:0/4163914744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa244104cb0 0x7fa24419b140 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.591+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 >> 172.21.15.138:0/4163914744 conn(0x7fa244100720 msgr2=0x7fa244102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:02.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.592+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 shutdown_connections 2024-08-20T00:59:02.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.592+0000 7fa24a400640 1 -- 172.21.15.138:0/4163914744 wait complete. 2024-08-20T00:59:03.099 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:02 smithi138 ceph-mon[18076]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:03.099 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:02 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.099 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:02 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:02 smithi167 ceph-mon[21965]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:03.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:02 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:02 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:02 smithi142 ceph-mon[21799]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:03.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:02 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:02 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:59:03.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.293+0000 7f9087200640 1 -- 172.21.15.138:0/3481465579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9088104b00 msgr2=0x7f9088104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.293+0000 7f9087200640 1 --2- 172.21.15.138:0/3481465579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9088104b00 0x7f9088104ee0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9078009900 tx=0x7f907802a410 comp rx=0 tx=0).stop 2024-08-20T00:59:03.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 -- 172.21.15.138:0/3481465579 shutdown_connections 2024-08-20T00:59:03.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 --2- 172.21.15.138:0/3481465579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f908810ae90 0x7f9088111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 --2- 172.21.15.138:0/3481465579 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 0x7f908810a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 --2- 172.21.15.138:0/3481465579 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9088104b00 0x7f9088104ee0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 -- 172.21.15.138:0/3481465579 >> 172.21.15.138:0/3481465579 conn(0x7f9088100550 msgr2=0x7f9088102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 -- 172.21.15.138:0/3481465579 shutdown_connections 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.294+0000 7f9087200640 1 -- 172.21.15.138:0/3481465579 wait complete. 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 Processor -- start 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 -- start start 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9088104b00 0x7f908819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 0x7f908819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9088114ba0 con 0x7f908810ae90 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9088114a20 con 0x7f9088104b00 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9087200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9088114d20 con 0x7f908810a510 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9086800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9085e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9088104b00 0x7f908819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9086800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60754/0 (socket says 172.21.15.138:60754) 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.295+0000 7f9086800640 1 -- 172.21.15.138:0/2715271759 learned_addr learned my addr 172.21.15.138:0/2715271759 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f907d400640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 0x7f908819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 -- 172.21.15.138:0/2715271759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 msgr2=0x7f908819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 0x7f908819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 -- 172.21.15.138:0/2715271759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9088104b00 msgr2=0x7f908819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9088104b00 0x7f908819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 -- 172.21.15.138:0/2715271759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90881a0ff0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9086800640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080004b20 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9080004cc0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9080010670 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90881a12e0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.296+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90881a79a0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.297+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9088104ee0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.299+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f9080005020 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.301+0000 7f907f400640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 0x7f906c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.301+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f90800e25d0 con 0x7f908810ae90 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.301+0000 7f9085e00640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 0x7f906c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.302+0000 7f9085e00640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 0x7f906c0b6180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9078005ab0 tx=0x7f9078005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:03.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.304+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f90800ab780 con 0x7f908810ae90 2024-08-20T00:59:03.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.499+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7f90881a7d30 con 0x7f908810ae90 2024-08-20T00:59:03.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.503+0000 7f907f400640 1 -- 172.21.15.138:0/2715271759 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v33)=0 v33) ==== 116+0+0 (secure 0 0 0) 0x7f90800ab080 con 0x7f908810ae90 2024-08-20T00:59:03.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 msgr2=0x7f906c0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 0x7f906c0b6180 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9078005ab0 tx=0x7f9078005a20 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 msgr2=0x7f90881a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f908000eb10 tx=0x7f908000c2d0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 shutdown_connections 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f906c0b3cc0 0x7f906c0b6180 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f908810ae90 0x7f90881a0820 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f908810a510 0x7f908819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 --2- 172.21.15.138:0/2715271759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9088104b00 0x7f908819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 >> 172.21.15.138:0/2715271759 conn(0x7f9088100550 msgr2=0x7f9088109c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:03.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 shutdown_connections 2024-08-20T00:59:03.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:03.506+0000 7f9087200640 1 -- 172.21.15.138:0/2715271759 wait complete. 2024-08-20T00:59:03.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/840096090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 msgr2=0x7fcbb010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- 172.21.15.138:0/840096090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb010c620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009900 tx=0x7fcb9c02a410 comp rx=0 tx=0).stop 2024-08-20T00:59:03.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/840096090 shutdown_connections 2024-08-20T00:59:03.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- 172.21.15.138:0/840096090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb010cc70 0x7fcbb010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- 172.21.15.138:0/840096090 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb010c620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- 172.21.15.138:0/840096090 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb0103810 0x7fcbb0103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/840096090 >> 172.21.15.138:0/840096090 conn(0x7fcbb00fd2d0 msgr2=0x7fcbb00ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/840096090 shutdown_connections 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/840096090 wait complete. 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 Processor -- start 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- start start 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb010cc70 0x7fcbb0101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01128e0 con 0x7fcbb0104130 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0112760 con 0x7fcbb0103810 2024-08-20T00:59:03.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0112a60 con 0x7fcbb010cc70 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37586/0 (socket says 172.21.15.138:37586) 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb0101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60744/0 (socket says 172.21.15.138:60744) 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 -- 172.21.15.138:0/2583818852 learned_addr learned my addr 172.21.15.138:0/2583818852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbae800640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb010cc70 0x7fcbb0101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 -- 172.21.15.138:0/2583818852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb010cc70 msgr2=0x7fcbb0101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb010cc70 0x7fcbb0101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 -- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 msgr2=0x7fcbb0101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb0101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 -- 172.21.15.138:0/2583818852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb01a7190 con 0x7fcbb0103810 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbade00640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcba000e9f0 tx=0x7fcba000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:03.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba000cd80 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba00054d0 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb01a7480 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0010690 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.619+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb01a7940 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0103bf0 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fcba0004e90 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcba6a00640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 0x7fcb940b6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcba00e2490 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcbad400640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 0x7fcb940b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.625+0000 7fcbad400640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 0x7fcb940b6100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c000c00 tx=0x7fcb9c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.630+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fcba00ab640 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.827+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7fcbb01a7cd0 con 0x7fcbb0103810 2024-08-20T00:59:03.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.833+0000 7fcba6a00640 1 -- 172.21.15.138:0/2583818852 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v32)=0 v32) ==== 127+0+0 (secure 0 0 0) 0x7fcba00aaf40 con 0x7fcbb0103810 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.835+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 msgr2=0x7fcb940b6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.835+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 0x7fcb940b6100 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c000c00 tx=0x7fcb9c031400 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 msgr2=0x7fcbb01014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcba000e9f0 tx=0x7fcba000edf0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 shutdown_connections 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcb940b3c40 0x7fcb940b6100 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcbb010cc70 0x7fcbb0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbb0104130 0x7fcbb0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 --2- 172.21.15.138:0/2583818852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcbb0103810 0x7fcbb01014c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 >> 172.21.15.138:0/2583818852 conn(0x7fcbb00fd2d0 msgr2=0x7fcbb00fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 shutdown_connections 2024-08-20T00:59:03.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:02.836+0000 7fcbaf200640 1 -- 172.21.15.138:0/2583818852 wait complete. 2024-08-20T00:59:04.328 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi138') 2024-08-20T00:59:04.328 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi138') 2024-08-20T00:59:04.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2715271759' entity='client.admin' 2024-08-20T00:59:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2715271759' entity='client.admin' 2024-08-20T00:59:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2715271759' entity='client.admin' 2024-08-20T00:59:05.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:05 smithi138 ceph-mon[18076]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 -- 172.21.15.138:0/1887268983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 msgr2=0x7f7a7c1050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:05.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 --2- 172.21.15.138:0/1887268983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c1050a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009ca0 tx=0x7f7a6c02a8c0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 -- 172.21.15.138:0/1887268983 shutdown_connections 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 --2- 172.21.15.138:0/1887268983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 --2- 172.21.15.138:0/1887268983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 --2- 172.21.15.138:0/1887268983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c1050a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 -- 172.21.15.138:0/1887268983 >> 172.21.15.138:0/1887268983 conn(0x7f7a7c1006b0 msgr2=0x7f7a7c102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 -- 172.21.15.138:0/1887268983 shutdown_connections 2024-08-20T00:59:05.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.853+0000 7f7a83600640 1 -- 172.21.15.138:0/1887268983 wait complete. 2024-08-20T00:59:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.854+0000 7f7a83600640 1 Processor -- start 2024-08-20T00:59:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.854+0000 7f7a83600640 1 -- start start 2024-08-20T00:59:05.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114d60 con 0x7f7a7c104cc0 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114be0 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a83600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7c114ee0 con 0x7f7a7c10b050 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a80c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a7be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37608/0 (socket says 172.21.15.138:37608) 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a7be00640 1 -- 172.21.15.138:0/560599937 learned_addr learned my addr 172.21.15.138:0/560599937 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.855+0000 7f7a81600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:05.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 -- 172.21.15.138:0/560599937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 msgr2=0x7f7a7c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 -- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 msgr2=0x7f7a7c19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 -- 172.21.15.138:0/560599937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7c1a1260 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a81600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T00:59:05.856 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a7be00640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800d950 tx=0x7f7a6800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68017030 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7c1a1550 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a68004020 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a680047a0 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.856+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a7c1a7de0 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.857+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a7c1050a0 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.859+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f7a68009a30 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.861+0000 7f7a79600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 0x7f7a600b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:05.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.861+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f7a680e1ba0 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.861+0000 7f7a80c00640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 0x7f7a600b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:05.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.862+0000 7f7a80c00640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 0x7f7a600b6050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c006010 tx=0x7f7a6c009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:05.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:05.864+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7a680aad50 con 0x7f7a7c10a6d0 2024-08-20T00:59:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:05 smithi167 ceph-mon[21965]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:05 smithi142 ceph-mon[21799]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:06.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.051+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f7a7c19c2e0 con 0x7f7a7c10a6d0 2024-08-20T00:59:06.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.057+0000 7f7a79600640 1 -- 172.21.15.138:0/560599937 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v34)=0 v34) ==== 133+0+0 (secure 0 0 0) 0x7f7a680aa650 con 0x7f7a7c10a6d0 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.059+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 msgr2=0x7f7a600b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 0x7f7a600b6050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c006010 tx=0x7f7a6c009820 comp rx=0 tx=0).stop 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 msgr2=0x7f7a7c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800d950 tx=0x7f7a6800de10 comp rx=0 tx=0).stop 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 shutdown_connections 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a600b3b90 0x7f7a600b6050 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a7c10b050 0x7f7a7c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a7c10a6d0 0x7f7a7c19b6f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 --2- 172.21.15.138:0/560599937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a7c104cc0 0x7f7a7c19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 >> 172.21.15.138:0/560599937 conn(0x7f7a7c1006b0 msgr2=0x7f7a7c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:06.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 shutdown_connections 2024-08-20T00:59:06.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.060+0000 7f7a83600640 1 -- 172.21.15.138:0/560599937 wait complete. 2024-08-20T00:59:06.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 -- 172.21.15.138:0/3408265888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c104ca0 msgr2=0x7f3e4c105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 --2- 172.21.15.138:0/3408265888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c105080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3e40009ca0 tx=0x7f3e4002a8c0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 -- 172.21.15.138:0/3408265888 shutdown_connections 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 --2- 172.21.15.138:0/3408265888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 0x7f3e4c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 --2- 172.21.15.138:0/3408265888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 --2- 172.21.15.138:0/3408265888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c105080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 -- 172.21.15.138:0/3408265888 >> 172.21.15.138:0/3408265888 conn(0x7f3e4c100730 msgr2=0x7f3e4c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 -- 172.21.15.138:0/3408265888 shutdown_connections 2024-08-20T00:59:06.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.700+0000 7f3e54000640 1 -- 172.21.15.138:0/3408265888 wait complete. 2024-08-20T00:59:06.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 Processor -- start 2024-08-20T00:59:06.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 -- start start 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 0x7f3e4c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114d00 con 0x7f3e4c10a6b0 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114b80 con 0x7f3e4c10b030 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.701+0000 7f3e54000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e4c114e80 con 0x7f3e4c104ca0 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e50c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39450/0 (socket says 172.21.15.138:39450) 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 -- 172.21.15.138:0/1921888560 learned_addr learned my addr 172.21.15.138:0/1921888560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e52000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 0x7f3e4c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:06.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 -- 172.21.15.138:0/1921888560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 msgr2=0x7f3e4c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 0x7f3e4c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 -- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 msgr2=0x7f3e4c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 -- 172.21.15.138:0/1921888560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4c1a1260 con 0x7f3e4c104ca0 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e50c00640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:59:06.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.702+0000 7f3e51600640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005e30 tx=0x7f3e40031840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.703+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e40003b70 con 0x7f3e4c104ca0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.703+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e40003d10 con 0x7f3e4c104ca0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.703+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e4c1a14f0 con 0x7f3e4c104ca0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.703+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4002fa50 con 0x7f3e4c104ca0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.703+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e4c1a7c10 con 0x7f3e4c104ca0 2024-08-20T00:59:06.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.704+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e4c105080 con 0x7f3e4c104ca0 2024-08-20T00:59:06.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.706+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f3e4002fbf0 con 0x7f3e4c104ca0 2024-08-20T00:59:06.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.707+0000 7f3e4a000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 0x7f3e340b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:06.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.708+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3e401033a0 con 0x7f3e4c104ca0 2024-08-20T00:59:06.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.708+0000 7f3e50c00640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 0x7f3e340b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:06.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.708+0000 7f3e50c00640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 0x7f3e340b60d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c109d80 tx=0x7f3e3800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:06.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.711+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3e400cc550 con 0x7f3e4c104ca0 2024-08-20T00:59:06.900 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.901+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3e4c02e340 con 0x7f3e4c104ca0 2024-08-20T00:59:06.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.907+0000 7f3e4a000640 1 -- 172.21.15.138:0/1921888560 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v35)=0 v35) ==== 121+0+0 (secure 0 0 0) 0x7f3e400cbe50 con 0x7f3e4c104ca0 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 msgr2=0x7f3e340b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 0x7f3e340b60d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c109d80 tx=0x7f3e3800b390 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 msgr2=0x7f3e4c19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e40005e30 tx=0x7f3e40031840 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 shutdown_connections 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e340b3c10 0x7f3e340b60d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4c10b030 0x7f3e4c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4c10a6b0 0x7f3e4c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 --2- 172.21.15.138:0/1921888560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4c104ca0 0x7f3e4c19b1b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 >> 172.21.15.138:0/1921888560 conn(0x7f3e4c100730 msgr2=0x7f3e4c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:06.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.909+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 shutdown_connections 2024-08-20T00:59:06.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:06.910+0000 7f3e54000640 1 -- 172.21.15.138:0/1921888560 wait complete. 2024-08-20T00:59:07.234 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:07.235 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:07 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' 2024-08-20T00:59:07.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:59:07.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:07.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:07 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' 2024-08-20T00:59:07.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:59:07.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:07.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.557 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:07.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:07.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:07.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:07.558 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:07 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:08.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:08.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.272 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.273 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:59:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:59:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.386 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:08.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:08.387 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: Reconfiguring mon.b (monmap changed)... 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: Reconfiguring daemon mon.b on smithi142 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-08-20T00:59:08.393 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:08.765 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.765 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi138') 2024-08-20T00:59:08.765 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi138') 2024-08-20T00:59:09.079 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'd876cd00-5e8e-11ef-bccf-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi138') 2024-08-20T00:59:09.079 INFO:tasks.cephadm:Deploying OSDs... 2024-08-20T00:59:09.079 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T00:59:09.079 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T00:59:09.115 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T00:59:09.115 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-08-20T00:59:09.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:09 smithi167 ceph-mon[21965]: Reconfiguring mon.c (monmap changed)... 2024-08-20T00:59:09.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:09 smithi167 ceph-mon[21965]: Reconfiguring daemon mon.c on smithi167 2024-08-20T00:59:09.119 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:09 smithi167 ceph-mon[21965]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-08-20 00:57:46.960364107 +0000 2024-08-20T00:59:09.192 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-08-20 00:52:28.464824027 +0000 2024-08-20T00:59:09.193 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-08-20 00:52:28.464824027 +0000 2024-08-20T00:59:09.193 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-08-20 00:52:28.464824027 +0000 2024-08-20T00:59:09.193 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T00:59:09.290 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-08-20T00:59:09.290 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-08-20T00:59:09.291 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000295699 s, 1.7 MB/s 2024-08-20T00:59:09.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:09 smithi138 ceph-mon[18076]: Reconfiguring mon.c (monmap changed)... 2024-08-20T00:59:09.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:09 smithi138 ceph-mon[18076]: Reconfiguring daemon mon.c on smithi167 2024-08-20T00:59:09.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:09 smithi138 ceph-mon[18076]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:09.293 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T00:59:09.329 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 652 Links: 1 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:09.405 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-08-20 00:57:47.014362938 +0000 2024-08-20T00:59:09.406 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-08-20 00:52:28.927813716 +0000 2024-08-20T00:59:09.406 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-08-20 00:52:28.927813716 +0000 2024-08-20T00:59:09.406 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-08-20 00:52:28.927813716 +0000 2024-08-20T00:59:09.406 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T00:59:09.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:09 smithi142 ceph-mon[21799]: Reconfiguring mon.c (monmap changed)... 2024-08-20T00:59:09.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:09 smithi142 ceph-mon[21799]: Reconfiguring daemon mon.c on smithi167 2024-08-20T00:59:09.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:09 smithi142 ceph-mon[21799]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:09.497 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-08-20T00:59:09.497 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-08-20T00:59:09.497 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000280608 s, 1.8 MB/s 2024-08-20T00:59:09.500 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T00:59:09.536 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-08-20T00:59:09.611 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-08-20 00:57:47.067361791 +0000 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-08-20 00:52:29.439802314 +0000 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-08-20 00:52:29.439802314 +0000 2024-08-20T00:59:09.612 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-08-20 00:52:29.439802314 +0000 2024-08-20T00:59:09.612 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T00:59:09.704 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-08-20T00:59:09.704 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-08-20T00:59:09.704 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000289167 s, 1.8 MB/s 2024-08-20T00:59:09.706 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T00:59:09.742 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-08-20T00:59:09.817 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T00:59:09.817 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:09.817 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-08-20T00:59:09.817 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:09.817 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:09.818 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-08-20 00:57:47.120360644 +0000 2024-08-20T00:59:09.818 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-08-20 00:52:29.927791446 +0000 2024-08-20T00:59:09.818 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-08-20 00:52:29.927791446 +0000 2024-08-20T00:59:09.818 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-08-20 00:52:29.927791446 +0000 2024-08-20T00:59:09.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T00:59:09.911 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-08-20T00:59:09.911 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-08-20T00:59:09.911 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000256228 s, 2.0 MB/s 2024-08-20T00:59:09.913 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T00:59:09.950 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-08-20T00:59:09.950 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T00:59:09.969 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T00:59:09.969 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-08-20T00:59:10.027 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-20 00:58:45.274965580 +0000 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-20 00:52:38.831504173 +0000 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-20 00:52:38.831504173 +0000 2024-08-20T00:59:10.028 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-20 00:52:38.831504173 +0000 2024-08-20T00:59:10.028 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T00:59:10.098 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-20T00:59:10.098 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-20T00:59:10.098 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000284837 s, 1.8 MB/s 2024-08-20T00:59:10.100 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T00:59:10.159 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-08-20T00:59:10.216 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-20 00:58:45.274965580 +0000 2024-08-20T00:59:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-20 00:52:39.163497699 +0000 2024-08-20T00:59:10.218 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-20 00:52:39.163497699 +0000 2024-08-20T00:59:10.218 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-20 00:52:39.163497699 +0000 2024-08-20T00:59:10.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T00:59:10.282 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-20T00:59:10.282 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-20T00:59:10.282 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000219478 s, 2.3 MB/s 2024-08-20T00:59:10.283 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T00:59:10.340 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.396 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-20 00:58:45.274965580 +0000 2024-08-20T00:59:10.397 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-20 00:52:39.477491576 +0000 2024-08-20T00:59:10.397 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-20 00:52:39.477491576 +0000 2024-08-20T00:59:10.397 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-20 00:52:39.477491576 +0000 2024-08-20T00:59:10.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:10.458 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:10 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.460 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-20T00:59:10.460 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-20T00:59:10.460 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000210755 s, 2.4 MB/s 2024-08-20T00:59:10.462 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:10 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.518 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-08-20T00:59:10.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:10.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:10.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:10.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:10 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:10.574 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T00:59:10.574 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.574 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-08-20 00:58:45.275965561 +0000 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-08-20 00:52:39.802485239 +0000 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-08-20 00:52:39.802485239 +0000 2024-08-20T00:59:10.575 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: 2024-08-20 00:52:39.802485239 +0000 2024-08-20T00:59:10.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T00:59:10.638 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-08-20T00:59:10.638 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-08-20T00:59:10.638 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000216173 s, 2.4 MB/s 2024-08-20T00:59:10.639 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T00:59:10.696 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-08-20T00:59:10.696 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-08-20T00:59:10.716 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-08-20T00:59:10.716 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-20 00:58:43.034239835 +0000 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-20 00:52:34.226009601 +0000 2024-08-20T00:59:10.774 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-20 00:52:34.226009601 +0000 2024-08-20T00:59:10.775 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-20 00:52:34.226009601 +0000 2024-08-20T00:59:10.775 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-08-20T00:59:10.843 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-20T00:59:10.843 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-20T00:59:10.843 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000289381 s, 1.8 MB/s 2024-08-20T00:59:10.844 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-08-20T00:59:10.903 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-08-20T00:59:10.959 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-08-20T00:59:10.959 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:10.959 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-08-20T00:59:10.959 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:10.960 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:10.960 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-20 00:58:43.034239835 +0000 2024-08-20T00:59:10.960 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-20 00:52:34.552003651 +0000 2024-08-20T00:59:10.960 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-20 00:52:34.552003651 +0000 2024-08-20T00:59:10.960 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-20 00:52:34.552003651 +0000 2024-08-20T00:59:10.960 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-08-20T00:59:11.025 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-20T00:59:11.025 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-20T00:59:11.025 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000207148 s, 2.5 MB/s 2024-08-20T00:59:11.027 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-08-20T00:59:11.084 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-08-20T00:59:11.140 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-08-20T00:59:11.141 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:11.141 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-08-20T00:59:11.141 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:11.141 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:11.141 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-20 00:58:43.035239817 +0000 2024-08-20T00:59:11.142 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-20 00:52:34.876997720 +0000 2024-08-20T00:59:11.142 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-20 00:52:34.876997720 +0000 2024-08-20T00:59:11.142 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-20 00:52:34.876997720 +0000 2024-08-20T00:59:11.142 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-08-20T00:59:11.205 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-20T00:59:11.206 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-20T00:59:11.206 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000210027 s, 2.4 MB/s 2024-08-20T00:59:11.207 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-08-20T00:59:11.264 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-08-20T00:59:11.320 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-08-20T00:59:11.320 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-08-20T00:59:11.320 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-08-20T00:59:11.320 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T00:59:11.320 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2024-08-20T00:59:11.321 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-08-20 00:58:43.035239817 +0000 2024-08-20T00:59:11.321 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-08-20 00:52:35.218991479 +0000 2024-08-20T00:59:11.321 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-08-20 00:52:35.218991479 +0000 2024-08-20T00:59:11.321 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: 2024-08-20 00:52:35.218991479 +0000 2024-08-20T00:59:11.321 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-08-20T00:59:11.385 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-08-20T00:59:11.385 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-08-20T00:59:11.385 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000268211 s, 1.9 MB/s 2024-08-20T00:59:11.386 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-08-20T00:59:11.444 INFO:tasks.cephadm:Deploying osd.0 on smithi138 with /dev/vg_nvme/lv_4... 2024-08-20T00:59:11.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T00:59:11.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:11 smithi167 ceph-mon[21965]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:11.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:11 smithi142 ceph-mon[21799]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:11.497 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:11 smithi138 ceph-mon[18076]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:11.810 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:59:13.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:13 smithi167 ceph-mon[21965]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:13.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:13 smithi142 ceph-mon[21799]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:13 smithi138 ceph-mon[18076]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:14.894 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T00:59:14.925 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-08-20T00:59:15.304 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T00:59:15.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:15 smithi138 ceph-mon[18076]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:15 smithi167 ceph-mon[21965]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:15.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:15 smithi142 ceph-mon[21799]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:15.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 -- 172.21.15.138:0/353603619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 msgr2=0x7fd2d010ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:15.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/353603619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d010ab10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009860 tx=0x7fd2bc02a340 comp rx=0 tx=0).stop 2024-08-20T00:59:15.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 -- 172.21.15.138:0/353603619 shutdown_connections 2024-08-20T00:59:15.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/353603619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 0x7fd2d0111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:15.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/353603619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d010ab10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:15.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/353603619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d01050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:15.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 -- 172.21.15.138:0/353603619 >> 172.21.15.138:0/353603619 conn(0x7fd2d01006b0 msgr2=0x7fd2d0102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:15.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 -- 172.21.15.138:0/353603619 shutdown_connections 2024-08-20T00:59:15.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.848+0000 7fd2d5a00640 1 -- 172.21.15.138:0/353603619 wait complete. 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 Processor -- start 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 -- start start 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 0x7fd2d01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0114d10 con 0x7fd2d010b050 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0114b90 con 0x7fd2d010a6d0 2024-08-20T00:59:15.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2d5a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0114e90 con 0x7fd2d0104cc0 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2cea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33168/0 (socket says 172.21.15.138:33168) 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2cea00640 1 -- 172.21.15.138:0/4096908223 learned_addr learned my addr 172.21.15.138:0/4096908223 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.849+0000 7fd2ce000640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cf400640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 0x7fd2d01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 -- 172.21.15.138:0/4096908223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 msgr2=0x7fd2d019b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d019b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 -- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 msgr2=0x7fd2d01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 0x7fd2d01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 -- 172.21.15.138:0/4096908223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d01a1290 con 0x7fd2d0104cc0 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2ce000640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T00:59:15.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2cea00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000eb90 tx=0x7fd2c000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:15.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0004d20 con 0x7fd2d0104cc0 2024-08-20T00:59:15.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d01a1580 con 0x7fd2d0104cc0 2024-08-20T00:59:15.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c000ee90 con 0x7fd2d0104cc0 2024-08-20T00:59:15.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.850+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0010670 con 0x7fd2d0104cc0 2024-08-20T00:59:15.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.851+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d01a7c40 con 0x7fd2d0104cc0 2024-08-20T00:59:15.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.851+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d01050a0 con 0x7fd2d0104cc0 2024-08-20T00:59:15.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.854+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fd2c0010850 con 0x7fd2d0104cc0 2024-08-20T00:59:15.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.855+0000 7fd2c7400640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 0x7fd2b40b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:15.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.855+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd2c00e2a00 con 0x7fd2d0104cc0 2024-08-20T00:59:15.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.855+0000 7fd2ce000640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 0x7fd2b40b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:15.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.856+0000 7fd2ce000640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 0x7fd2b40b6050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009830 tx=0x7fd2bc030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:15.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:15.857+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd2c0055e20 con 0x7fd2d0104cc0 2024-08-20T00:59:16.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:16.053+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fd2d00632c0 con 0x7fd2b40b3b90 2024-08-20T00:59:16.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:16.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:16.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:16.727 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:16.727 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:16.727 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:16.766 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:16.766 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:16.766 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:17.629 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:17 smithi138 ceph-mon[18076]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:17.629 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:17 smithi138 ceph-mon[18076]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:17.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:17 smithi167 ceph-mon[21965]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:17.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:17 smithi167 ceph-mon[21965]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:17.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:17 smithi142 ceph-mon[21799]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:17.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:17 smithi142 ceph-mon[21799]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3599824791' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"}]': finished 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T00:59:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:19 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3599824791' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"}]': finished 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T00:59:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:19 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3599824791' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"} : dispatch 2024-08-20T00:59:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27269bf1-da9b-45c9-930c-b9db2c82afb0"}]': finished 2024-08-20T00:59:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: osdmap e6: 1 total, 0 up, 1 in 2024-08-20T00:59:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:19 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:20.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:20 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3082419516' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:20 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3082419516' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:20.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:20 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3082419516' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:21.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:21 smithi167 ceph-mon[21965]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:21.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:21 smithi142 ceph-mon[21799]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:21 smithi138 ceph-mon[18076]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:23.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:23 smithi138 ceph-mon[18076]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:23.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:23 smithi142 ceph-mon[21799]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:23.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:23 smithi167 ceph-mon[21965]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:25 smithi142 ceph-mon[21799]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:25.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:25 smithi138 ceph-mon[18076]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:25 smithi167 ceph-mon[21965]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:27.558 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:27 smithi138 ceph-mon[18076]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:27.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:27 smithi142 ceph-mon[21799]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:27 smithi167 ceph-mon[21965]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:28.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:28 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T00:59:28.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:28 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:28.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:28 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T00:59:28.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:28 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:28 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-08-20T00:59:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:28 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:29.634 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:29 smithi138 ceph-mon[18076]: Deploying daemon osd.0 on smithi138 2024-08-20T00:59:29.634 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:29 smithi138 ceph-mon[18076]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:29.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:29 smithi142 ceph-mon[21799]: Deploying daemon osd.0 on smithi138 2024-08-20T00:59:29.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:29 smithi142 ceph-mon[21799]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:29 smithi167 ceph-mon[21965]: Deploying daemon osd.0 on smithi138 2024-08-20T00:59:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:29 smithi167 ceph-mon[21965]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:31.658 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:31 smithi138 ceph-mon[18076]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:31.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:31 smithi142 ceph-mon[21799]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:31 smithi167 ceph-mon[21965]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:33 smithi142 ceph-mon[21799]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:33 smithi138 ceph-mon[18076]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:33 smithi167 ceph-mon[21965]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:34.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:34.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:34.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:35.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:35 smithi142 ceph-mon[21799]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:35.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:35 smithi138 ceph-mon[18076]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:35 smithi167 ceph-mon[21965]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:37.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:37.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:37.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:38.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.994+0000 7fd2c7400640 1 -- 172.21.15.138:0/4096908223 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd2d00632c0 con 0x7fd2b40b3b90 2024-08-20T00:59:38.993 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 0 on host 'smithi138' 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 msgr2=0x7fd2b40b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 0x7fd2b40b6050 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc009830 tx=0x7fd2bc030340 comp rx=0 tx=0).stop 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 msgr2=0x7fd2d019b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000eb90 tx=0x7fd2c000c6a0 comp rx=0 tx=0).stop 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 shutdown_connections 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3b90 0x7fd2b40b6050 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2d010b050 0x7fd2d01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2d010a6d0 0x7fd2d019b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:38.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 --2- 172.21.15.138:0/4096908223 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2d0104cc0 0x7fd2d019b1e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.996+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 >> 172.21.15.138:0/4096908223 conn(0x7fd2d01006b0 msgr2=0x7fd2d0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.997+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 shutdown_connections 2024-08-20T00:59:38.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T00:59:38.997+0000 7fd2d5a00640 1 -- 172.21.15.138:0/4096908223 wait complete. 2024-08-20T00:59:39.686 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:39 smithi138 ceph-mon[18076]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:39.686 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:39.686 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.686 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.738 DEBUG:teuthology.orchestra.run.smithi138:osd.0> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.0.service 2024-08-20T00:59:39.742 INFO:tasks.cephadm:Deploying osd.1 on smithi142 with /dev/vg_nvme/lv_4... 2024-08-20T00:59:39.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T00:59:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:39 smithi142 ceph-mon[21799]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.765 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:39 smithi167 ceph-mon[21965]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:39.978 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T00:59:40.290 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 00:59:40 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T00:59:40.031+0000 7ff448022640 -1 osd.0 0 log_to_monitors true 2024-08-20T00:59:40.622 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:40 smithi138 ceph-mon[18076]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:40.622 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:40 smithi138 ceph-mon[18076]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:40 smithi142 ceph-mon[21799]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:40 smithi142 ceph-mon[21799]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:40 smithi167 ceph-mon[21965]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:40 smithi167 ceph-mon[21965]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-08-20T00:59:41.652 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T00:59:41.525+0000 7ff442c00640 -1 osd.0 0 waiting for initial osdmap 2024-08-20T00:59:41.652 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T00:59:41.530+0000 7ff43dc00640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T00:59:41.652 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:41.652 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T00:59:41.653 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:41.653 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:41.653 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T00:59:41.653 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:41 smithi138 ceph-mon[18076]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:41.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T00:59:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T00:59:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:41 smithi167 ceph-mon[21965]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: from='osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: osdmap e7: 1 total, 0 up, 1 in 2024-08-20T00:59:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:41 smithi142 ceph-mon[21799]: from='osd.0 ' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T00:59:42.160 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T00:59:42.186 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-08-20T00:59:42.478 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T00:59:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:42 smithi142 ceph-mon[21799]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T00:59:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:42 smithi142 ceph-mon[21799]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T00:59:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:42 smithi138 ceph-mon[18076]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T00:59:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:42 smithi138 ceph-mon[18076]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T00:59:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:42 smithi167 ceph-mon[21965]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T00:59:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:42 smithi167 ceph-mon[21965]: osdmap e8: 1 total, 0 up, 1 in 2024-08-20T00:59:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.963+0000 7f6a1288c640 1 -- 172.21.15.142:0/1257843316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 msgr2=0x7f6a0c107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.963+0000 7f6a1288c640 1 --2- 172.21.15.142:0/1257843316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c107800 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009900 tx=0x7f69f402a410 comp rx=0 tx=0).stop 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 -- 172.21.15.142:0/1257843316 shutdown_connections 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 --2- 172.21.15.142:0/1257843316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c107d40 0x7f6a0c10e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 --2- 172.21.15.142:0/1257843316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c107800 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 --2- 172.21.15.142:0/1257843316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c1019b0 0x7f6a0c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 -- 172.21.15.142:0/1257843316 >> 172.21.15.142:0/1257843316 conn(0x7f6a0c0fd400 msgr2=0x7f6a0c0ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T00:59:42.978 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 -- 172.21.15.142:0/1257843316 shutdown_connections 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 -- 172.21.15.142:0/1257843316 wait complete. 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 Processor -- start 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.964+0000 7f6a1288c640 1 -- start start 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c1019b0 0x7f6a0c198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c107d40 0x7f6a0c19e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c111a00 con 0x7f6a0c107d40 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c111880 con 0x7f6a0c1073c0 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0c111b80 con 0x7f6a0c1019b0 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59256/0 (socket says 172.21.15.142:59256) 2024-08-20T00:59:42.979 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 -- 172.21.15.142:0/3175214537 learned_addr learned my addr 172.21.15.142:0/3175214537 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0bfff640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c1019b0 0x7f6a0c198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a10e29640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c107d40 0x7f6a0c19e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 -- 172.21.15.142:0/3175214537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c1019b0 msgr2=0x7f6a0c198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c1019b0 0x7f6a0c198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 -- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c107d40 msgr2=0x7f6a0c19e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c107d40 0x7f6a0c19e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 -- 172.21.15.142:0/3175214537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0c19f050 con 0x7f6a0c1073c0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a0b7fe640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005940 tx=0x7f69f40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f403f030 con 0x7f6a0c1073c0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0c19f2e0 con 0x7f6a0c1073c0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69f4003cb0 con 0x7f6a0c1073c0 2024-08-20T00:59:42.980 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4035070 con 0x7f6a0c1073c0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.965+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0c1a5a00 con 0x7f6a0c1073c0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.967+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0c101d90 con 0x7f6a0c1073c0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.968+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f69f4042030 con 0x7f6a0c1073c0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.969+0000 7f6a097fa640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 0x7f69f80b6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.969+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1982+0+0 (secure 0 0 0) 0x7f69f4102d40 con 0x7f6a0c1073c0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.969+0000 7f6a0bfff640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 0x7f69f80b6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.969+0000 7f6a0bfff640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 0x7f69f80b6050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00bbe0 tx=0x7f69fc00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T00:59:42.981 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:42.972+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f69f40cb6e0 con 0x7f6a0c1073c0 2024-08-20T00:59:43.161 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T00:59:43.161+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f6a0c104980 con 0x7f69f80b3b90 2024-08-20T00:59:43.667 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] boot 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:43.668 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:43.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] boot 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T00:59:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T00:59:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: osd.0 [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] boot 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: osdmap e9: 1 total, 1 up, 1 in 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T00:59:43.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T00:59:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:44.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:44.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='client.24329 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T00:59:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: osdmap e10: 1 total, 1 up, 1 in 2024-08-20T00:59:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T00:59:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T00:59:44.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi138 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 17089M 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/448542001' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"}]': finished 2024-08-20T00:59:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T00:59:45.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:45 smithi138 ceph-mon[18076]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T00:59:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi138 2024-08-20T00:59:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 17089M 2024-08-20T00:59:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/448542001' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"}]': finished 2024-08-20T00:59:45.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T00:59:45.865 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:45 smithi167 ceph-mon[21965]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T00:59:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi138 2024-08-20T00:59:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 17089M 2024-08-20T00:59:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: pgmap v33: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/448542001' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"} : dispatch 2024-08-20T00:59:45.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a799e88-8e7f-48d0-a9bf-0c7fef85efd0"}]': finished 2024-08-20T00:59:45.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T00:59:45.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:45 smithi142 ceph-mon[21799]: osdmap e11: 2 total, 1 up, 2 in 2024-08-20T00:59:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:46 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/126751557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:46 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/126751557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:46 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/126751557' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T00:59:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:47 smithi138 ceph-mon[18076]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:47.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:47 smithi167 ceph-mon[21965]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:47 smithi142 ceph-mon[21799]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:48.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:48 smithi167 ceph-mon[21965]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:48 smithi142 ceph-mon[21799]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:49.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:48 smithi138 ceph-mon[18076]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:51.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:51 smithi167 ceph-mon[21965]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:51.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:51 smithi142 ceph-mon[21799]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:51 smithi138 ceph-mon[18076]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:53.639 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:53 smithi142 ceph-mon[21799]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:53.639 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:53 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T00:59:53.640 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:53 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:53 smithi167 ceph-mon[21965]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:53 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T00:59:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:53 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:53 smithi138 ceph-mon[18076]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:53 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-08-20T00:59:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:53 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T00:59:54.561 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:54 smithi142 ceph-mon[21799]: Deploying daemon osd.1 on smithi142 2024-08-20T00:59:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:54 smithi167 ceph-mon[21965]: Deploying daemon osd.1 on smithi142 2024-08-20T00:59:54.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:54 smithi138 ceph-mon[18076]: Deploying daemon osd.1 on smithi142 2024-08-20T00:59:55.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:55 smithi167 ceph-mon[21965]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:55.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:55 smithi142 ceph-mon[21799]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:55.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:55 smithi138 ceph-mon[18076]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:57.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:57 smithi167 ceph-mon[21965]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:57.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:57 smithi142 ceph-mon[21799]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:57 smithi138 ceph-mon[18076]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:58.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:58.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:58 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:58.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:58 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:58.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:58.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:58 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:58.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:58 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:58.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T00:59:58.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:58 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:58.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:58 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T00:59:59.629 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 00:59:59 smithi142 ceph-mon[21799]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 00:59:59 smithi138 ceph-mon[18076]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T00:59:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 00:59:59 smithi167 ceph-mon[21965]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: overall HEALTH_OK 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: overall HEALTH_OK 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: overall HEALTH_OK 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:01.599 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:01 smithi142 ceph-mon[21799]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:01.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:01 smithi138 ceph-mon[18076]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:01.938 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 1 on host 'smithi142' 2024-08-20T01:00:01.938 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.939+0000 7f6a097fa640 1 -- 172.21.15.142:0/3175214537 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6a0c104980 con 0x7f69f80b3b90 2024-08-20T01:00:01.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.941+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 msgr2=0x7f69f80b6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:01.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.941+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 0x7f69f80b6050 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00bbe0 tx=0x7f69fc00b390 comp rx=0 tx=0).stop 2024-08-20T01:00:01.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.941+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 msgr2=0x7f6a0c1994e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:01.940 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.941+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005940 tx=0x7f69f40042f0 comp rx=0 tx=0).stop 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 shutdown_connections 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69f80b3b90 0x7f69f80b6050 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a0c107d40 0x7f6a0c19e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a0c1073c0 0x7f6a0c1994e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 --2- 172.21.15.142:0/3175214537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6a0c1019b0 0x7f6a0c198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 >> 172.21.15.142:0/3175214537 conn(0x7f6a0c0fd400 msgr2=0x7f6a0c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 shutdown_connections 2024-08-20T01:00:01.941 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:00:01.942+0000 7f6a1288c640 1 -- 172.21.15.142:0/3175214537 wait complete. 2024-08-20T01:00:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:01 smithi167 ceph-mon[21965]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:02.519 DEBUG:teuthology.orchestra.run.smithi142:osd.1> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.1.service 2024-08-20T01:00:02.522 INFO:tasks.cephadm:Deploying osd.2 on smithi167 with /dev/vg_nvme/lv_4... 2024-08-20T01:00:02.522 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-08-20T01:00:02.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:02.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:02.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:02.773 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:00:02.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:02.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:02.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:02.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:02.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:02.893 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:03.735 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:03 smithi142 ceph-mon[21799]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:03.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:03 smithi138 ceph-mon[18076]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:03.897 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:03 smithi167 ceph-mon[21965]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:04.474 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:00:04.068+0000 7f2d22a06640 -1 osd.1 0 log_to_monitors true 2024-08-20T01:00:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:04 smithi142 ceph-mon[21799]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:04 smithi142 ceph-mon[21799]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:04 smithi138 ceph-mon[18076]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:04 smithi138 ceph-mon[18076]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:04 smithi167 ceph-mon[21965]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:04 smithi167 ceph-mon[21965]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-08-20T01:00:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T01:00:05.319 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_4 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.552 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:05.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:05.553 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.570 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:00:05.741 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:00:05.492+0000 7f2d1e989640 -1 osd.1 0 waiting for initial osdmap 2024-08-20T01:00:05.741 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:00:05.495+0000 7f2d1a7b0640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:00:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:05.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.745 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:05.745 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: osdmap e12: 2 total, 1 up, 2 in 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:05.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:05.747 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:05.747 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:06.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 -- 172.21.15.167:0/1674395195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01084a0 msgr2=0x7f4bf01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:06.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1674395195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01084a0 0x7f4bf01088e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009900 tx=0x7f4be002a410 comp rx=0 tx=0).stop 2024-08-20T01:00:06.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 -- 172.21.15.167:0/1674395195 shutdown_connections 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1674395195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 0x7f4bf010f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1674395195 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf01084a0 0x7f4bf01088e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1674395195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102a90 0x7f4bf0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 -- 172.21.15.167:0/1674395195 >> 172.21.15.167:0/1674395195 conn(0x7f4bf00fe540 msgr2=0x7f4bf0100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 -- 172.21.15.167:0/1674395195 shutdown_connections 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.062+0000 7f4bf7958640 1 -- 172.21.15.167:0/1674395195 wait complete. 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 Processor -- start 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 -- start start 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf01084a0 0x7f4bf0199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 0x7f4bf019e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112ae0 con 0x7f4bf01084a0 2024-08-20T01:00:06.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112960 con 0x7f4bf0108e20 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf7958640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4bf0112c60 con 0x7f4bf0102a90 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:48716/0 (socket says 172.21.15.167:48716) 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 -- 172.21.15.167:0/1000106019 learned_addr learned my addr 172.21.15.167:0/1000106019 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf4ef3640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf01084a0 0x7f4bf0199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf5ef5640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 0x7f4bf019e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 -- 172.21.15.167:0/1000106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 msgr2=0x7f4bf019e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 0x7f4bf019e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 -- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf01084a0 msgr2=0x7f4bf0199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf01084a0 0x7f4bf0199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.063+0000 7f4bf56f4640 1 -- 172.21.15.167:0/1000106019 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bf019efe0 con 0x7f4bf0102a90 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4bf56f4640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d950 tx=0x7f4bec00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec017070 con 0x7f4bf0102a90 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bec009c30 con 0x7f4bf0102a90 2024-08-20T01:00:06.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bf019f2d0 con 0x7f4bf0102a90 2024-08-20T01:00:06.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00f620 con 0x7f4bf0102a90 2024-08-20T01:00:06.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.064+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bf0075280 con 0x7f4bf0102a90 2024-08-20T01:00:06.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.065+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bf0103d40 con 0x7f4bf0102a90 2024-08-20T01:00:06.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.067+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7f4bec010460 con 0x7f4bf0102a90 2024-08-20T01:00:06.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.067+0000 7f4be67fc640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 0x7f4bdc0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:06.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.067+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f4bec0e1e80 con 0x7f4bf0102a90 2024-08-20T01:00:06.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.067+0000 7f4bf4ef3640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 0x7f4bdc0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:06.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.068+0000 7f4bf4ef3640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 0x7f4bdc0b60d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be0005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:00:06.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.074+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4bec009de0 con 0x7f4bf0102a90 2024-08-20T01:00:06.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:06.265+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f4bf00632c0 con 0x7f4bdc0b3c10 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi142 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 17088M 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:06.669 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:06.670 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:06 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi142 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 17088M 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:06 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi142 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 17088M 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: osdmap e13: 2 total, 1 up, 2 in 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:06.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:06.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:06.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:06 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:07.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:00:07.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:00:07.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:07.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:07.615 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] boot 2024-08-20T01:00:07.616 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:07 smithi167 ceph-mon[21965]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T01:00:07.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] boot 2024-08-20T01:00:07.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:07 smithi138 ceph-mon[18076]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: osd.1 [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] boot 2024-08-20T01:00:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:07 smithi142 ceph-mon[21799]: osdmap e14: 2 total, 2 up, 2 in 2024-08-20T01:00:08.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:08 smithi167 ceph-mon[21965]: osdmap e15: 2 total, 2 up, 2 in 2024-08-20T01:00:08.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:08 smithi138 ceph-mon[18076]: osdmap e15: 2 total, 2 up, 2 in 2024-08-20T01:00:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:08 smithi142 ceph-mon[21799]: osdmap e15: 2 total, 2 up, 2 in 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/2624691471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"}]': finished 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T01:00:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:09 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/2335359750' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/2624691471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"}]': finished 2024-08-20T01:00:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T01:00:09.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/2335359750' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: pgmap v50: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/2624691471' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"} : dispatch 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56932528-00db-4491-b933-3db45acdb6e1"}]': finished 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: osdmap e16: 3 total, 2 up, 3 in 2024-08-20T01:00:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:09 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/2335359750' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:11 smithi138 ceph-mon[18076]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:11 smithi167 ceph-mon[21965]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:11 smithi142 ceph-mon[21799]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:13 smithi138 ceph-mon[18076]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:13.823 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:13 smithi167 ceph-mon[21965]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:13 smithi142 ceph-mon[21799]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:15.789 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:15 smithi167 ceph-mon[21965]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:15 smithi138 ceph-mon[18076]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:15.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:15 smithi142 ceph-mon[21799]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:17.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:17 smithi138 ceph-mon[18076]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:17.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:17 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T01:00:17.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:17 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:17.904 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:17 smithi167 ceph-mon[21965]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:17.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:17 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T01:00:17.905 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:17 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:17 smithi142 ceph-mon[21799]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:17 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-08-20T01:00:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:17 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:18.758 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:18 smithi167 ceph-mon[21965]: Deploying daemon osd.2 on smithi167 2024-08-20T01:00:18.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:18 smithi138 ceph-mon[18076]: Deploying daemon osd.2 on smithi167 2024-08-20T01:00:18.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:18 smithi142 ceph-mon[21799]: Deploying daemon osd.2 on smithi167 2024-08-20T01:00:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:19 smithi167 ceph-mon[21965]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:19 smithi138 ceph-mon[18076]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:19 smithi142 ceph-mon[21799]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:21 smithi138 ceph-mon[18076]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:21.824 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:21 smithi167 ceph-mon[21965]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:21 smithi142 ceph-mon[21799]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:22.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:22 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:22.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:22 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:22.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:22 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:22 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:22 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:22 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:23.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:22 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:23.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:22 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:23.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:22 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:23.852 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:23 smithi167 ceph-mon[21965]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:23 smithi142 ceph-mon[21799]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:23 smithi138 ceph-mon[18076]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:24.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:24 smithi167 ceph-mon[21965]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:24 smithi142 ceph-mon[21799]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:25.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:24 smithi138 ceph-mon[18076]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:26.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:26.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:26.004 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.052+0000 7f4be67fc640 1 -- 172.21.15.167:0/1000106019 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4bf00632c0 con 0x7f4bdc0b3c10 2024-08-20T01:00:26.050 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 2 on host 'smithi167' 2024-08-20T01:00:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 msgr2=0x7f4bdc0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 0x7f4bdc0b60d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4be0031440 tx=0x7f4be0005b50 comp rx=0 tx=0).stop 2024-08-20T01:00:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 msgr2=0x7f4bf0198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d950 tx=0x7f4bec00de10 comp rx=0 tx=0).stop 2024-08-20T01:00:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 shutdown_connections 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4bdc0b3c10 0x7f4bdc0b60d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bf0108e20 0x7f4bf019e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf01084a0 0x7f4bf0199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 --2- 172.21.15.167:0/1000106019 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bf0102a90 0x7f4bf0198f30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 >> 172.21.15.167:0/1000106019 conn(0x7f4bf00fe540 msgr2=0x7f4bf0100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 shutdown_connections 2024-08-20T01:00:26.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:00:26.055+0000 7f4bf7958640 1 -- 172.21.15.167:0/1000106019 wait complete. 2024-08-20T01:00:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:26.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:26.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:26.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.710 DEBUG:teuthology.orchestra.run.smithi167:osd.2> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.2.service 2024-08-20T01:00:26.712 INFO:tasks.cephadm:Deploying osd.3 on smithi138 with /dev/vg_nvme/lv_3... 2024-08-20T01:00:26.712 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T01:00:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:26 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:26 smithi167 ceph-mon[21965]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:26 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:26 smithi138 ceph-mon[18076]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:27.088 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:00:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:26 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:26 smithi142 ceph-mon[21799]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:28.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:28 smithi167 ceph-mon[21965]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:28.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:28 smithi167 ceph-mon[21965]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:28.184 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:00:27.881+0000 7ff39b9f4640 -1 osd.2 0 log_to_monitors true 2024-08-20T01:00:28.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:28 smithi138 ceph-mon[18076]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:28.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:28 smithi138 ceph-mon[18076]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:28.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:28 smithi142 ceph-mon[21799]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:28.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:28 smithi142 ceph-mon[21799]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-08-20T01:00:29.213 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:00:29.060+0000 7ff397977640 -1 osd.2 0 waiting for initial osdmap 2024-08-20T01:00:29.213 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:00:29.063+0000 7ff39379e640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:00:29.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:00:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:29 smithi167 ceph-mon[21965]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:00:29.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:29 smithi138 ceph-mon[18076]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: osdmap e17: 3 total, 2 up, 3 in 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: from='osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:29.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:00:29.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:29 smithi142 ceph-mon[21799]: osdmap e18: 3 total, 2 up, 3 in 2024-08-20T01:00:30.167 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.167 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.168 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:30.169 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:30.169 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:30 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:30.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:30 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:30.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:30.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:30.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:30 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:30.504 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:00:30.538 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-08-20T01:00:30.916 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:00:31.376 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:00:31.376 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:00:31.376 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi167 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] boot 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.377 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:31 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi167 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] boot 2024-08-20T01:00:31.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T01:00:31.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:31.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:31 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi167 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 17088M 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: osd.2 [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] boot 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: osdmap e19: 3 total, 3 up, 3 in 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:31 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 -- 172.21.15.138:0/2009751399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58104c90 msgr2=0x7fbf58105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 --2- 172.21.15.138:0/2009751399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58104c90 0x7fbf58105070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009ca0 tx=0x7fbf4802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 -- 172.21.15.138:0/2009751399 shutdown_connections 2024-08-20T01:00:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 --2- 172.21.15.138:0/2009751399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810b020 0x7fbf58111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 --2- 172.21.15.138:0/2009751399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf5810a6a0 0x7fbf5810aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 --2- 172.21.15.138:0/2009751399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf58104c90 0x7fbf58105070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 -- 172.21.15.138:0/2009751399 >> 172.21.15.138:0/2009751399 conn(0x7fbf581006c0 msgr2=0x7fbf58102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 -- 172.21.15.138:0/2009751399 shutdown_connections 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.489+0000 7fbf5f600640 1 -- 172.21.15.138:0/2009751399 wait complete. 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.490+0000 7fbf5f600640 1 Processor -- start 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.490+0000 7fbf5f600640 1 -- start start 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.490+0000 7fbf5f600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58104c90 0x7fbf5819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5f600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810a6a0 0x7fbf5819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5f600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:31.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5f600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58114ce0 con 0x7fbf5810b020 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5f600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58114b60 con 0x7fbf58104c90 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5f600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58114e60 con 0x7fbf5810a6a0 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56296/0 (socket says 172.21.15.138:56296) 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 -- 172.21.15.138:0/1851352386 learned_addr learned my addr 172.21.15.138:0/1851352386 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf57e00640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810a6a0 0x7fbf5819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5cc00640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58104c90 0x7fbf5819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 -- 172.21.15.138:0/1851352386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810a6a0 msgr2=0x7fbf5819b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810a6a0 0x7fbf5819b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 -- 172.21.15.138:0/1851352386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58104c90 msgr2=0x7fbf5819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58104c90 0x7fbf5819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:31.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.491+0000 7fbf5d600640 1 -- 172.21.15.138:0/1851352386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581a1240 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf5d600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e970 tx=0x7fbf5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000ccc0 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5000ce60 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50010430 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581a1530 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.492+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581a7bf0 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.493+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf58105070 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.495+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) ==== 151021+0+0 (secure 0 0 0) 0x7fbf50004fb0 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.497+0000 7fbf55600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 0x7fbf3c0b60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.497+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3441+0+0 (secure 0 0 0) 0x7fbf500e2e50 con 0x7fbf5810b020 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.497+0000 7fbf5cc00640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 0x7fbf3c0b60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:00:31.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.498+0000 7fbf5cc00640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 0x7fbf3c0b60d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009430 tx=0x7fbf48009360 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:00:31.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.499+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbf50055e20 con 0x7fbf5810b020 2024-08-20T01:00:31.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:31.692+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbf580632c0 con 0x7fbf3c0b3c10 2024-08-20T01:00:32.446 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:32.446 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T01:00:32.447 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.447 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.447 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:32.447 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:32.447 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:32 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:32.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:32 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: osdmap e20: 3 total, 3 up, 3 in 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:00:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:32 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:33.355 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 ceph-mon[18076]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:33.355 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:33.356 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 ceph-mon[18076]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T01:00:33.356 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 ceph-mon[18076]: pgmap v68: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41260]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41260]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41260]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41260]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41263]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41263]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41263]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.356 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41263]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 ceph-mon[21965]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:33.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:33.419 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 ceph-mon[21965]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T01:00:33.420 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 ceph-mon[21965]: pgmap v68: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32028]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32028]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32028]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32028]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32031]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32031]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32031]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.420 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32031]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 ceph-mon[21799]: from='client.14457 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:00:33.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:00:33.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 ceph-mon[21799]: osdmap e21: 3 total, 3 up, 3 in 2024-08-20T01:00:33.460 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 ceph-mon[21799]: pgmap v68: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:33.460 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31272]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.460 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31272]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.460 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31272]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.460 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31272]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.461 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31275]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.461 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31275]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.461 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31275]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.461 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31275]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.707 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41266]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41266]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41266]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41266]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41270]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:33 smithi138 sudo[41270]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32034]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32034]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32034]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32034]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32037]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32037]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32037]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:33 smithi167 sudo[32037]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31278]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31278]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31278]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31278]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31281]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31281]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31281]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-08-20T01:00:33.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:33 smithi142 sudo[31281]: pam_unix(sudo:session): session closed for user root 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: osdmap e22: 3 total, 3 up, 3 in 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: osdmap e22: 3 total, 3 up, 3 in 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: osdmap e22: 3 total, 3 up, 3 in 2024-08-20T01:00:34.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:34.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-08-20T01:00:34.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-08-20T01:00:34.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-08-20T01:00:34.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-08-20T01:00:35.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:35.112+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fbf500052a0 con 0x7fbf5810b020 2024-08-20T01:00:35.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:35 smithi138 ceph-mon[18076]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:35.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:35 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"} : dispatch 2024-08-20T01:00:35.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:35 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"}]': finished 2024-08-20T01:00:35.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:35.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:35 smithi138 ceph-mon[18076]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T01:00:35.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:35 smithi167 ceph-mon[21965]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:35.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:35 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"} : dispatch 2024-08-20T01:00:35.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:35 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"}]': finished 2024-08-20T01:00:35.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:35.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:35 smithi167 ceph-mon[21965]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T01:00:35.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:35 smithi142 ceph-mon[21799]: pgmap v70: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:35.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:35 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"} : dispatch 2024-08-20T01:00:35.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:35 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/103540099' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73bed35d-9793-4b09-ba0b-41150f576f8a"}]': finished 2024-08-20T01:00:35.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:35.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:35 smithi142 ceph-mon[21799]: osdmap e23: 4 total, 3 up, 4 in 2024-08-20T01:00:36.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:36 smithi167 ceph-mon[21965]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-20T01:00:36.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:36 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/479274429' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:36.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:36 smithi142 ceph-mon[21799]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-20T01:00:36.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:36 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/479274429' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:36.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:36 smithi138 ceph-mon[18076]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-08-20T01:00:36.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:36 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/479274429' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:00:37.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:37 smithi167 ceph-mon[21965]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:37 smithi142 ceph-mon[21799]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:37.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:37 smithi138 ceph-mon[18076]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:39.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:39 smithi138 ceph-mon[18076]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:39 smithi167 ceph-mon[21965]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:39 smithi142 ceph-mon[21799]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:41.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:41 smithi167 ceph-mon[21965]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:41 smithi142 ceph-mon[21799]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:41.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:41 smithi138 ceph-mon[18076]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:43 smithi142 ceph-mon[21799]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:43.748 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:43 smithi138 ceph-mon[18076]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:43 smithi167 ceph-mon[21965]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T01:00:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:44.771 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T01:00:44.771 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-08-20T01:00:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:45.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:45 smithi142 ceph-mon[21799]: Deploying daemon osd.3 on smithi138 2024-08-20T01:00:45.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:45 smithi142 ceph-mon[21799]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:45.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:45 smithi138 ceph-mon[18076]: Deploying daemon osd.3 on smithi138 2024-08-20T01:00:45.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:45 smithi138 ceph-mon[18076]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:45 smithi167 ceph-mon[21965]: Deploying daemon osd.3 on smithi138 2024-08-20T01:00:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:45 smithi167 ceph-mon[21965]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:47.684 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:47 smithi138 ceph-mon[18076]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:47.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:47 smithi142 ceph-mon[21799]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:47.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:47 smithi167 ceph-mon[21965]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:49.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:49 smithi142 ceph-mon[21799]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:49.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:49 smithi138 ceph-mon[18076]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:49 smithi167 ceph-mon[21965]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:51.740 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:51 smithi138 ceph-mon[18076]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:51.740 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:51.740 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:51.740 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:51.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:51 smithi142 ceph-mon[21799]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:51.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:51.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:51.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:51 smithi167 ceph-mon[21965]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:53 smithi142 ceph-mon[21799]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:53.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:53 smithi138 ceph-mon[18076]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:53 smithi167 ceph-mon[21965]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:54 smithi167 ceph-mon[21965]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:54 smithi142 ceph-mon[21799]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:55.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:55.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:55.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:00:55.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:00:55.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:55.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:54 smithi138 ceph-mon[18076]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:55.816 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 3 on host 'smithi138' 2024-08-20T01:00:55.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.818+0000 7fbf55600640 1 -- 172.21.15.138:0/1851352386 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf580632c0 con 0x7fbf3c0b3c10 2024-08-20T01:00:55.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.820+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 msgr2=0x7fbf3c0b60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:55.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.820+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 0x7fbf3c0b60d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009430 tx=0x7fbf48009360 comp rx=0 tx=0).stop 2024-08-20T01:00:55.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.820+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 msgr2=0x7fbf581a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.820+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000e970 tx=0x7fbf5000ed70 comp rx=0 tx=0).stop 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 shutdown_connections 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf3c0b3c10 0x7fbf3c0b60d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf5810b020 0x7fbf581a0a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf5810a6a0 0x7fbf5819b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:55.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 --2- 172.21.15.138:0/1851352386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf58104c90 0x7fbf5819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:00:55.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 >> 172.21.15.138:0/1851352386 conn(0x7fbf581006c0 msgr2=0x7fbf58101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:00:55.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 shutdown_connections 2024-08-20T01:00:55.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:00:55.821+0000 7fbf5f600640 1 -- 172.21.15.138:0/1851352386 wait complete. 2024-08-20T01:00:56.526 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:56.526 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.526 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.562 DEBUG:teuthology.orchestra.run.smithi138:osd.3> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.3.service 2024-08-20T01:00:56.566 INFO:tasks.cephadm:Deploying osd.4 on smithi142 with /dev/vg_nvme/lv_3... 2024-08-20T01:00:56.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T01:00:56.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:56.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:00:56.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:00:56.821 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:00:57.400 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:00:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:00:57.286+0000 7f123861b640 -1 osd.3 0 log_to_monitors true 2024-08-20T01:00:57.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:57 smithi142 ceph-mon[21799]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:57.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:57 smithi142 ceph-mon[21799]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T01:00:57.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:57 smithi167 ceph-mon[21965]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:57.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:57 smithi167 ceph-mon[21965]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T01:00:57.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:57 smithi138 ceph-mon[18076]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:57.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:57 smithi138 ceph-mon[18076]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-08-20T01:00:58.539 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:00:58.428+0000 7f1233200640 -1 osd.3 0 waiting for initial osdmap 2024-08-20T01:00:58.540 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:00:58.436+0000 7f122e200640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:00:58.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-mon[18076]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T01:00:58.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:58.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-mon[18076]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T01:00:58.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:58 smithi138 ceph-mon[18076]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:00:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:58 smithi142 ceph-mon[21799]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T01:00:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:58 smithi142 ceph-mon[21799]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T01:00:58.644 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:58 smithi142 ceph-mon[21799]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:00:58.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:58 smithi167 ceph-mon[21965]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-08-20T01:00:58.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:58.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:58 smithi167 ceph-mon[21965]: osdmap e24: 4 total, 3 up, 4 in 2024-08-20T01:00:58.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:58 smithi167 ceph-mon[21965]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:00:59.352 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T01:00:59.379 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-08-20T01:00:59.628 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:00:59.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:00:59.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] boot 2024-08-20T01:00:59.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:00:59 smithi142 ceph-mon[21799]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T01:00:59.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:00:59.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T01:00:59.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:59.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] boot 2024-08-20T01:00:59.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:00:59 smithi167 ceph-mon[21965]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: from='osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: osdmap e25: 4 total, 3 up, 4 in 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: osd.3 [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] boot 2024-08-20T01:00:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:00:59 smithi138 ceph-mon[18076]: osdmap e26: 4 total, 4 up, 4 in 2024-08-20T01:01:00.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 -- 172.21.15.142:0/1354835294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 msgr2=0x7ff8ec101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:00.048 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 --2- 172.21.15.142:0/1354835294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec101d80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009900 tx=0x7ff8e802a410 comp rx=0 tx=0).stop 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 -- 172.21.15.142:0/1354835294 shutdown_connections 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 --2- 172.21.15.142:0/1354835294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 0x7ff8ec10e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 --2- 172.21.15.142:0/1354835294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 0x7ff8ec1077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 --2- 172.21.15.142:0/1354835294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec101d80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 -- 172.21.15.142:0/1354835294 >> 172.21.15.142:0/1354835294 conn(0x7ff8ec0fd430 msgr2=0x7ff8ec0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 -- 172.21.15.142:0/1354835294 shutdown_connections 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.048+0000 7ff8f4039640 1 -- 172.21.15.142:0/1354835294 wait complete. 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 Processor -- start 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 -- start start 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 0x7ff8ec199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 0x7ff8ec19e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111a30 con 0x7ff8ec107d30 2024-08-20T01:01:00.049 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec1118b0 con 0x7ff8ec1019a0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f4039640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8ec111bb0 con 0x7ff8ec1073b0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:54418/0 (socket says 172.21.15.142:54418) 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 -- 172.21.15.142:0/618516840 learned_addr learned my addr 172.21.15.142:0/618516840 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f15d4640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 0x7ff8ec199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f25d6640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 0x7ff8ec19e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 -- 172.21.15.142:0/618516840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 msgr2=0x7ff8ec199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 0x7ff8ec199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 -- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 msgr2=0x7ff8ec19e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 0x7ff8ec19e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.049+0000 7ff8f1dd5640 1 -- 172.21.15.142:0/618516840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8ec19f1c0 con 0x7ff8ec1019a0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8f1dd5640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006010 tx=0x7ff8e80042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:00.050 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e803f030 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8e8003800 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8035070 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8ec19f450 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.050+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8ec1a5b70 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.051+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8ec101e00 con 0x7ff8ec1019a0 2024-08-20T01:01:00.051 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.053+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7ff8e8003cb0 con 0x7ff8ec1019a0 2024-08-20T01:01:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.054+0000 7ff8e2ffd640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 0x7ff8d80b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.054+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3917+0+0 (secure 0 0 0) 0x7ff8e8103400 con 0x7ff8ec1019a0 2024-08-20T01:01:00.052 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.054+0000 7ff8f15d4640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 0x7ff8d80b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:00.053 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.055+0000 7ff8f15d4640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 0x7ff8d80b6160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec100fb0 tx=0x7ff8dc009110 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:00.057 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.058+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff8e80cbb80 con 0x7ff8ec1019a0 2024-08-20T01:01:00.249 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:00.251+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff8ec104970 con 0x7ff8d80b3ca0 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi138 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 8544M 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:01 smithi142 ceph-mon[21799]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:01.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:01:01.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:01:01.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T01:01:01.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi138 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 8544M 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:01.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:01 smithi167 ceph-mon[21965]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:01.493 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:01:01.493 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:01:01.493 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: osdmap e27: 4 total, 4 up, 4 in 2024-08-20T01:01:01.493 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:01.493 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi138 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 8544M 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.494 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:01.495 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:01.495 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:01.495 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:01 smithi138 ceph-mon[18076]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:02.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:02 smithi167 ceph-mon[21965]: osdmap e28: 4 total, 4 up, 4 in 2024-08-20T01:01:02.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:02 smithi142 ceph-mon[21799]: osdmap e28: 4 total, 4 up, 4 in 2024-08-20T01:01:02.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:02 smithi138 ceph-mon[18076]: osdmap e28: 4 total, 4 up, 4 in 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/2159522825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"}]': finished 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T01:01:03.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/2159522825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"}]': finished 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T01:01:03.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/2159522825' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"} : dispatch 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "495167fb-1b60-4d94-ba5b-ff307d382779"}]': finished 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: osdmap e29: 5 total, 4 up, 5 in 2024-08-20T01:01:03.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:04.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/3408693870' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:04.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/3408693870' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:04.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/3408693870' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:05.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:05 smithi167 ceph-mon[21965]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:05.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:05 smithi142 ceph-mon[21799]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:05.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:05 smithi138 ceph-mon[18076]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:07.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:07 smithi142 ceph-mon[21799]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:07.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:07 smithi138 ceph-mon[18076]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:07 smithi167 ceph-mon[21965]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:09.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:09 smithi142 ceph-mon[21799]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:01:09.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:09 smithi138 ceph-mon[18076]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:01:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:09 smithi167 ceph-mon[21965]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:01:11.682 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:11 smithi142 ceph-mon[21799]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:01:11.682 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T01:01:11.683 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:11.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:11 smithi138 ceph-mon[18076]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:01:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T01:01:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:11 smithi167 ceph-mon[21965]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:01:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-08-20T01:01:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:12.688 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:12 smithi142 ceph-mon[21799]: Deploying daemon osd.4 on smithi142 2024-08-20T01:01:12.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:12 smithi138 ceph-mon[18076]: Deploying daemon osd.4 on smithi142 2024-08-20T01:01:12.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:12 smithi167 ceph-mon[21965]: Deploying daemon osd.4 on smithi142 2024-08-20T01:01:13.635 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:13 smithi142 ceph-mon[21799]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:01:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:13 smithi138 ceph-mon[18076]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:01:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:13 smithi167 ceph-mon[21965]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:01:15.685 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:15 smithi142 ceph-mon[21799]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:15.765 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:15 smithi167 ceph-mon[21965]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:15 smithi138 ceph-mon[18076]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:16.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:16.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:16.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:16.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:16.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:16.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:17.687 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:17 smithi142 ceph-mon[21799]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:17.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:17 smithi138 ceph-mon[18076]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:17 smithi167 ceph-mon[21965]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:19.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:19.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.139 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 4 on host 'smithi142' 2024-08-20T01:01:20.140 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.141+0000 7ff8e2ffd640 1 -- 172.21.15.142:0/618516840 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff8ec104970 con 0x7ff8d80b3ca0 2024-08-20T01:01:20.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 msgr2=0x7ff8d80b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:20.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 0x7ff8d80b6160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec100fb0 tx=0x7ff8dc009110 comp rx=0 tx=0).stop 2024-08-20T01:01:20.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 msgr2=0x7ff8ec199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:20.142 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8006010 tx=0x7ff8e80042f0 comp rx=0 tx=0).stop 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 shutdown_connections 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8d80b3ca0 0x7ff8d80b6160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8ec107d30 0x7ff8ec19e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8ec1073b0 0x7ff8ec199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 --2- 172.21.15.142:0/618516840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8ec1019a0 0x7ff8ec199110 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 >> 172.21.15.142:0/618516840 conn(0x7ff8ec0fd430 msgr2=0x7ff8ec06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 shutdown_connections 2024-08-20T01:01:20.143 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:01:20.144+0000 7ff8f4039640 1 -- 172.21.15.142:0/618516840 wait complete. 2024-08-20T01:01:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:20 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:20 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:20 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.786 DEBUG:teuthology.orchestra.run.smithi142:osd.4> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.4.service 2024-08-20T01:01:20.789 INFO:tasks.cephadm:Deploying osd.5 on smithi167 with /dev/vg_nvme/lv_3... 2024-08-20T01:01:20.789 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-08-20T01:01:20.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:20 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:20.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:20 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:20 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.817 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:20 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:20 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:20.818 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:20 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:21.045 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:01:21.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:21 smithi167 ceph-mon[21965]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:21 smithi138 ceph-mon[18076]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:21.907 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:21 smithi142 ceph-mon[21799]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:22.492 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:01:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:01:22.368+0000 7ff1b6320640 -1 osd.4 0 log_to_monitors true 2024-08-20T01:01:22.763 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:22 smithi167 ceph-mon[21965]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:22.764 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:22 smithi167 ceph-mon[21965]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:22 smithi138 ceph-mon[18076]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:22.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:22 smithi138 ceph-mon[18076]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:22.917 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:22 smithi142 ceph-mon[21799]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:22.917 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:22 smithi142 ceph-mon[21799]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-08-20T01:01:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T01:01:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T01:01:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:23 smithi138 ceph-mon[18076]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.826 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:23 smithi167 ceph-mon[21965]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.828 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T01:01:23.852 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_3 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: osdmap e30: 5 total, 4 up, 5 in 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: from='osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-mon[21799]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:01:23.991 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:01:23.512+0000 7ff1b22a3640 -1 osd.4 0 waiting for initial osdmap 2024-08-20T01:01:23.992 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:01:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:01:23.518+0000 7ff1ae0ca640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:01:24.132 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:01:24.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 -- 172.21.15.167:0/2470966706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 msgr2=0x7fd8c41018e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:24.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/2470966706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c41018e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4009900 tx=0x7fd8b402a410 comp rx=0 tx=0).stop 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 -- 172.21.15.167:0/2470966706 shutdown_connections 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/2470966706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c410cf40 0x7fd8c410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/2470966706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8c410a610 0x7fd8c410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/2470966706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c41018e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 -- 172.21.15.167:0/2470966706 >> 172.21.15.167:0/2470966706 conn(0x7fd8c4076cc0 msgr2=0x7fd8c40770d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 -- 172.21.15.167:0/2470966706 shutdown_connections 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.599+0000 7fd8c9b79640 1 -- 172.21.15.167:0/2470966706 wait complete. 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 Processor -- start 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 -- start start 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c410a610 0x7fd8c4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8c410cf40 0x7fd8c419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4112aa0 con 0x7fd8c410a610 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4112920 con 0x7fd8c410cf40 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c9b79640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c4112c20 con 0x7fd8c4101500 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c37fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:58330/0 (socket says 172.21.15.167:58330) 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.600+0000 7fd8c37fe640 1 -- 172.21.15.167:0/3621194551 learned_addr learned my addr 172.21.15.167:0/3621194551 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 -- 172.21.15.167:0/3621194551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8c410cf40 msgr2=0x7fd8c419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8c410cf40 0x7fd8c419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 -- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c410a610 msgr2=0x7fd8c4199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c410a610 0x7fd8c4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 -- 172.21.15.167:0/3621194551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c419ef80 con 0x7fd8c4101500 2024-08-20T01:01:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c37fe640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4006010 tx=0x7fd8b40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b403f030 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b4003800 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4035070 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c419f210 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.601+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c41a5930 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.602+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c41018e0 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.603+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fd8b4003cb0 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.604+0000 7fd8c0ff9640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 0x7fd8ac0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.604+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4316+0+0 (secure 0 0 0) 0x7fd8b4103900 con 0x7fd8c4101500 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.605+0000 7fd8c2ffd640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 0x7fd8ac0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.605+0000 7fd8c2ffd640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 0x7fd8ac0b60e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8c410c070 tx=0x7fd8b8005c70 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.607+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd8b40cbef0 con 0x7fd8c4101500 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: osdmap e31: 5 total, 4 up, 5 in 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi142 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:24.792+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd8c41009a0 con 0x7fd8ac0b3c20 2024-08-20T01:01:24.792 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: osdmap e31: 5 total, 4 up, 5 in 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi142 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:24.793 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: osdmap e31: 5 total, 4 up, 5 in 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi142 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 8544M 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] boot 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 8544M 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] boot 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:01:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:01:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 8544M 2024-08-20T01:01:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-08-20T01:01:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-08-20T01:01:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: osd.4 [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] boot 2024-08-20T01:01:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: osdmap e32: 5 total, 5 up, 5 in 2024-08-20T01:01:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:26.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:26 smithi167 ceph-mon[21965]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:26.790 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:26 smithi167 ceph-mon[21965]: osdmap e33: 5 total, 5 up, 5 in 2024-08-20T01:01:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:26 smithi138 ceph-mon[18076]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:26 smithi138 ceph-mon[18076]: osdmap e33: 5 total, 5 up, 5 in 2024-08-20T01:01:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:26 smithi142 ceph-mon[21799]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:26 smithi142 ceph-mon[21799]: osdmap e33: 5 total, 5 up, 5 in 2024-08-20T01:01:27.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/4073593845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"}]': finished 2024-08-20T01:01:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T01:01:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:27 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:27.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:27.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/4073593845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"}]': finished 2024-08-20T01:01:27.863 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T01:01:27.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:27 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/4073593845' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"} : dispatch 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "520885a9-d0be-4c97-9744-fd7f32d98544"}]': finished 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: osdmap e34: 6 total, 5 up, 6 in 2024-08-20T01:01:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:27 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:28.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:28 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/1235072981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:28 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/1235072981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:28 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/1235072981' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:29 smithi138 ceph-mon[18076]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:29 smithi167 ceph-mon[21965]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:29 smithi142 ceph-mon[21799]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:31 smithi138 ceph-mon[18076]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:31 smithi167 ceph-mon[21965]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:31 smithi142 ceph-mon[21799]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:33 smithi138 ceph-mon[18076]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:33.815 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:33 smithi167 ceph-mon[21965]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:33 smithi142 ceph-mon[21799]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:35.788 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:35 smithi167 ceph-mon[21965]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:35.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:35 smithi138 ceph-mon[18076]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:35 smithi142 ceph-mon[21799]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:36.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:36 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T01:01:36.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:36 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:36 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T01:01:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:36 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:37.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:36 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-08-20T01:01:37.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:36 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:37.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:37 smithi167 ceph-mon[21965]: Deploying daemon osd.5 on smithi167 2024-08-20T01:01:37.870 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:37 smithi167 ceph-mon[21965]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:37 smithi142 ceph-mon[21799]: Deploying daemon osd.5 on smithi167 2024-08-20T01:01:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:37 smithi142 ceph-mon[21799]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:37 smithi138 ceph-mon[18076]: Deploying daemon osd.5 on smithi167 2024-08-20T01:01:38.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:37 smithi138 ceph-mon[18076]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:39 smithi167 ceph-mon[21965]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:39 smithi142 ceph-mon[21799]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:40.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:39 smithi138 ceph-mon[18076]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:41.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:41 smithi167 ceph-mon[21965]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:41.836 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:41 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:41.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:41.837 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:41 smithi142 ceph-mon[21799]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:41 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:41 smithi138 ceph-mon[18076]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:41 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:43 smithi167 ceph-mon[21965]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:43 smithi142 ceph-mon[21799]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:43 smithi138 ceph-mon[18076]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:44.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.883+0000 7fd8c0ff9640 1 -- 172.21.15.167:0/3621194551 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd8c41009a0 con 0x7fd8ac0b3c20 2024-08-20T01:01:44.882 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 5 on host 'smithi167' 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 msgr2=0x7fd8ac0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 0x7fd8ac0b60e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd8c410c070 tx=0x7fd8b8005c70 comp rx=0 tx=0).stop 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 msgr2=0x7fd8c4198ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4006010 tx=0x7fd8b40042f0 comp rx=0 tx=0).stop 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 shutdown_connections 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8ac0b3c20 0x7fd8ac0b60e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:44.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8c410cf40 0x7fd8c419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:44.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8c410a610 0x7fd8c4199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:44.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 --2- 172.21.15.167:0/3621194551 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd8c4101500 0x7fd8c4198ed0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:44.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 >> 172.21.15.167:0/3621194551 conn(0x7fd8c4076cc0 msgr2=0x7fd8c4100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:01:44.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 shutdown_connections 2024-08-20T01:01:44.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:01:44.886+0000 7fd8c9b79640 1 -- 172.21.15.167:0/3621194551 wait complete. 2024-08-20T01:01:45.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:45.182 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:45.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:45.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:45.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.183 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:45.510 DEBUG:teuthology.orchestra.run.smithi167:osd.5> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.5.service 2024-08-20T01:01:45.513 INFO:tasks.cephadm:Deploying osd.6 on smithi138 with /dev/vg_nvme/lv_2... 2024-08-20T01:01:45.513 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T01:01:45.886 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:01:47.574 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:47 smithi138 ceph-mon[18076]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:47.574 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:47 smithi138 ceph-mon[18076]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:47.574 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:47 smithi138 ceph-mon[18076]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:47.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:47 smithi167 ceph-mon[21965]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:47.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:47 smithi167 ceph-mon[21965]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:47.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:47 smithi167 ceph-mon[21965]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:47.714 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:01:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:01:47.359+0000 7f1abedbc640 -1 osd.5 0 log_to_monitors true 2024-08-20T01:01:47.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:47 smithi142 ceph-mon[21799]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:47.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:47 smithi142 ceph-mon[21799]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:47.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:47 smithi142 ceph-mon[21799]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-08-20T01:01:48.571 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T01:01:48.571 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T01:01:48.571 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:48.571 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.571 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:48.572 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.713 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:01:48.498+0000 7f1abad3f640 -1 osd.5 0 waiting for initial osdmap 2024-08-20T01:01:48.713 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:01:48.504+0000 7f1ab6b66640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:01:48.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T01:01:48.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T01:01:48.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:48.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:48.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-08-20T01:01:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: osdmap e35: 6 total, 5 up, 6 in 2024-08-20T01:01:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:01:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:01:49.289 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:01:49.324 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-08-20T01:01:49.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi167 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: osdmap e36: 6 total, 5 up, 6 in 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] boot 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T01:01:49.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:49 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:49.698 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi167 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: osdmap e36: 6 total, 5 up, 6 in 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] boot 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T01:01:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:49 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 134 MiB used, 447 GiB / 447 GiB avail 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi167 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 8544M 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: osdmap e36: 6 total, 5 up, 6 in 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: osd.5 [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] boot 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: osdmap e37: 6 total, 6 up, 6 in 2024-08-20T01:01:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:49 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-08-20T01:01:50.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.251+0000 7f426ec00640 1 -- 172.21.15.138:0/423759176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4268104cb0 msgr2=0x7f4268105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.251+0000 7f426ec00640 1 --2- 172.21.15.138:0/423759176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4268104cb0 0x7f4268105090 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4258009900 tx=0x7f425802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 -- 172.21.15.138:0/423759176 shutdown_connections 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 --2- 172.21.15.138:0/423759176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f426810b040 0x7f4268111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 --2- 172.21.15.138:0/423759176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810a6c0 0x7f426810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 --2- 172.21.15.138:0/423759176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4268104cb0 0x7f4268105090 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 -- 172.21.15.138:0/423759176 >> 172.21.15.138:0/423759176 conn(0x7f4268100720 msgr2=0x7f4268102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 -- 172.21.15.138:0/423759176 shutdown_connections 2024-08-20T01:01:50.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.252+0000 7f426ec00640 1 -- 172.21.15.138:0/423759176 wait complete. 2024-08-20T01:01:50.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 Processor -- start 2024-08-20T01:01:50.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 -- start start 2024-08-20T01:01:50.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4268104cb0 0x7f426819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:50.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:50.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810b040 0x7f42681a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4268114d10 con 0x7f426810a6c0 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4268114b90 con 0x7f4268104cb0 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f426ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4268114e90 con 0x7f426810b040 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f425fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f425fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33428/0 (socket says 172.21.15.138:33428) 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f4267e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4268104cb0 0x7f426819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.253+0000 7f425fe00640 1 -- 172.21.15.138:0/2011236942 learned_addr learned my addr 172.21.15.138:0/2011236942 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f426cc00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810b040 0x7f42681a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 -- 172.21.15.138:0/2011236942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810b040 msgr2=0x7f42681a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810b040 0x7f42681a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 -- 172.21.15.138:0/2011236942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4268104cb0 msgr2=0x7f426819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:01:50.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4268104cb0 0x7f426819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 -- 172.21.15.138:0/2011236942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42681a1230 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f425fe00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000cd80 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42500108e0 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.254+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42681a1520 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.255+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4250018980 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.255+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42681a7be0 con 0x7f426810a6c0 2024-08-20T01:01:50.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.256+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4268105090 con 0x7f426810a6c0 2024-08-20T01:01:50.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.258+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f4250010430 con 0x7f426810a6c0 2024-08-20T01:01:50.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.259+0000 7f4265600640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 0x7f424c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:01:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.259+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4748+0+0 (secure 0 0 0) 0x7f42500e3210 con 0x7f426810a6c0 2024-08-20T01:01:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.259+0000 7f4267e00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 0x7f424c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:01:50.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.260+0000 7f4267e00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 0x7f424c0b60e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4258005ab0 tx=0x7f4258005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:01:50.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.263+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f42500aaf60 con 0x7f426810a6c0 2024-08-20T01:01:50.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:01:50.457+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f42680632c0 con 0x7f424c0b3c20 2024-08-20T01:01:51.365 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:01:51.365 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:01:51.365 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: osdmap e38: 6 total, 6 up, 6 in 2024-08-20T01:01:51.366 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:51.366 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:51.366 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:51.366 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:51.366 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: osdmap e38: 6 total, 6 up, 6 in 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:51.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:51.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: osdmap e38: 6 total, 6 up, 6 in 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: from='client.14508 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:01:51.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:01:51.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:01:51.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:01:52.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:52 smithi138 ceph-mon[18076]: osdmap e39: 6 total, 6 up, 6 in 2024-08-20T01:01:52.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:52 smithi167 ceph-mon[21965]: osdmap e39: 6 total, 6 up, 6 in 2024-08-20T01:01:52.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:52 smithi142 ceph-mon[21799]: osdmap e39: 6 total, 6 up, 6 in 2024-08-20T01:01:53.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:53 smithi138 ceph-mon[18076]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:53.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:53 smithi167 ceph-mon[21965]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:53.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:53 smithi142 ceph-mon[21799]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:01:54.448 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:54 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"} : dispatch 2024-08-20T01:01:54.448 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:54 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"}]': finished 2024-08-20T01:01:54.449 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:01:54.449 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:54 smithi138 ceph-mon[18076]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T01:01:54.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:54 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"} : dispatch 2024-08-20T01:01:54.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:54 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"}]': finished 2024-08-20T01:01:54.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:01:54.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:54 smithi167 ceph-mon[21965]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T01:01:54.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:54 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"} : dispatch 2024-08-20T01:01:54.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:54 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/535843917' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b985af05-d307-4762-a760-8ab5fca6da56"}]': finished 2024-08-20T01:01:54.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:01:54.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:54 smithi142 ceph-mon[21799]: osdmap e40: 7 total, 6 up, 7 in 2024-08-20T01:01:55.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:55 smithi167 ceph-mon[21965]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:55.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:55 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2254417606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:55.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:55 smithi142 ceph-mon[21799]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:55.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:55 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2254417606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:55.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:55 smithi138 ceph-mon[18076]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-08-20T01:01:55.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:55 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2254417606' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:01:57.731 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:57 smithi167 ceph-mon[21965]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:57.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:57 smithi142 ceph-mon[21799]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:57 smithi138 ceph-mon[18076]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-08-20T01:01:59.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:01:59 smithi142 ceph-mon[21799]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:01:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:01:59 smithi138 ceph-mon[18076]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:01:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:01:59 smithi167 ceph-mon[21965]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:02:01.569 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:01 smithi138 ceph-mon[18076]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:02:01.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:01 smithi142 ceph-mon[21799]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:02:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:01 smithi167 ceph-mon[21965]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-08-20T01:02:03.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:03 smithi142 ceph-mon[21799]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:02:03.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:03 smithi138 ceph-mon[18076]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:02:03.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:03 smithi167 ceph-mon[21965]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:02:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T01:02:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:04 smithi142 ceph-mon[21799]: Deploying daemon osd.6 on smithi138 2024-08-20T01:02:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T01:02:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:04 smithi138 ceph-mon[18076]: Deploying daemon osd.6 on smithi138 2024-08-20T01:02:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-08-20T01:02:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:04 smithi167 ceph-mon[21965]: Deploying daemon osd.6 on smithi138 2024-08-20T01:02:05.690 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:05 smithi138 ceph-mon[18076]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:05 smithi142 ceph-mon[21799]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:05 smithi167 ceph-mon[21965]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:07.618 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:07 smithi138 ceph-mon[18076]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:07 smithi167 ceph-mon[21965]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:07 smithi142 ceph-mon[21799]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:09 smithi138 ceph-mon[18076]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:09 smithi167 ceph-mon[21965]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:09 smithi142 ceph-mon[21799]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:11.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:11 smithi138 ceph-mon[18076]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:11.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:11.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:11 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:11.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:11 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:11 smithi167 ceph-mon[21965]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:11 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:11 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:11 smithi142 ceph-mon[21799]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:11 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:11 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:13 smithi138 ceph-mon[18076]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:13 smithi167 ceph-mon[21965]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:13 smithi142 ceph-mon[21799]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:15.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:15.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:15.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:15.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:15.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:15.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:15.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.498+0000 7f4265600640 1 -- 172.21.15.138:0/2011236942 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f42680632c0 con 0x7f424c0b3c20 2024-08-20T01:02:15.496 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 6 on host 'smithi138' 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 msgr2=0x7f424c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 0x7f424c0b60e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4258005ab0 tx=0x7f4258005a20 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 msgr2=0x7f426819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f425000e9f0 tx=0x7f425000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 shutdown_connections 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f424c0b3c20 0x7f424c0b60e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f426810b040 0x7f42681a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f426810a6c0 0x7f426819b6c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 --2- 172.21.15.138:0/2011236942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4268104cb0 0x7f426819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 >> 172.21.15.138:0/2011236942 conn(0x7f4268100720 msgr2=0x7f4268102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 shutdown_connections 2024-08-20T01:02:15.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:02:15.501+0000 7f426ec00640 1 -- 172.21.15.138:0/2011236942 wait complete. 2024-08-20T01:02:16.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:16.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.383 DEBUG:teuthology.orchestra.run.smithi138:osd.6> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.6.service 2024-08-20T01:02:16.388 INFO:tasks.cephadm:Deploying osd.7 on smithi142 with /dev/vg_nvme/lv_2... 2024-08-20T01:02:16.388 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T01:02:16.411 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:16.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.412 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:16.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:16.641 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:02:17.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:17 smithi138 ceph-mon[18076]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:17.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:17 smithi138 ceph-mon[18076]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T01:02:17.291 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:02:17 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:02:17.025+0000 7f84da69a640 -1 osd.6 0 log_to_monitors true 2024-08-20T01:02:17.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:17 smithi167 ceph-mon[21965]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:17.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:17 smithi167 ceph-mon[21965]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T01:02:17.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:17 smithi142 ceph-mon[21799]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:17.471 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:17 smithi142 ceph-mon[21799]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-08-20T01:02:18.345 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:02:18.095+0000 7f84d5600640 -1 osd.6 0 waiting for initial osdmap 2024-08-20T01:02:18.346 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:02:18.105+0000 7f84d0600640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:02:18.346 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-mon[18076]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T01:02:18.346 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:18.346 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-mon[18076]: osdmap e41: 7 total, 6 up, 7 in 2024-08-20T01:02:18.346 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:18 smithi138 ceph-mon[18076]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:02:18.380 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:18 smithi142 ceph-mon[21799]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T01:02:18.380 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:18 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:18.380 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:18 smithi142 ceph-mon[21799]: osdmap e41: 7 total, 6 up, 7 in 2024-08-20T01:02:18.380 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:18 smithi142 ceph-mon[21799]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:02:18.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:18 smithi167 ceph-mon[21965]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-08-20T01:02:18.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:18 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:18.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:18 smithi167 ceph-mon[21965]: osdmap e41: 7 total, 6 up, 7 in 2024-08-20T01:02:18.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:18 smithi167 ceph-mon[21965]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:02:19.094 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T01:02:19.120 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-08-20T01:02:19.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:19 smithi142 ceph-mon[21799]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:02:19.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:19 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:19 smithi142 ceph-mon[21799]: osdmap e42: 7 total, 6 up, 7 in 2024-08-20T01:02:19.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:19 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.184 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:19 smithi142 ceph-mon[21799]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:19.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:19 smithi138 ceph-mon[18076]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:02:19.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:19 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:19 smithi138 ceph-mon[18076]: osdmap e42: 7 total, 6 up, 7 in 2024-08-20T01:02:19.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:19 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:19 smithi138 ceph-mon[18076]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:19.402 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:02:19.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:19 smithi167 ceph-mon[21965]: from='osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:02:19.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:19 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:19 smithi167 ceph-mon[21965]: osdmap e42: 7 total, 6 up, 7 in 2024-08-20T01:02:19.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:19 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:19.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:19 smithi167 ceph-mon[21965]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-08-20T01:02:19.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 -- 172.21.15.142:0/3871738325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 msgr2=0x7fbab01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:19.853 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 --2- 172.21.15.142:0/3871738325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab01088f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba98009900 tx=0x7fba9802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 -- 172.21.15.142:0/3871738325 shutdown_connections 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 --2- 172.21.15.142:0/3871738325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0108e30 0x7fbab010f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 --2- 172.21.15.142:0/3871738325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab01088f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 --2- 172.21.15.142:0/3871738325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0102aa0 0x7fbab0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 -- 172.21.15.142:0/3871738325 >> 172.21.15.142:0/3871738325 conn(0x7fbab00fe550 msgr2=0x7fbab0100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 -- 172.21.15.142:0/3871738325 shutdown_connections 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.838+0000 7fbab68fb640 1 -- 172.21.15.142:0/3871738325 wait complete. 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 Processor -- start 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 -- start start 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0102aa0 0x7fbab00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:19.854 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0108e30 0x7fbab0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112af0 con 0x7fbab0102aa0 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112970 con 0x7fbab01084b0 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab68fb640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbab0112c70 con 0x7fbab0108e30 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:57040/0 (socket says 172.21.15.142:57040) 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 -- 172.21.15.142:0/11322884 learned_addr learned my addr 172.21.15.142:0/11322884 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaaffff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0102aa0 0x7fbab00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbab4e98640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0108e30 0x7fbab0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 -- 172.21.15.142:0/11322884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0108e30 msgr2=0x7fbab0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0108e30 0x7fbab0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 -- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0102aa0 msgr2=0x7fbab00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:19.855 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0102aa0 0x7fbab00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 -- 172.21.15.142:0/11322884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab01a72e0 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.839+0000 7fbaa7fff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba980042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.840+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9803f030 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.840+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba98003800 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.840+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab01a7570 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.840+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98035070 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.840+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab01a7a50 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.841+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab0102e80 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.842+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fba98003cb0 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.843+0000 7fbaadffb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 0x7fba9c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.843+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5180+0+0 (secure 0 0 0) 0x7fba98103970 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.847+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fba980cbc00 con 0x7fbab01084b0 2024-08-20T01:02:19.856 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.847+0000 7fbaaffff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 0x7fba9c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:19.857 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:19.847+0000 7fbaaffff640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 0x7fba9c0b60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000abe0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:02:20.029 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:20.030+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbab0105a70 con 0x7fba9c0b3c20 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] boot 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: osdmap e43: 7 total, 7 up, 7 in 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:20.437 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:20.438 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:20 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:20.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] boot 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: osdmap e43: 7 total, 7 up, 7 in 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:20.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:20 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:20.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: osd.6 [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] boot 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: osdmap e43: 7 total, 7 up, 7 in 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:20.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:20 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:21.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: osdmap e44: 7 total, 7 up, 7 in 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi138 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 5696M 2024-08-20T01:02:21.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:21.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:21.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.404 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:21 smithi142 ceph-mon[21799]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:21.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:21.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: osdmap e44: 7 total, 7 up, 7 in 2024-08-20T01:02:21.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi138 2024-08-20T01:02:21.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 5696M 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:21 smithi167 ceph-mon[21965]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:21.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: osdmap e44: 7 total, 7 up, 7 in 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi138 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 5696M 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:21.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:21.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:21.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:21 smithi138 ceph-mon[18076]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:22 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/422176319' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:22 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:22 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"}]': finished 2024-08-20T01:02:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:22 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:22 smithi167 ceph-mon[21965]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T01:02:22.727 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:22 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/422176319' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:22 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:22 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"}]': finished 2024-08-20T01:02:22.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:22 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:22.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:22 smithi142 ceph-mon[21799]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T01:02:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:22 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/422176319' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:22 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"} : dispatch 2024-08-20T01:02:22.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:22 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03d4be9b-678d-41bc-813e-1e1d5bb2a29f"}]': finished 2024-08-20T01:02:22.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:22 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:22.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:22 smithi138 ceph-mon[18076]: osdmap e45: 8 total, 7 up, 8 in 2024-08-20T01:02:23.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:23 smithi167 ceph-mon[21965]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:23.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:23 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/264771330' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:23.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:23 smithi142 ceph-mon[21799]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:23.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:23 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/264771330' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:23 smithi138 ceph-mon[18076]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:23 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/264771330' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:25 smithi167 ceph-mon[21965]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:25 smithi142 ceph-mon[21799]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:25.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:25 smithi138 ceph-mon[18076]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:27.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:27 smithi142 ceph-mon[21799]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:27.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:27 smithi167 ceph-mon[21965]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:27.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:27 smithi138 ceph-mon[18076]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:29.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:29 smithi138 ceph-mon[18076]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:29.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:29 smithi167 ceph-mon[21965]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:29.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:29 smithi142 ceph-mon[21799]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:31.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:31 smithi167 ceph-mon[21965]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:31.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:31 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T01:02:31.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:31 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:31.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:31 smithi142 ceph-mon[21799]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:31.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:31 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T01:02:31.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:31 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:31 smithi138 ceph-mon[18076]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:31 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-08-20T01:02:31.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:31 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:32.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:32 smithi142 ceph-mon[21799]: Deploying daemon osd.7 on smithi142 2024-08-20T01:02:32.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:32 smithi138 ceph-mon[18076]: Deploying daemon osd.7 on smithi142 2024-08-20T01:02:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:32 smithi167 ceph-mon[21965]: Deploying daemon osd.7 on smithi142 2024-08-20T01:02:33.728 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:33 smithi142 ceph-mon[21799]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:33 smithi138 ceph-mon[18076]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:33 smithi167 ceph-mon[21965]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:35.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:35 smithi142 ceph-mon[21799]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:35.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:35 smithi138 ceph-mon[18076]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:35 smithi167 ceph-mon[21965]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:36 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:36.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:36 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:36.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:36.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:36 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:37.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:37 smithi142 ceph-mon[21799]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:37.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:37 smithi138 ceph-mon[18076]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:37 smithi167 ceph-mon[21965]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:39.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:39 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:39.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:39 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:39 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:39.989 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.990+0000 7fbaadffb640 1 -- 172.21.15.142:0/11322884 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbab0105a70 con 0x7fba9c0b3c20 2024-08-20T01:02:39.989 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 7 on host 'smithi142' 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 msgr2=0x7fba9c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 0x7fba9c0b60e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbaa000abe0 tx=0x7fbaa000a2d0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 msgr2=0x7fbab0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fba98006010 tx=0x7fba980042f0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 shutdown_connections 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba9c0b3c20 0x7fba9c0b60e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.991 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbab0108e30 0x7fbab0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbab01084b0 0x7fbab0075710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 --2- 172.21.15.142:0/11322884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbab0102aa0 0x7fbab00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:39.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.992+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 >> 172.21.15.142:0/11322884 conn(0x7fbab00fe550 msgr2=0x7fbab01019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:02:39.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.993+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 shutdown_connections 2024-08-20T01:02:39.992 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:02:39.993+0000 7fbab68fb640 1 -- 172.21.15.142:0/11322884 wait complete. 2024-08-20T01:02:40.627 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:40.627 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.627 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.655 DEBUG:teuthology.orchestra.run.smithi142:osd.7> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.7.service 2024-08-20T01:02:40.698 INFO:tasks.cephadm:Deploying osd.8 on smithi167 with /dev/vg_nvme/lv_2... 2024-08-20T01:02:40.698 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-08-20T01:02:40.729 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:40.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.730 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:02:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:40.953 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:02:41.631 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:41 smithi167 ceph-mon[21965]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:41.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:41 smithi142 ceph-mon[21799]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:41.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:41 smithi138 ceph-mon[18076]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:42.488 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:02:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:02:42.155+0000 7f4f6da14640 -1 osd.7 0 log_to_monitors true 2024-08-20T01:02:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:42 smithi142 ceph-mon[21799]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:42 smithi142 ceph-mon[21799]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:42.746 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:42 smithi167 ceph-mon[21965]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:42.747 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:42 smithi167 ceph-mon[21965]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:42 smithi138 ceph-mon[18076]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:42 smithi138 ceph-mon[18076]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:43.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: osdmap e46: 8 total, 7 up, 8 in 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:02:43.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:02:43.855 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T01:02:43.882 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_2 2024-08-20T01:02:44.174 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:02:44.241 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:02:43.885+0000 7f4f69997640 -1 osd.7 0 waiting for initial osdmap 2024-08-20T01:02:44.241 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:02:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:02:43.896+0000 7f4f617b6640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:02:44.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi142 2024-08-20T01:02:44.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 5696M 2024-08-20T01:02:44.581 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: osdmap e47: 8 total, 7 up, 8 in 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: from='osd.7 ' entity='osd.7' 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] boot 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T01:02:44.582 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:44.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 -- 172.21.15.167:0/2157963431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064107d20 msgr2=0x7f306410e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 --2- 172.21.15.167:0/2157963431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064107d20 0x7f306410e520 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f305c009cc0 tx=0x7f305c02cad0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 -- 172.21.15.167:0/2157963431 shutdown_connections 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 --2- 172.21.15.167:0/2157963431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064107d20 0x7f306410e520 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 --2- 172.21.15.167:0/2157963431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 0x7f30641077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 --2- 172.21.15.167:0/2157963431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064101990 0x7f3064101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 -- 172.21.15.167:0/2157963431 >> 172.21.15.167:0/2157963431 conn(0x7f30640fd400 msgr2=0x7f30640ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 -- 172.21.15.167:0/2157963431 shutdown_connections 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 -- 172.21.15.167:0/2157963431 wait complete. 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.667+0000 7f306c44b640 1 Processor -- start 2024-08-20T01:02:44.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 -- start start 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 0x7f3064197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064107d20 0x7f306419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30641108e0 con 0x7f3064107d20 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3064110760 con 0x7f30641073a0 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306c44b640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3064110a60 con 0x7f3064101990 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42110/0 (socket says 172.21.15.167:42110) 2024-08-20T01:02:44.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 -- 172.21.15.167:0/642180056 learned_addr learned my addr 172.21.15.167:0/642180056 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f30699e6640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 0x7f3064197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a9e8640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064107d20 0x7f306419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 -- 172.21.15.167:0/642180056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 msgr2=0x7f3064197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 0x7f3064197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 -- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064107d20 msgr2=0x7f306419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064107d20 0x7f306419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 -- 172.21.15.167:0/642180056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f306419cde0 con 0x7f3064101990 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f306a1e7640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f305400c890 tx=0x7f305400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.668+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054004220 con 0x7f3064101990 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.669+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f306419d0d0 con 0x7f3064101990 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.669+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30540043c0 con 0x7f3064101990 2024-08-20T01:02:44.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.669+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3054004a00 con 0x7f3064101990 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.669+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30641a3960 con 0x7f3064101990 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.670+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3064101d70 con 0x7f3064101990 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.671+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f30540079b0 con 0x7f3064101990 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.672+0000 7f305b7fe640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 0x7f30500b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.672+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5645+0+0 (secure 0 0 0) 0x7f30540e3350 con 0x7f3064101990 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.672+0000 7f30699e6640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 0x7f30500b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.673+0000 7f30699e6640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 0x7f30500b60e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3064100c70 tx=0x7f306000b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:02:44.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.675+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3054014030 con 0x7f3064101990 2024-08-20T01:02:44.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi142 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 5696M 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: osdmap e47: 8 total, 7 up, 8 in 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: from='osd.7 ' entity='osd.7' 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] boot 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T01:02:44.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:44.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:02:44.861+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f30640632c0 con 0x7f30500b3c20 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi142 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 5696M 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: osdmap e47: 8 total, 7 up, 8 in 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: from='osd.7 ' entity='osd.7' 2024-08-20T01:02:44.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: osd.7 [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] boot 2024-08-20T01:02:44.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: osdmap e48: 8 total, 8 up, 8 in 2024-08-20T01:02:44.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-08-20T01:02:45.592 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:45.593 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:45 smithi167 ceph-mon[21965]: osdmap e49: 8 total, 8 up, 8 in 2024-08-20T01:02:45.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:45 smithi138 ceph-mon[18076]: osdmap e49: 8 total, 8 up, 8 in 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:45.893 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:45 smithi142 ceph-mon[21799]: osdmap e49: 8 total, 8 up, 8 in 2024-08-20T01:02:46.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:46 smithi167 ceph-mon[21965]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:46.587 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:46 smithi167 ceph-mon[21965]: osdmap e50: 8 total, 8 up, 8 in 2024-08-20T01:02:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:46 smithi138 ceph-mon[18076]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:46 smithi138 ceph-mon[18076]: osdmap e50: 8 total, 8 up, 8 in 2024-08-20T01:02:46.908 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:46 smithi142 ceph-mon[21799]: from='client.24487 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:02:46.909 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:46 smithi142 ceph-mon[21799]: osdmap e50: 8 total, 8 up, 8 in 2024-08-20T01:02:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/1865611230' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"}]': finished 2024-08-20T01:02:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T01:02:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:02:47.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:47.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/1865611230' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"}]': finished 2024-08-20T01:02:47.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T01:02:47.882 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:02:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/1865611230' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"} : dispatch 2024-08-20T01:02:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "42122c07-2099-4411-8845-faa6f97fd38d"}]': finished 2024-08-20T01:02:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: osdmap e51: 9 total, 8 up, 9 in 2024-08-20T01:02:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:02:48.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:48 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/1626150010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:48.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:48 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/1626150010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:48 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/1626150010' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:02:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:49 smithi138 ceph-mon[18076]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:49 smithi167 ceph-mon[21965]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:49 smithi142 ceph-mon[21799]: pgmap v166: 1 pgs: 1 remapped; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:02:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:51 smithi138 ceph-mon[18076]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:02:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:51 smithi167 ceph-mon[21965]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:02:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:51 smithi142 ceph-mon[21799]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:02:53.779 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:53 smithi167 ceph-mon[21965]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-08-20T01:02:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:53 smithi138 ceph-mon[18076]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-08-20T01:02:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:53 smithi142 ceph-mon[21799]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 158 KiB/s, 0 objects/s recovering 2024-08-20T01:02:55.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:55 smithi167 ceph-mon[21965]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-20T01:02:55.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:55 smithi138 ceph-mon[18076]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-20T01:02:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:55 smithi142 ceph-mon[21799]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-08-20T01:02:56.682 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T01:02:56.682 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:56.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T01:02:56.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-08-20T01:02:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:02:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:57 smithi138 ceph-mon[18076]: Deploying daemon osd.8 on smithi167 2024-08-20T01:02:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:57 smithi138 ceph-mon[18076]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-08-20T01:02:57.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:57 smithi167 ceph-mon[21965]: Deploying daemon osd.8 on smithi167 2024-08-20T01:02:57.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:57 smithi167 ceph-mon[21965]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-08-20T01:02:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:57 smithi142 ceph-mon[21799]: Deploying daemon osd.8 on smithi167 2024-08-20T01:02:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:57 smithi142 ceph-mon[21799]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-08-20T01:02:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:02:59 smithi138 ceph-mon[18076]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-08-20T01:02:59.802 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:02:59 smithi167 ceph-mon[21965]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-08-20T01:02:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:02:59 smithi142 ceph-mon[21799]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-08-20T01:03:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:01 smithi167 ceph-mon[21965]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:03:01.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:01 smithi142 ceph-mon[21799]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:03:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:01 smithi138 ceph-mon[18076]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:03:02.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:02.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:02.947 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:02.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:03.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:03.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:03.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:03.674 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:03 smithi167 ceph-mon[21965]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:03 smithi142 ceph-mon[21799]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:03 smithi138 ceph-mon[18076]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:05.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.496+0000 7f305b7fe640 1 -- 172.21.15.167:0/642180056 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f30640632c0 con 0x7f30500b3c20 2024-08-20T01:03:05.495 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 8 on host 'smithi167' 2024-08-20T01:03:05.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 msgr2=0x7f30500b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:05.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 0x7f30500b60e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3064100c70 tx=0x7f306000b390 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 msgr2=0x7f3064196d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f305400c890 tx=0x7f305400cd50 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 shutdown_connections 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30500b3c20 0x7f30500b60e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3064107d20 0x7f306419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30641073a0 0x7f3064197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 --2- 172.21.15.167:0/642180056 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3064101990 0x7f3064196d30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.498+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 >> 172.21.15.167:0/642180056 conn(0x7f30640fd400 msgr2=0x7f30640ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.499+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 shutdown_connections 2024-08-20T01:03:05.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:03:05.499+0000 7f306c44b640 1 -- 172.21.15.167:0/642180056 wait complete. 2024-08-20T01:03:05.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:05 smithi167 ceph-mon[21965]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:05 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.843 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:05 smithi138 ceph-mon[18076]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:05.843 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.843 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.843 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:05 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:05 smithi142 ceph-mon[21799]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:05 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:06.164 DEBUG:teuthology.orchestra.run.smithi167:osd.8> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.8.service 2024-08-20T01:03:06.167 INFO:tasks.cephadm:Deploying osd.9 on smithi138 with /dev/vg_nvme/lv_1... 2024-08-20T01:03:06.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T01:03:06.548 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:03:06.657 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:06 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:06.878 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:06 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:06.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:06 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:07.692 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:07 smithi167 ceph-mon[21965]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:07.745 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:07 smithi138 ceph-mon[18076]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:07.963 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:03:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:03:07.821+0000 7ff7fe4c9640 -1 osd.8 0 log_to_monitors true 2024-08-20T01:03:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:07 smithi142 ceph-mon[21799]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:08.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:08 smithi138 ceph-mon[18076]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:08.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:08 smithi138 ceph-mon[18076]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:08.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:08 smithi167 ceph-mon[21965]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:08.862 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:08 smithi167 ceph-mon[21965]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:08 smithi142 ceph-mon[21799]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:08 smithi142 ceph-mon[21799]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-08-20T01:03:09.599 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:03:09.587+0000 7ff7fa44c640 -1 osd.8 0 waiting for initial osdmap 2024-08-20T01:03:09.599 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:03:09.599+0000 7ff7f6273640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:03:09.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:09.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T01:03:09.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: osdmap e52: 9 total, 8 up, 9 in 2024-08-20T01:03:09.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:09.599 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.600 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.601 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:09 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: osdmap e52: 9 total, 8 up, 9 in 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.647 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:09.648 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.649 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:09 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:09.767 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:03:09.800 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_1 2024-08-20T01:03:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-08-20T01:03:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: osdmap e52: 9 total, 8 up, 9 in 2024-08-20T01:03:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:09.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:09.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:09.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:09.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:09 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:10.173 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 -- 172.21.15.138:0/3154534738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 msgr2=0x7f1ba4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 --2- 172.21.15.138:0/3154534738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba4104ee0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1b94009900 tx=0x7f1b9402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 -- 172.21.15.138:0/3154534738 shutdown_connections 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 --2- 172.21.15.138:0/3154534738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410ae90 0x7f1ba4111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 --2- 172.21.15.138:0/3154534738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410a510 0x7f1ba410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 --2- 172.21.15.138:0/3154534738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba4104ee0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.715+0000 7f1ba3200640 1 -- 172.21.15.138:0/3154534738 >> 172.21.15.138:0/3154534738 conn(0x7f1ba4100550 msgr2=0x7f1ba4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.716+0000 7f1ba3200640 1 -- 172.21.15.138:0/3154534738 shutdown_connections 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.716+0000 7f1ba3200640 1 -- 172.21.15.138:0/3154534738 wait complete. 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.716+0000 7f1ba3200640 1 Processor -- start 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.716+0000 7f1ba3200640 1 -- start start 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 0x7f1ba419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410ae90 0x7f1ba41a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4114ba0 con 0x7f1ba4104b00 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4114a20 con 0x7f1ba410ae90 2024-08-20T01:03:10.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1ba4114d20 con 0x7f1ba410a510 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35872/0 (socket says 172.21.15.138:35872) 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba1e00640 1 -- 172.21.15.138:0/127761194 learned_addr learned my addr 172.21.15.138:0/127761194 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba1400640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 0x7f1ba419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.717+0000 7f1ba2800640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410ae90 0x7f1ba41a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 -- 172.21.15.138:0/127761194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 msgr2=0x7f1ba419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 0x7f1ba419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 -- 172.21.15.138:0/127761194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410ae90 msgr2=0x7f1ba41a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410ae90 0x7f1ba41a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:10.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 -- 172.21.15.138:0/127761194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ba41a0fa0 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1400640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 0x7f1ba419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba1e00640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005ab0 tx=0x7f1b94003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9403f070 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b94031b60 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b94035030 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ba41a1230 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.718+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ba41a7950 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.719+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ba4104ee0 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.723+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f1b94042070 con 0x7f1ba4104b00 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.725+0000 7f1b9aa00640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 0x7f1b880b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:10.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.725+0000 7f1ba1400640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 0x7f1b880b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:10.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.725+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6077+0+0 (secure 0 0 0) 0x7f1b94103f90 con 0x7f1ba4104b00 2024-08-20T01:03:10.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.726+0000 7f1ba1400640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 0x7f1b880b6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4102df0 tx=0x7f1b90005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:03:10.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.727+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1b940cbe00 con 0x7f1ba4104b00 2024-08-20T01:03:10.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:10.922+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1ba4107ad0 con 0x7f1b880b3cc0 2024-08-20T01:03:10.920 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi167 2024-08-20T01:03:10.920 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-20T01:03:10.920 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:10.920 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:03:10.921 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: osdmap e53: 9 total, 8 up, 9 in 2024-08-20T01:03:10.921 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.921 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.921 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:10 smithi138 ceph-mon[18076]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi167 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: osdmap e53: 9 total, 8 up, 9 in 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:10 smithi167 ceph-mon[21965]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi167 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 5696M 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: osdmap e53: 9 total, 8 up, 9 in 2024-08-20T01:03:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:10.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:10 smithi142 ceph-mon[21799]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 219 MiB used, 715 GiB / 715 GiB avail 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] boot 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:11.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:11.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:11 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] boot 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:11.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:11 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: osd.8 [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] boot 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: osdmap e54: 9 total, 9 up, 9 in 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:11 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:12.913 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:12 smithi138 ceph-mon[18076]: osdmap e55: 9 total, 9 up, 9 in 2024-08-20T01:03:12.913 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:12 smithi138 ceph-mon[18076]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:12.913 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:12 smithi138 ceph-mon[18076]: osdmap e56: 9 total, 9 up, 9 in 2024-08-20T01:03:12.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:12 smithi167 ceph-mon[21965]: osdmap e55: 9 total, 9 up, 9 in 2024-08-20T01:03:12.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:12 smithi167 ceph-mon[21965]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:12.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:12 smithi167 ceph-mon[21965]: osdmap e56: 9 total, 9 up, 9 in 2024-08-20T01:03:12.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:12 smithi142 ceph-mon[21799]: osdmap e55: 9 total, 9 up, 9 in 2024-08-20T01:03:12.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:12 smithi142 ceph-mon[21799]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:12.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:12 smithi142 ceph-mon[21799]: osdmap e56: 9 total, 9 up, 9 in 2024-08-20T01:03:14.405 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:14 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"} : dispatch 2024-08-20T01:03:14.405 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:14 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"}]': finished 2024-08-20T01:03:14.405 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:14 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:14.405 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:14 smithi138 ceph-mon[18076]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T01:03:14.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:14 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"} : dispatch 2024-08-20T01:03:14.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:14 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"}]': finished 2024-08-20T01:03:14.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:14 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:14.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:14 smithi167 ceph-mon[21965]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T01:03:14.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:14 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"} : dispatch 2024-08-20T01:03:14.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:14 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3512992809' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a1167294-30ac-465b-a5ff-e50abac4436f"}]': finished 2024-08-20T01:03:14.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:14 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:14.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:14 smithi142 ceph-mon[21799]: osdmap e57: 10 total, 9 up, 10 in 2024-08-20T01:03:15.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:15 smithi167 ceph-mon[21965]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:15.364 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:15 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3878266730' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:15.392 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:15 smithi142 ceph-mon[21799]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:15.392 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:15 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3878266730' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:15.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:15 smithi138 ceph-mon[18076]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:15.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:15 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3878266730' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:17.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:17 smithi142 ceph-mon[21799]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-08-20T01:03:17.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:17 smithi138 ceph-mon[18076]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-08-20T01:03:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:17 smithi167 ceph-mon[21965]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-08-20T01:03:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:19 smithi138 ceph-mon[18076]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-20T01:03:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:19 smithi167 ceph-mon[21965]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-20T01:03:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:19 smithi142 ceph-mon[21799]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-08-20T01:03:21.689 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:21 smithi138 ceph-mon[18076]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:03:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:21 smithi167 ceph-mon[21965]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:03:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:21 smithi142 ceph-mon[21799]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-08-20T01:03:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:23 smithi138 ceph-mon[18076]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T01:03:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:23 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T01:03:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:23 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:23 smithi167 ceph-mon[21965]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T01:03:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:23 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T01:03:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:23 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:23 smithi142 ceph-mon[21799]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-08-20T01:03:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:23 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-08-20T01:03:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:23 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:24.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:24 smithi138 ceph-mon[18076]: Deploying daemon osd.9 on smithi138 2024-08-20T01:03:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:24 smithi167 ceph-mon[21965]: Deploying daemon osd.9 on smithi138 2024-08-20T01:03:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:24 smithi142 ceph-mon[21799]: Deploying daemon osd.9 on smithi138 2024-08-20T01:03:25.676 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:25 smithi138 ceph-mon[18076]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T01:03:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:25 smithi167 ceph-mon[21965]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T01:03:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:25 smithi142 ceph-mon[21799]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-08-20T01:03:27.559 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:27 smithi138 ceph-mon[18076]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:03:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:27 smithi167 ceph-mon[21965]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:03:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:27 smithi142 ceph-mon[21799]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:03:29.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:29 smithi138 ceph-mon[18076]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:29 smithi167 ceph-mon[21965]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:29 smithi142 ceph-mon[21799]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:30.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:31 smithi138 ceph-mon[18076]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:31.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:31 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:31.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:31 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:31 smithi167 ceph-mon[21965]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:31 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:31 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:31 smithi142 ceph-mon[21799]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:31 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:31 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:33 smithi138 ceph-mon[18076]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:33 smithi167 ceph-mon[21965]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:33 smithi142 ceph-mon[21799]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:34.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.866+0000 7f1b9aa00640 1 -- 172.21.15.138:0/127761194 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1ba4107ad0 con 0x7f1b880b3cc0 2024-08-20T01:03:34.921 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 9 on host 'smithi138' 2024-08-20T01:03:34.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 msgr2=0x7f1b880b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:34.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 0x7f1b880b6180 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4102df0 tx=0x7f1b90005f90 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 msgr2=0x7f1ba419aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1b94005ab0 tx=0x7f1b94003c10 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 shutdown_connections 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1b880b3cc0 0x7f1b880b6180 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ba410ae90 0x7f1ba41a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1ba410a510 0x7f1ba419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 --2- 172.21.15.138:0/127761194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ba4104b00 0x7f1ba419aef0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.869+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 >> 172.21.15.138:0/127761194 conn(0x7f1ba4100550 msgr2=0x7f1ba4103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.870+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 shutdown_connections 2024-08-20T01:03:34.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:03:34.870+0000 7f1ba3200640 1 -- 172.21.15.138:0/127761194 wait complete. 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.270 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:35.271 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.272 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:35.688 DEBUG:teuthology.orchestra.run.smithi138:osd.9> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.9.service 2024-08-20T01:03:35.693 INFO:tasks.cephadm:Deploying osd.10 on smithi142 with /dev/vg_nvme/lv_1... 2024-08-20T01:03:35.693 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T01:03:35.946 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:03:36.391 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:36 smithi142 ceph-mon[21799]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:36.391 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:36 smithi142 ceph-mon[21799]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:36.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:36 smithi138 ceph-mon[18076]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:36.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:36 smithi138 ceph-mon[18076]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:36.541 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:03:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:03:36.216+0000 7ff77c722640 -1 osd.9 0 log_to_monitors true 2024-08-20T01:03:36.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:36 smithi167 ceph-mon[21965]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:36.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:36 smithi167 ceph-mon[21965]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-08-20T01:03:37.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T01:03:37.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:37.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.456 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T01:03:37.456 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: osdmap e58: 10 total, 9 up, 10 in 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:03:37.457 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:37 smithi142 ceph-mon[21799]: osdmap e59: 10 total, 9 up, 10 in 2024-08-20T01:03:37.477 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: osdmap e58: 10 total, 9 up, 10 in 2024-08-20T01:03:37.477 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.477 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:37.477 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:03:37.478 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-mon[18076]: osdmap e59: 10 total, 9 up, 10 in 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: from='osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: osdmap e58: 10 total, 9 up, 10 in 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: from='osd.9 ' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]} : dispatch 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-08-20T01:03:37.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:37 smithi167 ceph-mon[21965]: osdmap e59: 10 total, 9 up, 10 in 2024-08-20T01:03:38.040 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:03:37.800+0000 7ff776800640 -1 osd.9 0 waiting for initial osdmap 2024-08-20T01:03:38.041 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:03:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:03:37.823+0000 7ff76c800640 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:03:38.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:38 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:38 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:38 smithi142 ceph-mon[21799]: from='osd.9 ' entity='osd.9' 2024-08-20T01:03:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-08-20T01:03:38.584 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-08-20T01:03:38.645 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:38 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.645 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:38 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.645 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:38 smithi138 ceph-mon[18076]: from='osd.9 ' entity='osd.9' 2024-08-20T01:03:38.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:38 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:38 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:38.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:38 smithi167 ceph-mon[21965]: from='osd.9 ' entity='osd.9' 2024-08-20T01:03:38.838 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:03:39.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 -- 172.21.15.142:0/999598026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 msgr2=0x7fa554101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:39.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 --2- 172.21.15.142:0/999598026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa554101d70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa544009900 tx=0x7fa54402a410 comp rx=0 tx=0).stop 2024-08-20T01:03:39.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 -- 172.21.15.142:0/999598026 shutdown_connections 2024-08-20T01:03:39.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 --2- 172.21.15.142:0/999598026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa554107d20 0x7fa55410e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:39.325 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 --2- 172.21.15.142:0/999598026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa5541073a0 0x7fa5541077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 --2- 172.21.15.142:0/999598026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa554101d70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 -- 172.21.15.142:0/999598026 >> 172.21.15.142:0/999598026 conn(0x7fa5540fd400 msgr2=0x7fa5540ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 -- 172.21.15.142:0/999598026 shutdown_connections 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.305+0000 7fa5588ad640 1 -- 172.21.15.142:0/999598026 wait complete. 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 Processor -- start 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 -- start start 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 0x7fa55419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa554107d20 0x7fa5541a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa554111a30 con 0x7fa5541073a0 2024-08-20T01:03:39.326 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa5541118b0 con 0x7fa554101990 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa5588ad640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa554111bb0 con 0x7fa554107d20 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa55259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa55259c640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:59272/0 (socket says 172.21.15.142:59272) 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.306+0000 7fa55259c640 1 -- 172.21.15.142:0/226145159 learned_addr learned my addr 172.21.15.142:0/226145159 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa552d9d640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa554107d20 0x7fa5541a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa54bfff640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 0x7fa55419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 -- 172.21.15.142:0/226145159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa554107d20 msgr2=0x7fa5541a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa554107d20 0x7fa5541a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 -- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 msgr2=0x7fa55419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 0x7fa55419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 -- 172.21.15.142:0/226145159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5541a12a0 con 0x7fa554101990 2024-08-20T01:03:39.327 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa54bfff640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 0x7fa55419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa55259c640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa544005ab0 tx=0x7fa544003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54403f070 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa544031d60 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5541a1530 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa544035030 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.307+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5541a7c50 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.308+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa554101d70 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.310+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fa544042070 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.310+0000 7fa54b7fe640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 0x7fa5400b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.311+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 6476+0+0 (secure 0 0 0) 0x7fa5441043d0 con 0x7fa554101990 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.311+0000 7fa54bfff640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 0x7fa5400b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:03:39.328 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.312+0000 7fa54bfff640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 0x7fa5400b60e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa554100910 tx=0x7fa53c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:03:39.329 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.314+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa5440cc150 con 0x7fa554101990 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] boot 2024-08-20T01:03:39.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:39 smithi142 ceph-mon[21799]: osdmap e60: 10 total, 10 up, 10 in 2024-08-20T01:03:39.502 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:39.503+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fa55410d810 con 0x7fa5400b3c20 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] boot 2024-08-20T01:03:39.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:39 smithi167 ceph-mon[21965]: osdmap e60: 10 total, 10 up, 10 in 2024-08-20T01:03:39.763 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 804 GiB / 805 GiB avail 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: osd.9 [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] boot 2024-08-20T01:03:39.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:39 smithi138 ceph-mon[18076]: osdmap e60: 10 total, 10 up, 10 in 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='client.24548 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: osdmap e61: 10 total, 10 up, 10 in 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:40.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='client.24548 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: osdmap e61: 10 total, 10 up, 10 in 2024-08-20T01:03:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:40.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='client.24548 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:03:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:03:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:03:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: osdmap e61: 10 total, 10 up, 10 in 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:40.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:41 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi138 2024-08-20T01:03:41.402 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:41 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 4272M 2024-08-20T01:03:41.403 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:41 smithi142 ceph-mon[21799]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:41 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi138 2024-08-20T01:03:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:41 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 4272M 2024-08-20T01:03:41.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:41 smithi167 ceph-mon[21965]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:41.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:41 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi138 2024-08-20T01:03:41.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:41 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 4272M 2024-08-20T01:03:41.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:41 smithi138 ceph-mon[18076]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:42.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:42 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/1022368864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:42 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:42 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"}]': finished 2024-08-20T01:03:42.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:03:42.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:42 smithi142 ceph-mon[21799]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T01:03:42.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:42 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/1022368864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:42 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:42 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"}]': finished 2024-08-20T01:03:42.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:03:42.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:42 smithi167 ceph-mon[21965]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T01:03:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:42 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/1022368864' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:42 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"} : dispatch 2024-08-20T01:03:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:42 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7ee92625-f415-4027-952c-7eb2099d607c"}]': finished 2024-08-20T01:03:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:03:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:42 smithi138 ceph-mon[18076]: osdmap e62: 11 total, 10 up, 11 in 2024-08-20T01:03:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:43 smithi167 ceph-mon[21965]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:43 smithi167 ceph-mon[21965]: from='client.? 172.21.15.142:0/1385045114' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:43 smithi142 ceph-mon[21799]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:43.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:43 smithi142 ceph-mon[21799]: from='client.? 172.21.15.142:0/1385045114' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:43.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:43 smithi138 ceph-mon[18076]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:43.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:43 smithi138 ceph-mon[18076]: from='client.? 172.21.15.142:0/1385045114' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:03:45.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:45 smithi138 ceph-mon[18076]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:45 smithi167 ceph-mon[21965]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:45 smithi142 ceph-mon[21799]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:47 smithi138 ceph-mon[18076]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:47 smithi167 ceph-mon[21965]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:47.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:47 smithi142 ceph-mon[21799]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:49.670 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:49 smithi142 ceph-mon[21799]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:49 smithi138 ceph-mon[18076]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:49 smithi167 ceph-mon[21965]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:50.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:50 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T01:03:50.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:50 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:50 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T01:03:50.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:50 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:50 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-08-20T01:03:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:50 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:51 smithi138 ceph-mon[18076]: Deploying daemon osd.10 on smithi142 2024-08-20T01:03:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:51 smithi138 ceph-mon[18076]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:51.871 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:51 smithi142 ceph-mon[21799]: Deploying daemon osd.10 on smithi142 2024-08-20T01:03:51.871 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:51 smithi142 ceph-mon[21799]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:51 smithi167 ceph-mon[21965]: Deploying daemon osd.10 on smithi142 2024-08-20T01:03:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:51 smithi167 ceph-mon[21965]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:53 smithi142 ceph-mon[21799]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:53 smithi138 ceph-mon[18076]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:53 smithi167 ceph-mon[21965]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:55.785 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:55 smithi142 ceph-mon[21799]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:55.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:55 smithi138 ceph-mon[18076]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:55 smithi167 ceph-mon[21965]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:56.631 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:56.631 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:56.631 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:56.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:56.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:56.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:57.672 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:57 smithi142 ceph-mon[21799]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:57 smithi138 ceph-mon[18076]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:57 smithi167 ceph-mon[21965]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:59.405 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 10 on host 'smithi142' 2024-08-20T01:03:59.405 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.405+0000 7fa54b7fe640 1 -- 172.21.15.142:0/226145159 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fa55410d810 con 0x7fa5400b3c20 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 msgr2=0x7fa5400b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 0x7fa5400b60e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa554100910 tx=0x7fa53c005f90 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 msgr2=0x7fa55419b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa544005ab0 tx=0x7fa544003c80 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 shutdown_connections 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5400b3c20 0x7fa5400b60e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa554107d20 0x7fa5541a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa5541073a0 0x7fa55419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 --2- 172.21.15.142:0/226145159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa554101990 0x7fa55419b1f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 >> 172.21.15.142:0/226145159 conn(0x7fa5540fd400 msgr2=0x7fa55406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 shutdown_connections 2024-08-20T01:03:59.408 INFO:teuthology.orchestra.run.smithi142.stderr:2024-08-20T01:03:59.408+0000 7fa5588ad640 1 -- 172.21.15.142:0/226145159 wait complete. 2024-08-20T01:03:59.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:59.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:59.784 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:59.785 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.785 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:59.785 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.785 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:03:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:03:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:03:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:03:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:00.147 DEBUG:teuthology.orchestra.run.smithi142:osd.10> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.10.service 2024-08-20T01:04:00.150 INFO:tasks.cephadm:Deploying osd.11 on smithi167 with /dev/vg_nvme/lv_1... 2024-08-20T01:04:00.150 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-08-20T01:04:00.405 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:04:01.571 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:01 smithi142 ceph-mon[21799]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:01.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:01 smithi138 ceph-mon[18076]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:01.845 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:01 smithi167 ceph-mon[21965]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:01.992 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:04:01 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:04:01.708+0000 7f5fc99fa640 -1 osd.10 0 log_to_monitors true 2024-08-20T01:04:02.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:02 smithi142 ceph-mon[21799]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:02.659 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:02 smithi142 ceph-mon[21799]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:02.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:02 smithi167 ceph-mon[21965]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:02.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:02 smithi167 ceph-mon[21965]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:02.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:02 smithi138 ceph-mon[18076]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:02.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:02 smithi138 ceph-mon[18076]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-08-20T01:04:03.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-08-20T01:04:03.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch daemon add osd smithi167:vg_nvme/lv_1 2024-08-20T01:04:03.434 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:04:03.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:03.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T01:04:03.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: osdmap e63: 11 total, 10 up, 11 in 2024-08-20T01:04:03.876 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.877 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:03.878 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:03 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.918 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:03.918 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: osdmap e63: 11 total, 10 up, 11 in 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.919 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:03.920 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- 172.21.15.167:0/225704167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc410cdc0 msgr2=0x7effc410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 --2- 172.21.15.167:0/225704167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc410cdc0 0x7effc410f240 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7effbc009ca0 tx=0x7effbc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- 172.21.15.167:0/225704167 shutdown_connections 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 --2- 172.21.15.167:0/225704167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc410cdc0 0x7effc410f240 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 --2- 172.21.15.167:0/225704167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 0x7effc410c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 --2- 172.21.15.167:0/225704167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc4103810 0x7effc4103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- 172.21.15.167:0/225704167 >> 172.21.15.167:0/225704167 conn(0x7effc40fd400 msgr2=0x7effc40ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- 172.21.15.167:0/225704167 shutdown_connections 2024-08-20T01:04:03.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- 172.21.15.167:0/225704167 wait complete. 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 Processor -- start 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.961+0000 7effc99fe640 1 -- start start 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 0x7effc4199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc410cdc0 0x7effc419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7effc4112a90 con 0x7effc410cdc0 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7effc4112910 con 0x7effc4104210 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7effc4112c10 con 0x7effc4103810 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:42090/0 (socket says 172.21.15.167:42090) 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 -- 172.21.15.167:0/2728900605 learned_addr learned my addr 172.21.15.167:0/2728900605 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effbbfff640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 0x7effc4199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:03.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc37fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc410cdc0 0x7effc419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 -- 172.21.15.167:0/2728900605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 msgr2=0x7effc4199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 0x7effc4199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 -- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc410cdc0 msgr2=0x7effc419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc410cdc0 0x7effc419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 -- 172.21.15.167:0/2728900605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc419eff0 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc2ffd640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effb400b4a0 tx=0x7effb400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4014070 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc419f2e0 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effb4010ac0 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.962+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb4017c30 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.963+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc41a5b70 con 0x7effc4103810 2024-08-20T01:04:03.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.963+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc4103bf0 con 0x7effc4103810 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.965+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7effb4017070 con 0x7effc4103810 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.966+0000 7effc89fc640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 0x7effb00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.966+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 6629+0+0 (secure 0 0 0) 0x7effb40e2cd0 con 0x7effc4103810 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.966+0000 7effbbfff640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 0x7effb00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.967+0000 7effbbfff640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 0x7effb00b60e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effc40ffc50 tx=0x7effac00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:03.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:03.968+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7effb40aa9b0 con 0x7effc4103810 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: osdmap e63: 11 total, 10 up, 11 in 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]} : dispatch 2024-08-20T01:04:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:04.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:04.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:03 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:04.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:04.156+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7effc40ff010 con 0x7effb00b3c20 2024-08-20T01:04:04.241 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:04:03.918+0000 7f5fc6191640 -1 osd.10 0 waiting for initial osdmap 2024-08-20T01:04:04.241 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:04:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:04:03.928+0000 7f5fbd79c640 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:04:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi142 2024-08-20T01:04:04.820 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 4272M 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: osdmap e64: 11 total, 10 up, 11 in 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='osd.10 ' entity='osd.10' 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] boot 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:04:04.821 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi142 2024-08-20T01:04:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 4272M 2024-08-20T01:04:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: osdmap e64: 11 total, 10 up, 11 in 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='osd.10 ' entity='osd.10' 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] boot 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:04:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi142 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 4272M 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: osdmap e64: 11 total, 10 up, 11 in 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='osd.10 ' entity='osd.10' 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: osd.10 [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] boot 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: osdmap e65: 11 total, 11 up, 11 in 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-08-20T01:04:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-08-20T01:04:05.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:05 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:04:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:05 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:04:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:05 smithi167 ceph-mon[21965]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:05 smithi167 ceph-mon[21965]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:05 smithi167 ceph-mon[21965]: osdmap e66: 11 total, 11 up, 11 in 2024-08-20T01:04:05.942 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:05 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:04:05.942 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:05 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:04:05.942 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:05 smithi138 ceph-mon[18076]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:05.943 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:05 smithi138 ceph-mon[18076]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:05.943 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:05 smithi138 ceph-mon[18076]: osdmap e66: 11 total, 11 up, 11 in 2024-08-20T01:04:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:05 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:04:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:05 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:04:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:05 smithi142 ceph-mon[21799]: from='client.24562 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi167:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:05 smithi142 ceph-mon[21799]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 278 MiB used, 894 GiB / 894 GiB avail 2024-08-20T01:04:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:05 smithi142 ceph-mon[21799]: osdmap e66: 11 total, 11 up, 11 in 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: osdmap e67: 11 total, 11 up, 11 in 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/3850371150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"}]': finished 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T01:04:07.353 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:07 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: osdmap e67: 11 total, 11 up, 11 in 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/3850371150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"}]': finished 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T01:04:07.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:07 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:07.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: osdmap e67: 11 total, 11 up, 11 in 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/3850371150' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"} : dispatch 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a"}]': finished 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: osdmap e68: 12 total, 11 up, 12 in 2024-08-20T01:04:07.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:07 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:08.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:08 smithi167 ceph-mon[21965]: from='client.? 172.21.15.167:0/3770426120' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:04:08.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:08 smithi142 ceph-mon[21799]: from='client.? 172.21.15.167:0/3770426120' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:04:08.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:08 smithi138 ceph-mon[18076]: from='client.? 172.21.15.167:0/3770426120' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-08-20T01:04:09.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:09 smithi167 ceph-mon[21965]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:09.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:09 smithi142 ceph-mon[21799]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:09.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:09 smithi138 ceph-mon[18076]: pgmap v223: 1 pgs: 1 remapped+peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:11.694 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:11 smithi167 ceph-mon[21965]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:04:11.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:11 smithi138 ceph-mon[18076]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:04:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:11 smithi142 ceph-mon[21799]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-08-20T01:04:13.694 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:13 smithi167 ceph-mon[21965]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-20T01:04:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:13 smithi138 ceph-mon[18076]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-20T01:04:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:13 smithi142 ceph-mon[21799]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-08-20T01:04:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:15 smithi167 ceph-mon[21965]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-20T01:04:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T01:04:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:15 smithi138 ceph-mon[18076]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-20T01:04:15.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T01:04:15.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:15.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:15 smithi142 ceph-mon[21799]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-08-20T01:04:15.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-08-20T01:04:15.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:16.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:16 smithi167 ceph-mon[21965]: Deploying daemon osd.11 on smithi167 2024-08-20T01:04:16.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:16 smithi138 ceph-mon[18076]: Deploying daemon osd.11 on smithi167 2024-08-20T01:04:16.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:16 smithi142 ceph-mon[21799]: Deploying daemon osd.11 on smithi167 2024-08-20T01:04:17.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:17 smithi138 ceph-mon[18076]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:04:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:17 smithi167 ceph-mon[21965]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:04:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:17 smithi142 ceph-mon[21799]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-08-20T01:04:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:19 smithi167 ceph-mon[21965]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-20T01:04:19.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:19 smithi138 ceph-mon[18076]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-20T01:04:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:19 smithi142 ceph-mon[21799]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-08-20T01:04:21.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:21 smithi167 ceph-mon[21965]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:04:21.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:21.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:21.787 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:21 smithi138 ceph-mon[18076]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:04:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:21 smithi142 ceph-mon[21799]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-08-20T01:04:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:23.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:23 smithi167 ceph-mon[21965]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:23 smithi138 ceph-mon[18076]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:23 smithi142 ceph-mon[21799]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:25.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.212+0000 7effc89fc640 1 -- 172.21.15.167:0/2728900605 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7effc40ff010 con 0x7effb00b3c20 2024-08-20T01:04:25.211 INFO:teuthology.orchestra.run.smithi167.stdout:Created osd(s) 11 on host 'smithi167' 2024-08-20T01:04:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 msgr2=0x7effb00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 0x7effb00b60e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7effc40ffc50 tx=0x7effac00b390 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 msgr2=0x7effc4198f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effb400b4a0 tx=0x7effb400b960 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 shutdown_connections 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7effb00b3c20 0x7effb00b60e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effc410cdc0 0x7effc419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effc4104210 0x7effc4199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 --2- 172.21.15.167:0/2728900605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7effc4103810 0x7effc4198f40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 >> 172.21.15.167:0/2728900605 conn(0x7effc40fd400 msgr2=0x7effc410ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 shutdown_connections 2024-08-20T01:04:25.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:25.215+0000 7effc99fe640 1 -- 172.21.15.167:0/2728900605 wait complete. 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:25.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:25.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:25.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:25.933 DEBUG:teuthology.orchestra.run.smithi167:osd.11> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.11.service 2024-08-20T01:04:25.936 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-08-20T01:04:25.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T01:04:26.314 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:26.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2211673246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 msgr2=0x7fb3d4102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2211673246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d4102d00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009900 tx=0x7fb3c402a410 comp rx=0 tx=0).stop 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2211673246 shutdown_connections 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2211673246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 0x7fb3d410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2211673246 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2211673246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d4102d00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:26.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2211673246 >> 172.21.15.138:0/2211673246 conn(0x7fb3d40fe350 msgr2=0x7fb3d4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:26.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2211673246 shutdown_connections 2024-08-20T01:04:26.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.903+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2211673246 wait complete. 2024-08-20T01:04:26.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.904+0000 7fb3d3e00640 1 Processor -- start 2024-08-20T01:04:26.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 -- start start 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d40751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 0x7fb3d4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d41129c0 con 0x7fb3d4102920 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d4112840 con 0x7fb3d4108cb0 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.905+0000 7fb3d3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d4112b40 con 0x7fb3d4108330 2024-08-20T01:04:26.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d40751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d40751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57854/0 (socket says 172.21.15.138:57854) 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2a00640 1 -- 172.21.15.138:0/2266648538 learned_addr learned my addr 172.21.15.138:0/2266648538 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d3400640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 0x7fb3d4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 -- 172.21.15.138:0/2266648538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 msgr2=0x7fb3d4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 0x7fb3d4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:26.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 -- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 msgr2=0x7fb3d40751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d40751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 -- 172.21.15.138:0/2266648538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d41a71b0 con 0x7fb3d4108330 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d2000640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000e9f0 tx=0x7fb3c000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000ccc0 con 0x7fb3d4108330 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3c0005020 con 0x7fb3d4108330 2024-08-20T01:04:26.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d41a74a0 con 0x7fb3d4108330 2024-08-20T01:04:26.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.906+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c000ce60 con 0x7fb3d4108330 2024-08-20T01:04:26.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.907+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d41a7990 con 0x7fb3d4108330 2024-08-20T01:04:26.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.908+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d4102d00 con 0x7fb3d4108330 2024-08-20T01:04:26.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.910+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fb3c00051d0 con 0x7fb3d4108330 2024-08-20T01:04:26.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.911+0000 7fb3cb400640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 0x7fb3b80b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:26.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.911+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7024+0+0 (secure 0 0 0) 0x7fb3c00e3730 con 0x7fb3d4108330 2024-08-20T01:04:26.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.911+0000 7fb3d2a00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 0x7fb3b80b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:26.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.912+0000 7fb3d2a00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 0x7fb3b80b6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40096c0 tx=0x7fb3c4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:26.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:26.915+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb3c00ab210 con 0x7fb3d4108330 2024-08-20T01:04:26.963 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:04:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:04:26.611+0000 7ff248c2c640 -1 osd.11 0 log_to_monitors true 2024-08-20T01:04:27.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.099+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb3d41a7d20 con 0x7fb3d4108330 2024-08-20T01:04:27.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.100+0000 7fb3cb400640 1 -- 172.21.15.138:0/2266648538 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7fb3c00aab10 con 0x7fb3d4108330 2024-08-20T01:04:27.099 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 msgr2=0x7fb3b80b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 0x7fb3b80b6180 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40096c0 tx=0x7fb3c4005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 msgr2=0x7fb3d4075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4075710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000e9f0 tx=0x7fb3c000edf0 comp rx=0 tx=0).stop 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 shutdown_connections 2024-08-20T01:04:27.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3b80b3cc0 0x7fb3b80b6180 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3d4108cb0 0x7fb3d4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb3d4108330 0x7fb3d4075710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 --2- 172.21.15.138:0/2266648538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb3d4102920 0x7fb3d40751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 >> 172.21.15.138:0/2266648538 conn(0x7fb3d40fe350 msgr2=0x7fb3d4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 shutdown_connections 2024-08-20T01:04:27.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:27.103+0000 7fb3d3e00640 1 -- 172.21.15.138:0/2266648538 wait complete. 2024-08-20T01:04:27.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:27 smithi167 ceph-mon[21965]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:27.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:27 smithi167 ceph-mon[21965]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:27 smithi167 ceph-mon[21965]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:27 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2266648538' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:27.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:27 smithi138 ceph-mon[18076]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:27 smithi138 ceph-mon[18076]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:27 smithi138 ceph-mon[18076]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:27 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2266648538' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:27.933 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1724115844,"num_in_osds":12,"osd_in_since":1724115846,"num_remapped_pgs":0} 2024-08-20T01:04:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:27 smithi142 ceph-mon[21799]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:27 smithi142 ceph-mon[21799]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:27 smithi142 ceph-mon[21799]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-08-20T01:04:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:27 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2266648538' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:28 smithi138 ceph-mon[18076]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T01:04:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:28 smithi138 ceph-mon[18076]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:28 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:28 smithi138 ceph-mon[18076]: osdmap e69: 12 total, 11 up, 12 in 2024-08-20T01:04:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:28 smithi138 ceph-mon[18076]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:28.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-mon[21965]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T01:04:28.889 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-mon[21965]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:28.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:28.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-mon[21965]: osdmap e69: 12 total, 11 up, 12 in 2024-08-20T01:04:28.890 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-mon[21965]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:28.935 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd stat -f json 2024-08-20T01:04:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:28 smithi142 ceph-mon[21799]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-08-20T01:04:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:28 smithi142 ceph-mon[21799]: from='osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:28 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:28 smithi142 ceph-mon[21799]: osdmap e69: 12 total, 11 up, 12 in 2024-08-20T01:04:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:28 smithi142 ceph-mon[21799]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]} : dispatch 2024-08-20T01:04:29.213 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:04:28.917+0000 7ff244baf640 -1 osd.11 0 waiting for initial osdmap 2024-08-20T01:04:29.213 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:04:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:04:28.931+0000 7ff23c9ce640 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-08-20T01:04:29.335 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: osdmap e70: 12 total, 11 up, 12 in 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='osd.11 ' entity='osd.11' 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.631 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:29 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 -- 172.21.15.138:0/823267595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540075b90 msgr2=0x7f9540113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 --2- 172.21.15.138:0/823267595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540075b90 0x7f9540113130 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 -- 172.21.15.138:0/823267595 shutdown_connections 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 --2- 172.21.15.138:0/823267595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540113670 0x7f9540115a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 --2- 172.21.15.138:0/823267595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540075b90 0x7f9540113130 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 --2- 172.21.15.138:0/823267595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f9540075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 -- 172.21.15.138:0/823267595 >> 172.21.15.138:0/823267595 conn(0x7f95400fe3c0 msgr2=0x7f95401007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 -- 172.21.15.138:0/823267595 shutdown_connections 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.923+0000 7f9545a00640 1 -- 172.21.15.138:0/823267595 wait complete. 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 Processor -- start 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 -- start start 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f954019f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 0x7f95401a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:29.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9540118f40 con 0x7f9540113670 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9540118dc0 con 0x7f9540075270 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.924+0000 7f9545a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95401190c0 con 0x7f9540075b90 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f9545000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 0x7f95401a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f954019f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56412/0 (socket says 172.21.15.138:56412) 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953f400640 1 -- 172.21.15.138:0/1228913045 learned_addr learned my addr 172.21.15.138:0/1228913045 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 -- 172.21.15.138:0/1228913045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 msgr2=0x7f954019f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 -- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 msgr2=0x7f95401a4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 0x7f95401a4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953fe00640 1 -- 172.21.15.138:0/1228913045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a53c0 con 0x7f9540075270 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f953f400640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.925+0000 7f9545000640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 0x7f95401a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f953fe00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f954019f310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9530005fd0 tx=0x7f953000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530009c30 con 0x7f9540075270 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9530009dd0 con 0x7f9540075270 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9530003e20 con 0x7f9540075270 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a56b0 con 0x7f9540075270 2024-08-20T01:04:29.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.926+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401abd70 con 0x7f9540075270 2024-08-20T01:04:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.927+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540075650 con 0x7f9540075270 2024-08-20T01:04:29.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.929+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f9530004000 con 0x7f9540075270 2024-08-20T01:04:29.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.932+0000 7f953cc00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 0x7f95240b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:29.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.932+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7062+0+0 (secure 0 0 0) 0x7f95300e3380 con 0x7f9540075270 2024-08-20T01:04:29.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.932+0000 7f953f400640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 0x7f95240b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:29.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.933+0000 7f953f400640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 0x7f95240b6180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9540115df0 tx=0x7f952c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:29.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:29.935+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f9530017020 con 0x7f9540075270 2024-08-20T01:04:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:04:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: osdmap e70: 12 total, 11 up, 12 in 2024-08-20T01:04:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='osd.11 ' entity='osd.11' 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:29 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi167", "root=default"]}]': finished 2024-08-20T01:04:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: osdmap e70: 12 total, 11 up, 12 in 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='osd.11 ' entity='osd.11' 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:29.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:29 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:30.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.152+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f95401ac0e0 con 0x7f9540075270 2024-08-20T01:04:30.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.153+0000 7f953cc00640 1 -- 172.21.15.138:0/1228913045 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v71) ==== 74+0+133 (secure 0 0 0) 0x7f95300b30e0 con 0x7f9540075270 2024-08-20T01:04:30.151 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.155+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 msgr2=0x7f95240b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.155+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 0x7f95240b6180 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9540115df0 tx=0x7f952c005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.155+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 msgr2=0x7f954019f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.155+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f954019f310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9530005fd0 tx=0x7f953000dde0 comp rx=0 tx=0).stop 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 shutdown_connections 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95240b3cc0 0x7f95240b6180 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:30.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540113670 0x7f95401a4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540075b90 0x7f954019f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 --2- 172.21.15.138:0/1228913045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540075270 0x7f954019f310 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 >> 172.21.15.138:0/1228913045 conn(0x7f95400fe3c0 msgr2=0x7f95400fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 shutdown_connections 2024-08-20T01:04:30.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:30.156+0000 7f9545a00640 1 -- 172.21.15.138:0/1228913045 wait complete. 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: purged_snaps scrub starts 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: purged_snaps scrub ok 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: Detected new or changed devices on smithi167 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] boot 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:30 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1228913045' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: purged_snaps scrub starts 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: purged_snaps scrub ok 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: Detected new or changed devices on smithi167 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] boot 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:30 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1228913045' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: purged_snaps scrub starts 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: purged_snaps scrub ok 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: Detected new or changed devices on smithi167 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 4272M 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: osd.11 [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] boot 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: osdmap e71: 12 total, 12 up, 12 in 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-08-20T01:04:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:30 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1228913045' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-08-20T01:04:31.109 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":71,"num_osds":12,"num_up_osds":12,"osd_up_since":1724115869,"num_in_osds":12,"osd_in_since":1724115846,"num_remapped_pgs":0} 2024-08-20T01:04:31.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:04:31.474 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:31.785 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:31 smithi138 ceph-mon[18076]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:31.785 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:31 smithi138 ceph-mon[18076]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T01:04:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:31 smithi167 ceph-mon[21965]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:31 smithi167 ceph-mon[21965]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T01:04:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:31 smithi142 ceph-mon[21799]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 310 MiB used, 983 GiB / 983 GiB avail 2024-08-20T01:04:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:31 smithi142 ceph-mon[21799]: osdmap e72: 12 total, 12 up, 12 in 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 -- 172.21.15.138:0/659840669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec104cc0 msgr2=0x7f56ec1050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 --2- 172.21.15.138:0/659840669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec104cc0 0x7f56ec1050a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009900 tx=0x7f56dc02a410 comp rx=0 tx=0).stop 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 -- 172.21.15.138:0/659840669 shutdown_connections 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 --2- 172.21.15.138:0/659840669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec10b050 0x7f56ec111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 --2- 172.21.15.138:0/659840669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 0x7f56ec10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 --2- 172.21.15.138:0/659840669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec104cc0 0x7f56ec1050a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.077 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 -- 172.21.15.138:0/659840669 >> 172.21.15.138:0/659840669 conn(0x7f56ec1006b0 msgr2=0x7f56ec102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.038+0000 7f56f3000640 1 -- 172.21.15.138:0/659840669 shutdown_connections 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.039+0000 7f56f3000640 1 -- 172.21.15.138:0/659840669 wait complete. 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.039+0000 7f56f3000640 1 Processor -- start 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 -- start start 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 0x7f56ec19b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 0x7f56ec1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec114d60 con 0x7f56ec10b050 2024-08-20T01:04:32.078 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec114be0 con 0x7f56ec104cc0 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f3000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec114ee0 con 0x7f56ec10a6d0 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53270/0 (socket says 172.21.15.138:53270) 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56ebe00640 1 -- 172.21.15.138:0/1591453002 learned_addr learned my addr 172.21.15.138:0/1591453002 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56f1000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 0x7f56ec1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.040+0000 7f56eb400640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 0x7f56ec19b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 -- 172.21.15.138:0/1591453002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 msgr2=0x7f56ec19b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 0x7f56ec19b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 -- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 msgr2=0x7f56ec1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 0x7f56ec1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 -- 172.21.15.138:0/1591453002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec1a12b0 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56f1000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 0x7f56ec1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56ebe00640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56dc005ab0 tx=0x7f56dc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc03f070 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec1a1540 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc031b60 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56dc035030 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.041+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a7c60 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.042+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec1050a0 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.045+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f56dc042070 con 0x7f56ec104cc0 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.046+0000 7f56e8c00640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 0x7f56d00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:32.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.046+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f56dc103f80 con 0x7f56ec104cc0 2024-08-20T01:04:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.046+0000 7f56eb400640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 0x7f56d00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.047+0000 7f56eb400640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 0x7f56d00b60e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56ec103fc0 tx=0x7f56d8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:32.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.050+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f56dc0cb2b0 con 0x7f56ec104cc0 2024-08-20T01:04:32.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.237+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56ec10ce90 con 0x7f56ec104cc0 2024-08-20T01:04:32.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.238+0000 7f56e8c00640 1 -- 172.21.15.138:0/1591453002 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18867 (secure 0 0 0) 0x7f56dc0cf800 con 0x7f56ec104cc0 2024-08-20T01:04:32.236 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:32.236 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":72,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:04:30.931862+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:04:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 msgr2=0x7f56d00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 0x7f56d00b60e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f56ec103fc0 tx=0x7f56d8005f90 comp rx=0 tx=0).stop 2024-08-20T01:04:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 msgr2=0x7f56ec19b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:32.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f56dc005ab0 tx=0x7f56dc003c10 comp rx=0 tx=0).stop 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 shutdown_connections 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56d00b3c20 0x7f56d00b60e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f56ec10b050 0x7f56ec1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f56ec10a6d0 0x7f56ec19b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 --2- 172.21.15.138:0/1591453002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56ec104cc0 0x7f56ec19b200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 >> 172.21.15.138:0/1591453002 conn(0x7f56ec1006b0 msgr2=0x7f56ec102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.240+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 shutdown_connections 2024-08-20T01:04:32.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:32.241+0000 7f56f3000640 1 -- 172.21.15.138:0/1591453002 wait complete. 2024-08-20T01:04:32.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:32 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1591453002' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:04:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:32 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1591453002' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:04:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:32 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1591453002' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:04:33.270 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T01:00:30.697585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-08-20T01:04:33.271 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-08-20T01:04:33.634 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:33 smithi138 ceph-mon[18076]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:33 smithi167 ceph-mon[21965]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:33.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:33 smithi142 ceph-mon[21799]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:34.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.240+0000 7f8b64a00640 1 -- 172.21.15.138:0/3955826994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010f470 msgr2=0x7f8b60115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:34.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.240+0000 7f8b64a00640 1 --2- 172.21.15.138:0/3955826994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010f470 0x7f8b60115c70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009ca0 tx=0x7f8b5802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 -- 172.21.15.138:0/3955826994 shutdown_connections 2024-08-20T01:04:34.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 --2- 172.21.15.138:0/3955826994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010f470 0x7f8b60115c70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 --2- 172.21.15.138:0/3955826994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010eaf0 0x7f8b6010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 --2- 172.21.15.138:0/3955826994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 0x7f8b601094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 -- 172.21.15.138:0/3955826994 >> 172.21.15.138:0/3955826994 conn(0x7f8b60078760 msgr2=0x7f8b6007ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:34.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 -- 172.21.15.138:0/3955826994 shutdown_connections 2024-08-20T01:04:34.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.241+0000 7f8b64a00640 1 -- 172.21.15.138:0/3955826994 wait complete. 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 Processor -- start 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 -- start start 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 0x7f8b6019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010f470 0x7f8b601a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b601190f0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60118f70 con 0x7f8b6010f470 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b64a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b60119270 con 0x7f8b601090e0 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35032/0 (socket says 172.21.15.138:35032) 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b5d400640 1 -- 172.21.15.138:0/2276209230 learned_addr learned my addr 172.21.15.138:0/2276209230 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b5e800640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010f470 0x7f8b601a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.242+0000 7f8b5de00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 0x7f8b6019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 -- 172.21.15.138:0/2276209230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 msgr2=0x7f8b6019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 0x7f8b6019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 -- 172.21.15.138:0/2276209230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010f470 msgr2=0x7f8b601a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010f470 0x7f8b601a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 -- 172.21.15.138:0/2276209230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b601a5610 con 0x7f8b6010eaf0 2024-08-20T01:04:34.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b5d400640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ea60 tx=0x7f8b4c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00cde0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b601a5900 con 0x7f8b6010eaf0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b4c0035f0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.243+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c010690 con 0x7f8b6010eaf0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.244+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b601abfc0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.245+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b601094c0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.247+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f8b4c0037a0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.249+0000 7f8b56a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 0x7f8b440b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:34.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.249+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f8b4c0e3d70 con 0x7f8b6010eaf0 2024-08-20T01:04:34.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.249+0000 7f8b5de00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 0x7f8b440b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:34.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.250+0000 7f8b5de00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 0x7f8b440b6160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009bd0 tx=0x7f8b50005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:34.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.251+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8b4c055e20 con 0x7f8b6010eaf0 2024-08-20T01:04:34.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.436+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8b601ac400 con 0x7f8b6010eaf0 2024-08-20T01:04:34.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.437+0000 7f8b56a00640 1 -- 172.21.15.138:0/2276209230 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f8b4c0ab6f0 con 0x7f8b6010eaf0 2024-08-20T01:04:34.435 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 msgr2=0x7f8b440b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 0x7f8b440b6160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8b50009bd0 tx=0x7f8b50005f90 comp rx=0 tx=0).stop 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 msgr2=0x7f8b6019faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00ea60 tx=0x7f8b4c00ef20 comp rx=0 tx=0).stop 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 shutdown_connections 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b440b3ca0 0x7f8b440b6160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b6010f470 0x7f8b601a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b6010eaf0 0x7f8b6019faa0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 --2- 172.21.15.138:0/2276209230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b601090e0 0x7f8b6019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.439+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 >> 172.21.15.138:0/2276209230 conn(0x7f8b60078760 msgr2=0x7f8b6007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.440+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 shutdown_connections 2024-08-20T01:04:34.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:34.440+0000 7f8b64a00640 1 -- 172.21.15.138:0/2276209230 wait complete. 2024-08-20T01:04:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:34 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2276209230' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:04:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:34 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2276209230' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:04:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:34 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2276209230' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.a on smithi138 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.d on smithi138 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.g on smithi138 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.j on smithi138 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.b on smithi142 2024-08-20T01:04:35.363 INFO:tasks.cephadm:Adding mds.e on smithi142 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.h on smithi142 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.k on smithi142 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.c on smithi167 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.f on smithi167 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.i on smithi167 2024-08-20T01:04:35.364 INFO:tasks.cephadm:Adding mds.l on smithi167 2024-08-20T01:04:35.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph orch apply mds all '12;smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l' 2024-08-20T01:04:35.622 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:04:35.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:35 smithi167 ceph-mon[21965]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:35 smithi142 ceph-mon[21799]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:35 smithi138 ceph-mon[18076]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 -- 172.21.15.167:0/120020286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb481073a0 msgr2=0x7fdb481077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/120020286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb481073a0 0x7fdb481077e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009900 tx=0x7fdb3802a410 comp rx=0 tx=0).stop 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 -- 172.21.15.167:0/120020286 shutdown_connections 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/120020286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb48107d20 0x7fdb4810e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/120020286 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb481073a0 0x7fdb481077e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/120020286 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48101990 0x7fdb48101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 -- 172.21.15.167:0/120020286 >> 172.21.15.167:0/120020286 conn(0x7fdb480fd400 msgr2=0x7fdb480ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 -- 172.21.15.167:0/120020286 shutdown_connections 2024-08-20T01:04:36.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.107+0000 7fdb4fde7640 1 -- 172.21.15.167:0/120020286 wait complete. 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 Processor -- start 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 -- start start 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb481073a0 0x7fdb481994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 0x7fdb4819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112b30 con 0x7fdb48107d20 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481129b0 con 0x7fdb481073a0 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4fde7640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112cb0 con 0x7fdb48101990 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52072/0 (socket says 172.21.15.167:52072) 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 -- 172.21.15.167:0/252038813 learned_addr learned my addr 172.21.15.167:0/252038813 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-08-20T01:04:36.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4d382640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb481073a0 0x7fdb481994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 -- 172.21.15.167:0/252038813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb481073a0 msgr2=0x7fdb481994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb481073a0 0x7fdb481994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 -- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 msgr2=0x7fdb4819e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4e384640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 0x7fdb4819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 0x7fdb4819e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4db83640 1 -- 172.21.15.167:0/252038813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819f010 con 0x7fdb48101990 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.108+0000 7fdb4e384640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 0x7fdb4819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb4db83640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400d950 tx=0x7fdb4400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44017070 con 0x7fdb48101990 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4819f300 con 0x7fdb48101990 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb44009c30 con 0x7fdb48101990 2024-08-20T01:04:36.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400f620 con 0x7fdb48101990 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.109+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a59c0 con 0x7fdb48101990 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.110+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb48101d70 con 0x7fdb48101990 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.111+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fdb44010460 con 0x7fdb48101990 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.112+0000 7fdb3effd640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 0x7fdb340b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.112+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fdb440e3570 con 0x7fdb48101990 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.112+0000 7fdb4d382640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 0x7fdb340b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.112+0000 7fdb4d382640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 0x7fdb340b60e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb38031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:36.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.116+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdb440aaf90 con 0x7fdb48101990 2024-08-20T01:04:36.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.327+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}) -- 0x7fdb480633c0 con 0x7fdb340b3c20 2024-08-20T01:04:36.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.337+0000 7fdb3effd640 1 -- 172.21.15.167:0/252038813 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdb480633c0 con 0x7fdb340b3c20 2024-08-20T01:04:36.336 INFO:teuthology.orchestra.run.smithi167.stdout:Scheduled mds.all update... 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 msgr2=0x7fdb340b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 0x7fdb340b60e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb38031400 comp rx=0 tx=0).stop 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 msgr2=0x7fdb48198f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400d950 tx=0x7fdb4400de10 comp rx=0 tx=0).stop 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 shutdown_connections 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdb340b3c20 0x7fdb340b60e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb48107d20 0x7fdb4819e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb481073a0 0x7fdb481994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 --2- 172.21.15.167:0/252038813 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdb48101990 0x7fdb48198f60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:36.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 >> 172.21.15.167:0/252038813 conn(0x7fdb480fd400 msgr2=0x7fdb480ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:36.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 shutdown_connections 2024-08-20T01:04:36.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-08-20T01:04:36.339+0000 7fdb4fde7640 1 -- 172.21.15.167:0/252038813 wait complete. 2024-08-20T01:04:37.074 DEBUG:teuthology.orchestra.run.smithi138:mds.a> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.a.service 2024-08-20T01:04:37.078 DEBUG:teuthology.orchestra.run.smithi138:mds.d> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.d.service 2024-08-20T01:04:37.124 DEBUG:teuthology.orchestra.run.smithi138:mds.g> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.g.service 2024-08-20T01:04:37.170 DEBUG:teuthology.orchestra.run.smithi138:mds.j> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.j.service 2024-08-20T01:04:37.215 DEBUG:teuthology.orchestra.run.smithi142:mds.b> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.b.service 2024-08-20T01:04:37.218 DEBUG:teuthology.orchestra.run.smithi142:mds.e> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.e.service 2024-08-20T01:04:37.261 DEBUG:teuthology.orchestra.run.smithi142:mds.h> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.h.service 2024-08-20T01:04:37.304 DEBUG:teuthology.orchestra.run.smithi142:mds.k> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.k.service 2024-08-20T01:04:37.347 DEBUG:teuthology.orchestra.run.smithi167:mds.c> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.c.service 2024-08-20T01:04:37.349 DEBUG:teuthology.orchestra.run.smithi167:mds.f> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.f.service 2024-08-20T01:04:37.393 DEBUG:teuthology.orchestra.run.smithi167:mds.i> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.i.service 2024-08-20T01:04:37.436 DEBUG:teuthology.orchestra.run.smithi167:mds.l> sudo journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.l.service 2024-08-20T01:04:37.479 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-08-20T01:04:37.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: Saving service mds.all spec with placement smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:37.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:37 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: Saving service mds.all spec with placement smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.528 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:37.529 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:37 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='client.24613 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: Saving service mds.all spec with placement smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:37.595 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:37.596 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:37 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:37.840 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:38.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.424+0000 7f52d1c00640 1 -- 172.21.15.138:0/4002680358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 msgr2=0x7f52cc10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.424+0000 7f52d1c00640 1 --2- 172.21.15.138:0/4002680358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc10aaf0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f52b4009900 tx=0x7f52b402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 -- 172.21.15.138:0/4002680358 shutdown_connections 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 --2- 172.21.15.138:0/4002680358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc10b030 0x7f52cc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 --2- 172.21.15.138:0/4002680358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc10aaf0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 --2- 172.21.15.138:0/4002680358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc104ca0 0x7f52cc105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 -- 172.21.15.138:0/4002680358 >> 172.21.15.138:0/4002680358 conn(0x7f52cc1006f0 msgr2=0x7f52cc102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 -- 172.21.15.138:0/4002680358 shutdown_connections 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.425+0000 7f52d1c00640 1 -- 172.21.15.138:0/4002680358 wait complete. 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 Processor -- start 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 -- start start 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:38.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc10b030 0x7f52cc1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc114cd0 con 0x7f52cc10a6b0 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc114b50 con 0x7f52cc104ca0 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52d1c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc114e50 con 0x7f52cc10b030 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52caa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52caa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37868/0 (socket says 172.21.15.138:37868) 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52caa00640 1 -- 172.21.15.138:0/1535822070 learned_addr learned my addr 172.21.15.138:0/1535822070 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.426+0000 7f52cb400640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc10b030 0x7f52cc1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 -- 172.21.15.138:0/1535822070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc10b030 msgr2=0x7f52cc1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc10b030 0x7f52cc1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 -- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 msgr2=0x7f52cc19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:38.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc19b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 -- 172.21.15.138:0/1535822070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc1a11d0 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52caa00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00e9f0 tx=0x7f52bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00cd80 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52bc0108e0 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1a14c0 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc018980 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.427+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc1a7b80 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.428+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc105080 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.431+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f52bc010430 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.432+0000 7f52c3400640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 0x7f52b00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.432+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f52bc0e3ed0 con 0x7f52cc104ca0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.432+0000 7f52c3e00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 0x7f52b00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:38.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.433+0000 7f52c3e00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 0x7f52b00b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005ab0 tx=0x7f52b4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:38.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.433+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f52bc0ab200 con 0x7f52cc104ca0 2024-08-20T01:04:38.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:38 smithi167 ceph-mon[21965]: Deploying daemon mds.l on smithi167 2024-08-20T01:04:38.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.657+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52cc02e340 con 0x7f52cc104ca0 2024-08-20T01:04:38.656 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:38 smithi138 ceph-mon[18076]: Deploying daemon mds.l on smithi167 2024-08-20T01:04:38.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.658+0000 7f52c3400640 1 -- 172.21.15.138:0/1535822070 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f52bc0af750 con 0x7f52cc104ca0 2024-08-20T01:04:38.657 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:38.657 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"btime":"2024-08-20T00:56:38:482793+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-08-20T01:04:38.658 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 1 2024-08-20T01:04:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 msgr2=0x7f52b00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:38.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 0x7f52b00b60e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005ab0 tx=0x7f52b4005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 msgr2=0x7f52cc19b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00e9f0 tx=0x7f52bc00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 shutdown_connections 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52b00b3c20 0x7f52b00b60e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f52cc10b030 0x7f52cc1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52cc10a6b0 0x7f52cc19b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 --2- 172.21.15.138:0/1535822070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52cc104ca0 0x7f52cc19b120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 >> 172.21.15.138:0/1535822070 conn(0x7f52cc1006f0 msgr2=0x7f52cc102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 shutdown_connections 2024-08-20T01:04:38.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:38.661+0000 7f52d1c00640 1 -- 172.21.15.138:0/1535822070 wait complete. 2024-08-20T01:04:38.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:38 smithi142 ceph-mon[21799]: Deploying daemon mds.l on smithi167 2024-08-20T01:04:39.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:38 smithi167 systemd[1]: Starting Ceph mds.l for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:39.177 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 podman[52647]: 2024-08-20 01:04:39.081401472 +0000 UTC m=+0.015800898 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:39.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:39 smithi167 ceph-mon[21965]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:39.432 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:39 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1535822070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:39.432 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 podman[52647]: 2024-08-20 01:04:39.304561655 +0000 UTC m=+0.238961081 container create 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T01:04:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:39 smithi142 ceph-mon[21799]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:39.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:39 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1535822070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:39.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:39 smithi138 ceph-mon[18076]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:39 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1535822070' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 podman[52647]: 2024-08-20 01:04:39.557722195 +0000 UTC m=+0.492121623 container init 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 podman[52647]: 2024-08-20 01:04:39.560429409 +0000 UTC m=+0.494828836 container start 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 ceph-mds[52666]: -- 172.21.15.167:0/595883109 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b249a59860 con 0x55b249a37400 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l[52662]: starting mds.l at 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 bash[52647]: 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 2024-08-20T01:04:39.963 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:04:39 smithi167 systemd[1]: Started Ceph mds.l for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:40.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6832/2791099826,v1:172.21.15.167:6833/2791099826] up:boot 2024-08-20T01:04:40.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: fsmap 1 up:standby 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: Deploying daemon mds.k on smithi142 2024-08-20T01:04:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:40 smithi142 ceph-mon[21799]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6832/2791099826,v1:172.21.15.167:6833/2791099826] up:boot 2024-08-20T01:04:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: fsmap 1 up:standby 2024-08-20T01:04:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T01:04:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: Deploying daemon mds.k on smithi142 2024-08-20T01:04:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:40 smithi167 ceph-mon[21965]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6832/2791099826,v1:172.21.15.167:6833/2791099826] up:boot 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: fsmap 1 up:standby 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:41.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: Deploying daemon mds.k on smithi142 2024-08-20T01:04:41.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:40 smithi138 ceph-mon[18076]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:41.534 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:41.741 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:41 smithi142 systemd[1]: Starting Ceph mds.k for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:41.907 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:42.138 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:41 smithi142 podman[52048]: 2024-08-20 01:04:41.748684298 +0000 UTC m=+0.015885508 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:42.138 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:41 smithi142 podman[52048]: 2024-08-20 01:04:41.871201534 +0000 UTC m=+0.138402738 container create ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True) 2024-08-20T01:04:42.139 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 podman[52048]: 2024-08-20 01:04:42.140472785 +0000 UTC m=+0.407673990 container init ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-08-20T01:04:42.491 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 podman[52048]: 2024-08-20 01:04:42.143451951 +0000 UTC m=+0.410653158 container start ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:04:42.491 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 ceph-mds[52067]: -- 172.21.15.142:0/2214577179 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561c25e35860 con 0x561c25e13400 2024-08-20T01:04:42.491 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k[52063]: starting mds.k at 2024-08-20T01:04:42.491 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 bash[52048]: ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 2024-08-20T01:04:42.491 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:04:42 smithi142 systemd[1]: Started Ceph mds.k for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:42.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 -- 172.21.15.138:0/1623908422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 msgr2=0x7f6e90102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:42.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1623908422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90102cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009900 tx=0x7f6e8402a410 comp rx=0 tx=0).stop 2024-08-20T01:04:42.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 -- 172.21.15.138:0/1623908422 shutdown_connections 2024-08-20T01:04:42.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1623908422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108ca0 0x7f6e9010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1623908422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108320 0x7f6e90108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1623908422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90102cf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 -- 172.21.15.138:0/1623908422 >> 172.21.15.138:0/1623908422 conn(0x7f6e900fe3c0 msgr2=0x7f6e901007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 -- 172.21.15.138:0/1623908422 shutdown_connections 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.488+0000 7f6e95e00640 1 -- 172.21.15.138:0/1623908422 wait complete. 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 Processor -- start 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 -- start start 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108320 0x7f6e90199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 0x7f6e9019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e901129b0 con 0x7f6e90102910 2024-08-20T01:04:42.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90112830 con 0x7f6e90108320 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.489+0000 7f6e95e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e90112b30 con 0x7f6e90108ca0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e94a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 0x7f6e9019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e94a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44822/0 (socket says 172.21.15.138:44822) 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e94a00640 1 -- 172.21.15.138:0/1331633319 learned_addr learned my addr 172.21.15.138:0/1331633319 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e8fe00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108320 0x7f6e90199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 -- 172.21.15.138:0/1331633319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108320 msgr2=0x7f6e90199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108320 0x7f6e90199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 -- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 msgr2=0x7f6e90198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 -- 172.21.15.138:0/1331633319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9019ee70 con 0x7f6e90108ca0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.490+0000 7f6e95400640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 0x7f6e9019e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800ec50 tx=0x7f6e8800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:42.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.491+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88003b70 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.491+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9019f160 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.491+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e88003d10 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.491+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e88010830 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.491+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e901a5820 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.492+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e90102cf0 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.494+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f6e880109d0 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.495+0000 7f6e8d600640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 0x7f6e780b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.495+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f6e880e3e70 con 0x7f6e90108ca0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.496+0000 7f6e94a00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 0x7f6e780b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.496+0000 7f6e94a00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 0x7f6e780b6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e840096c0 tx=0x7f6e84005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:42.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.497+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6e880ab880 con 0x7f6e90108ca0 2024-08-20T01:04:42.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.731+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e901a5b70 con 0x7f6e90108ca0 2024-08-20T01:04:42.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.731+0000 7f6e8d600640 1 -- 172.21.15.138:0/1331633319 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2266 (secure 0 0 0) 0x7f6e880ab180 con 0x7f6e90108ca0 2024-08-20T01:04:42.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:42.730 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":3,"btime":"2024-08-20T01:04:42:222277+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-20T01:04:42.730 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 3 2024-08-20T01:04:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 msgr2=0x7f6e780b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 0x7f6e780b6180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6e840096c0 tx=0x7f6e84005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 msgr2=0x7f6e9019e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:42.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 0x7f6e9019e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800ec50 tx=0x7f6e8800c6a0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 shutdown_connections 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e780b3cc0 0x7f6e780b6180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e90108ca0 0x7f6e9019e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e90108320 0x7f6e90199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 --2- 172.21.15.138:0/1331633319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e90102910 0x7f6e90198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 >> 172.21.15.138:0/1331633319 conn(0x7f6e900fe3c0 msgr2=0x7f6e901007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 shutdown_connections 2024-08-20T01:04:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:42.734+0000 7f6e95e00640 1 -- 172.21.15.138:0/1331633319 wait complete. 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6832/3285613354,v1:172.21.15.142:6833/3285613354] up:boot 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: fsmap 2 up:standby 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: Deploying daemon mds.j on smithi138 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:43.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:43 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1331633319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:43.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6832/3285613354,v1:172.21.15.142:6833/3285613354] up:boot 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: fsmap 2 up:standby 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: Deploying daemon mds.j on smithi138 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:43.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:43 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1331633319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6832/3285613354,v1:172.21.15.142:6833/3285613354] up:boot 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: fsmap 2 up:standby 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:43.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: Deploying daemon mds.j on smithi138 2024-08-20T01:04:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:43.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:43 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1331633319' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:45.774 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:45 smithi138 ceph-mon[18076]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:45.800 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:45 smithi167 ceph-mon[21965]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:45 smithi142 ceph-mon[21799]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:46.354 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 podman[61933]: 2024-08-20 01:04:46.109015911 +0000 UTC m=+0.134494757 container create 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731) 2024-08-20T01:04:46.354 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 podman[61933]: 2024-08-20 01:04:46.355637578 +0000 UTC m=+0.381116437 container init 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-20T01:04:46.564 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:46.705 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 podman[61933]: 2024-08-20 01:04:46.362965154 +0000 UTC m=+0.388444001 container start 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:04:46.705 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 ceph-mds[61974]: -- 172.21.15.138:0/3865708623 <== mon.0 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558ba20c1860 con 0x558ba209f000 2024-08-20T01:04:46.705 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j[61959]: starting mds.j at 2024-08-20T01:04:46.705 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 bash[61933]: 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 2024-08-20T01:04:46.705 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:04:46 smithi138 systemd[1]: Started Ceph mds.j for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:47.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 -- 172.21.15.138:0/1230237062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 msgr2=0x7f704410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:47.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 --2- 172.21.15.138:0/1230237062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704410a960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7030009900 tx=0x7f703002a410 comp rx=0 tx=0).stop 2024-08-20T01:04:47.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 -- 172.21.15.138:0/1230237062 shutdown_connections 2024-08-20T01:04:47.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 --2- 172.21.15.138:0/1230237062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f704410aea0 0x7f70441116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 --2- 172.21.15.138:0/1230237062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704410a960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 --2- 172.21.15.138:0/1230237062 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7044104b10 0x7f7044104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 -- 172.21.15.138:0/1230237062 >> 172.21.15.138:0/1230237062 conn(0x7f7044100540 msgr2=0x7f7044102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 -- 172.21.15.138:0/1230237062 shutdown_connections 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.193+0000 7f7043e00640 1 -- 172.21.15.138:0/1230237062 wait complete. 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.194+0000 7f7043e00640 1 Processor -- start 2024-08-20T01:04:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.194+0000 7f7043e00640 1 -- start start 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 0x7f704419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f704410aea0 0x7f70441a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7044114bb0 con 0x7f704410a520 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7044114a30 con 0x7f7044104b10 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7044114d30 con 0x7f704410aea0 2024-08-20T01:04:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7042000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7042000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44860/0 (socket says 172.21.15.138:44860) 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7042000640 1 -- 172.21.15.138:0/2357014540 learned_addr learned my addr 172.21.15.138:0/2357014540 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7042a00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 0x7f704419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7043400640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f704410aea0 0x7f70441a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.195+0000 7f7042a00640 1 -- 172.21.15.138:0/2357014540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f704410aea0 msgr2=0x7f70441a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7042a00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f704410aea0 0x7f70441a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7042a00640 1 -- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 msgr2=0x7f704419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7042a00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7042a00640 1 -- 172.21.15.138:0/2357014540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441a1030 con 0x7f7044104b10 2024-08-20T01:04:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7042a00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 0x7f704419af80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f703400e9f0 tx=0x7f703400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400ccc0 con 0x7f7044104b10 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441a1320 con 0x7f7044104b10 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70340054d0 con 0x7f7044104b10 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034010430 con 0x7f7044104b10 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.196+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70441a79e0 con 0x7f7044104b10 2024-08-20T01:04:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.197+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7044104ef0 con 0x7f7044104b10 2024-08-20T01:04:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.200+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f70340046c0 con 0x7f7044104b10 2024-08-20T01:04:47.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.201+0000 7f703b400640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 0x7f70280b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:47.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.201+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f70340e3df0 con 0x7f7044104b10 2024-08-20T01:04:47.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.202+0000 7f7042000640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 0x7f70280b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:47.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.202+0000 7f7042000640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 0x7f70280b6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f704410a0a0 tx=0x7f7030005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.203+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7034055e20 con 0x7f7044104b10 2024-08-20T01:04:47.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.444+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70441a7d30 con 0x7f7044104b10 2024-08-20T01:04:47.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.445+0000 7f703b400640 1 -- 172.21.15.138:0/2357014540 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3094 (secure 0 0 0) 0x7f70340ab770 con 0x7f7044104b10 2024-08-20T01:04:47.443 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:47.443 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":4,"btime":"2024-08-20T01:04:46:510408+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-20T01:04:47.444 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 4 2024-08-20T01:04:47.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 msgr2=0x7f70280b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:47.445 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 0x7f70280b6180 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f704410a0a0 tx=0x7f7030005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 msgr2=0x7f704419af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 0x7f704419af80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f703400e9f0 tx=0x7f703400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 shutdown_connections 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70280b3cc0 0x7f70280b6180 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f704410aea0 0x7f70441a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f704410a520 0x7f704419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 --2- 172.21.15.138:0/2357014540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7044104b10 0x7f704419af80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.447+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 >> 172.21.15.138:0/2357014540 conn(0x7f7044100540 msgr2=0x7f7044102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.448+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 shutdown_connections 2024-08-20T01:04:47.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:47.448+0000 7f7043e00640 1 -- 172.21.15.138:0/2357014540 wait complete. 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6834/1131228617,v1:172.21.15.138:6835/1131228617] up:boot 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: fsmap 3 up:standby 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:47.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: Deploying daemon mds.d on smithi138 2024-08-20T01:04:47.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2357014540' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6834/1131228617,v1:172.21.15.138:6835/1131228617] up:boot 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: fsmap 3 up:standby 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: Deploying daemon mds.d on smithi138 2024-08-20T01:04:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2357014540' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6834/1131228617,v1:172.21.15.138:6835/1131228617] up:boot 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: fsmap 3 up:standby 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: Deploying daemon mds.d on smithi138 2024-08-20T01:04:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2357014540' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:49.659 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:49 smithi138 ceph-mon[18076]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:49 smithi167 ceph-mon[21965]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:49 smithi142 ceph-mon[21799]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:50.244 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:49 smithi138 systemd[1]: Starting Ceph mds.d for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:50.541 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 podman[62325]: 2024-08-20 01:04:50.150169037 +0000 UTC m=+0.027023252 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:50.541 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 podman[62325]: 2024-08-20 01:04:50.318135412 +0000 UTC m=+0.194989605 container create 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-08-20T01:04:50.769 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 podman[62325]: 2024-08-20 01:04:50.570568379 +0000 UTC m=+0.447422573 container init 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 podman[62325]: 2024-08-20 01:04:50.576848716 +0000 UTC m=+0.453702911 container start 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 ceph-mds[62344]: -- 172.21.15.138:0/3346704643 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a74f6e5520 con 0x55a74f6b9400 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 ceph-mds[62344]: -- 172.21.15.138:0/3346704643 <== mon.0 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a74f6db860 con 0x55a74f6b9400 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d[62340]: starting mds.d at 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 bash[62325]: 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd 2024-08-20T01:04:50.825 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:04:50 smithi138 systemd[1]: Started Ceph mds.d for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:51.145 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:51.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 -- 172.21.15.138:0/1895675537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 msgr2=0x7f75ac1158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 --2- 172.21.15.138:0/1895675537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1158e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009900 tx=0x7f75a402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 -- 172.21.15.138:0/1895675537 shutdown_connections 2024-08-20T01:04:51.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 --2- 172.21.15.138:0/1895675537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1158e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 --2- 172.21.15.138:0/1895675537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 --2- 172.21.15.138:0/1895675537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac0777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 -- 172.21.15.138:0/1895675537 >> 172.21.15.138:0/1895675537 conn(0x7f75ac100550 msgr2=0x7f75ac102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 -- 172.21.15.138:0/1895675537 shutdown_connections 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.754+0000 7f75abe00640 1 -- 172.21.15.138:0/1895675537 wait complete. 2024-08-20T01:04:51.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.755+0000 7f75abe00640 1 Processor -- start 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 -- start start 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac118f00 con 0x7f75ac1134f0 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac118d80 con 0x7f75ac077400 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75abe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75ac119080 con 0x7f75ac077d20 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:51.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75aa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53882/0 (socket says 172.21.15.138:53882) 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.756+0000 7f75aaa00640 1 -- 172.21.15.138:0/812813330 learned_addr learned my addr 172.21.15.138:0/812813330 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 -- 172.21.15.138:0/812813330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 msgr2=0x7f75ac19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 -- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 msgr2=0x7f75ac1a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 -- 172.21.15.138:0/812813330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75ac1a53e0 con 0x7f75ac077400 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aa000640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:04:51.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75aaa00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f759c00ebc0 tx=0x7f759c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c010630 con 0x7f75ac077400 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75ac1a56d0 con 0x7f75ac077400 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759c010c80 con 0x7f75ac077400 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c018b60 con 0x7f75ac077400 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.757+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75ac1abd90 con 0x7f75ac077400 2024-08-20T01:04:51.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.758+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75ac0777e0 con 0x7f75ac077400 2024-08-20T01:04:51.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.760+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f759c018d00 con 0x7f75ac077400 2024-08-20T01:04:51.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.762+0000 7f75a3400640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 0x7f75900b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:51.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.762+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f759c0e3ee0 con 0x7f75ac077400 2024-08-20T01:04:51.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.762+0000 7f75aa000640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 0x7f75900b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:51.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.763+0000 7f75aa000640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 0x7f75900b6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75ac079b60 tx=0x7f7598005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:51.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.765+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f759c0ab210 con 0x7f75ac077400 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:51.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.985+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75ac1ac100 con 0x7f75ac077400 2024-08-20T01:04:51.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.986+0000 7f75a3400640 1 -- 172.21.15.138:0/812813330 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3922 (secure 0 0 0) 0x7f759c0af760 con 0x7f75ac077400 2024-08-20T01:04:51.984 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:51.984 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":5,"btime":"2024-08-20T01:04:51:522376+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-08-20T01:04:51.984 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 5 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 msgr2=0x7f75900b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 0x7f75900b6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75ac079b60 tx=0x7f7598005f90 comp rx=0 tx=0).stop 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 msgr2=0x7f75ac19f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f759c00ebc0 tx=0x7f759c00c6a0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 shutdown_connections 2024-08-20T01:04:51.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75900b3cc0 0x7f75900b6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75ac1134f0 0x7f75ac1a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75ac077d20 0x7f75ac19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 --2- 172.21.15.138:0/812813330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75ac077400 0x7f75ac19f330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 >> 172.21.15.138:0/812813330 conn(0x7f75ac100550 msgr2=0x7f75ac102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 shutdown_connections 2024-08-20T01:04:51.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:51.989+0000 7f75abe00640 1 -- 172.21.15.138:0/812813330 wait complete. 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:52.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:52 smithi142 ceph-mon[21799]: Deploying daemon mds.e on smithi142 2024-08-20T01:04:52.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:52 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6836/3862920840,v1:172.21.15.138:6837/3862920840] up:boot 2024-08-20T01:04:52.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:52 smithi142 ceph-mon[21799]: fsmap 4 up:standby 2024-08-20T01:04:52.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:52 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T01:04:52.660 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:52 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/812813330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:52.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:52 smithi138 ceph-mon[18076]: Deploying daemon mds.e on smithi142 2024-08-20T01:04:52.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:52 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6836/3862920840,v1:172.21.15.138:6837/3862920840] up:boot 2024-08-20T01:04:52.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:52 smithi138 ceph-mon[18076]: fsmap 4 up:standby 2024-08-20T01:04:52.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:52 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T01:04:52.715 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:52 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/812813330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:52 smithi167 ceph-mon[21965]: Deploying daemon mds.e on smithi142 2024-08-20T01:04:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:52 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6836/3862920840,v1:172.21.15.138:6837/3862920840] up:boot 2024-08-20T01:04:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:52 smithi167 ceph-mon[21965]: fsmap 4 up:standby 2024-08-20T01:04:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:52 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-08-20T01:04:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:52 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/812813330' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:52.991 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:52 smithi142 systemd[1]: Starting Ceph mds.e for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:53.294 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 podman[52365]: 2024-08-20 01:04:52.944555569 +0000 UTC m=+0.015966223 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:53.294 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 podman[52365]: 2024-08-20 01:04:53.05178819 +0000 UTC m=+0.123198833 container create ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:04:53.294 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 podman[52365]: 2024-08-20 01:04:53.295903051 +0000 UTC m=+0.367313709 container init ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T01:04:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 podman[52365]: 2024-08-20 01:04:53.298908696 +0000 UTC m=+0.370319348 container start ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-mds[52383]: -- 172.21.15.142:0/173833827 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5645c6083860 con 0x5645c6061400 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: starting mds.e at 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 bash[52365]: ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 2024-08-20T01:04:53.742 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:04:53 smithi142 systemd[1]: Started Ceph mds.e for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:53 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:53.881 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:53 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:54.754 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:54.809 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:54 smithi138 ceph-mon[18076]: Deploying daemon mds.f on smithi167 2024-08-20T01:04:54.810 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:04:54.810 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:54 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6834/2224495590,v1:172.21.15.142:6835/2224495590] up:boot 2024-08-20T01:04:54.810 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:54 smithi138 ceph-mon[18076]: fsmap 5 up:standby 2024-08-20T01:04:54.917 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:54 smithi167 ceph-mon[21965]: Deploying daemon mds.f on smithi167 2024-08-20T01:04:54.918 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:04:54.918 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:54 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6834/2224495590,v1:172.21.15.142:6835/2224495590] up:boot 2024-08-20T01:04:54.918 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:54 smithi167 ceph-mon[21965]: fsmap 5 up:standby 2024-08-20T01:04:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:54 smithi142 ceph-mon[21799]: Deploying daemon mds.f on smithi167 2024-08-20T01:04:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:04:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:54 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6834/2224495590,v1:172.21.15.142:6835/2224495590] up:boot 2024-08-20T01:04:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:54 smithi142 ceph-mon[21799]: fsmap 5 up:standby 2024-08-20T01:04:55.133 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:55.548 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 systemd[1]: Starting Ceph mds.f for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 -- 172.21.15.138:0/1958416027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 msgr2=0x7fa764105070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 --2- 172.21.15.138:0/1958416027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa764105070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa754009900 tx=0x7fa75402a410 comp rx=0 tx=0).stop 2024-08-20T01:04:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 -- 172.21.15.138:0/1958416027 shutdown_connections 2024-08-20T01:04:55.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 --2- 172.21.15.138:0/1958416027 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa764111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 --2- 172.21.15.138:0/1958416027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 0x7fa76410aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 --2- 172.21.15.138:0/1958416027 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa764105070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 -- 172.21.15.138:0/1958416027 >> 172.21.15.138:0/1958416027 conn(0x7fa7641006c0 msgr2=0x7fa764102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:55.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 -- 172.21.15.138:0/1958416027 shutdown_connections 2024-08-20T01:04:55.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.692+0000 7fa76b800640 1 -- 172.21.15.138:0/1958416027 wait complete. 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 Processor -- start 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 -- start start 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa76419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 0x7fa76419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa7641a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:55.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa764114d30 con 0x7fa764104c90 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa764114bb0 con 0x7fa76410a6a0 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa76b800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa764114eb0 con 0x7fa76410b020 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.693+0000 7fa768e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa76419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa7641a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa768e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa76419b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49048/0 (socket says 172.21.15.138:49048) 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa768e00640 1 -- 172.21.15.138:0/2781475116 learned_addr learned my addr 172.21.15.138:0/2781475116 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa75be00640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 0x7fa76419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 -- 172.21.15.138:0/2781475116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 msgr2=0x7fa76419b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 0x7fa76419b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 -- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 msgr2=0x7fa76419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa76419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.693 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 -- 172.21.15.138:0/2781475116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7641a1240 con 0x7fa76410b020 2024-08-20T01:04:55.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa769800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa7641a0a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009fd0 tx=0x7fa75c00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:55.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00ce20 con 0x7fa76410b020 2024-08-20T01:04:55.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa75c005020 con 0x7fa76410b020 2024-08-20T01:04:55.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.694+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7641a1530 con 0x7fa76410b020 2024-08-20T01:04:55.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.695+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c010690 con 0x7fa76410b020 2024-08-20T01:04:55.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.695+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7641a7bf0 con 0x7fa76410b020 2024-08-20T01:04:55.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.696+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa764105070 con 0x7fa76410b020 2024-08-20T01:04:55.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.698+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fa75c0054d0 con 0x7fa76410b020 2024-08-20T01:04:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.699+0000 7fa762000640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 0x7fa7480b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.699+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa75c0e4090 con 0x7fa76410b020 2024-08-20T01:04:55.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.699+0000 7fa768e00640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 0x7fa7480b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:55.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.700+0000 7fa768e00640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 0x7fa7480b60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7540096c0 tx=0x7fa754005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:55.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.703+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa75c0aba10 con 0x7fa76410b020 2024-08-20T01:04:55.894 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:55 smithi167 ceph-mon[21965]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:55.895 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 podman[52951]: 2024-08-20 01:04:55.525695456 +0000 UTC m=+0.016015546 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:55.895 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 podman[52951]: 2024-08-20 01:04:55.652607727 +0000 UTC m=+0.142927800 container create faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS) 2024-08-20T01:04:55.895 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 podman[52951]: 2024-08-20 01:04:55.897171872 +0000 UTC m=+0.387491945 container init faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-08-20T01:04:55.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.921+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa76419c2c0 con 0x7fa76410b020 2024-08-20T01:04:55.919 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:55 smithi138 ceph-mon[18076]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:55.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.921+0000 7fa762000640 1 -- 172.21.15.138:0/2781475116 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4750 (secure 0 0 0) 0x7fa75c0ab310 con 0x7fa76410b020 2024-08-20T01:04:55.920 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:55.920 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":6,"btime":"2024-08-20T01:04:53:534671+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-08-20T01:04:55.920 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 6 2024-08-20T01:04:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 msgr2=0x7fa7480b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 0x7fa7480b60e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa7540096c0 tx=0x7fa754005a20 comp rx=0 tx=0).stop 2024-08-20T01:04:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 msgr2=0x7fa7641a0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa7641a0a70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009fd0 tx=0x7fa75c00ef10 comp rx=0 tx=0).stop 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 shutdown_connections 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa7480b3c20 0x7fa7480b60e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa76410b020 0x7fa7641a0a70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa76410a6a0 0x7fa76419b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 --2- 172.21.15.138:0/2781475116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa764104c90 0x7fa76419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 >> 172.21.15.138:0/2781475116 conn(0x7fa7641006c0 msgr2=0x7fa764101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 shutdown_connections 2024-08-20T01:04:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:55.924+0000 7fa76b800640 1 -- 172.21.15.138:0/2781475116 wait complete. 2024-08-20T01:04:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:55 smithi142 ceph-mon[21799]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:56.213 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 podman[52951]: 2024-08-20 01:04:55.900181792 +0000 UTC m=+0.390501866 container start faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T01:04:56.213 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 ceph-mds[52970]: -- 172.21.15.167:0/3353505262 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1b60f3860 con 0x55f1b60d1400 2024-08-20T01:04:56.213 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f[52966]: starting mds.f at 2024-08-20T01:04:56.213 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:56 smithi167 bash[52951]: faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 2024-08-20T01:04:56.213 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:04:56 smithi167 systemd[1]: Started Ceph mds.f for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:56.736 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2781475116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:56.737 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2781475116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:56.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:56.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:56.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:57.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2781475116' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:57.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:57 smithi167 ceph-mon[21965]: Deploying daemon mds.c on smithi167 2024-08-20T01:04:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:57 smithi167 ceph-mon[21965]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:57 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6834/902364258,v1:172.21.15.167:6835/902364258] up:boot 2024-08-20T01:04:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:57 smithi167 ceph-mon[21965]: fsmap 6 up:standby 2024-08-20T01:04:57.804 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:57 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T01:04:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:57 smithi142 ceph-mon[21799]: Deploying daemon mds.c on smithi167 2024-08-20T01:04:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:57 smithi142 ceph-mon[21799]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:57 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6834/902364258,v1:172.21.15.167:6835/902364258] up:boot 2024-08-20T01:04:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:57 smithi142 ceph-mon[21799]: fsmap 6 up:standby 2024-08-20T01:04:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:57 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T01:04:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:57 smithi138 ceph-mon[18076]: Deploying daemon mds.c on smithi167 2024-08-20T01:04:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:57 smithi138 ceph-mon[18076]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:57 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6834/902364258,v1:172.21.15.167:6835/902364258] up:boot 2024-08-20T01:04:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:57 smithi138 ceph-mon[18076]: fsmap 6 up:standby 2024-08-20T01:04:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:57 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-08-20T01:04:58.200 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:57 smithi167 systemd[1]: Starting Ceph mds.c for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:04:58.463 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 podman[53249]: 2024-08-20 01:04:58.105609445 +0000 UTC m=+0.016087227 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:04:58.463 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 podman[53249]: 2024-08-20 01:04:58.238972378 +0000 UTC m=+0.149450161 container create bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True) 2024-08-20T01:04:58.639 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 podman[53249]: 2024-08-20 01:04:58.487067434 +0000 UTC m=+0.397545216 container init bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 podman[53249]: 2024-08-20 01:04:58.490010769 +0000 UTC m=+0.400488554 container start bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True) 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 ceph-mds[53268]: -- 172.21.15.167:0/2203546453 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556ee51c9860 con 0x556ee51a6800 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: starting mds.c at 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 bash[53249]: bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 2024-08-20T01:04:58.963 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:04:58 smithi167 systemd[1]: Started Ceph mds.c for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:04:59.016 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:04:59.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.583+0000 7f1967200640 1 -- 172.21.15.138:0/4174333512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196010ce00 msgr2=0x7f196010f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:59.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.583+0000 7f1967200640 1 --2- 172.21.15.138:0/4174333512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196010ce00 0x7f196010f260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1958009be0 tx=0x7f195802a6c0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 -- 172.21.15.138:0/4174333512 shutdown_connections 2024-08-20T01:04:59.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 --2- 172.21.15.138:0/4174333512 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f196010ce00 0x7f196010f260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 --2- 172.21.15.138:0/4174333512 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f196010c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 --2- 172.21.15.138:0/4174333512 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f19601039a0 0x7f1960103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 -- 172.21.15.138:0/4174333512 >> 172.21.15.138:0/4174333512 conn(0x7f19600fd400 msgr2=0x7f19600ff820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:59.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 -- 172.21.15.138:0/4174333512 shutdown_connections 2024-08-20T01:04:59.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.584+0000 7f1967200640 1 -- 172.21.15.138:0/4174333512 wait complete. 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.585+0000 7f1967200640 1 Processor -- start 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.585+0000 7f1967200640 1 -- start start 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.585+0000 7f1967200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 0x7f1960101650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1967200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1967200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f196010ce00 0x7f19601020d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1967200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1960112a70 con 0x7f19601042c0 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1967200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f19601128f0 con 0x7f19601039a0 2024-08-20T01:04:59.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1967200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1960112bf0 con 0x7f196010ce00 2024-08-20T01:04:59.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:59.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33262/0 (socket says 172.21.15.138:33262) 2024-08-20T01:04:59.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195f400640 1 -- 172.21.15.138:0/4236182201 learned_addr learned my addr 172.21.15.138:0/4236182201 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 0x7f1960101650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f1965200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f196010ce00 0x7f19601020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 -- 172.21.15.138:0/4236182201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f196010ce00 msgr2=0x7f19601020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f196010ce00 0x7f19601020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 -- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 msgr2=0x7f1960101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195fe00640 1 -- 172.21.15.138:0/4236182201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19601a7380 con 0x7f19601039a0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.586+0000 7f195f400640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f195fe00640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 0x7f1960101650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9f0 tx=0x7f195000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195000ccc0 con 0x7f19601039a0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1950005020 con 0x7f19601039a0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19601a7670 con 0x7f19601039a0 2024-08-20T01:04:59.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950004a00 con 0x7f19601039a0 2024-08-20T01:04:59.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.587+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19601a7b80 con 0x7f19601039a0 2024-08-20T01:04:59.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.588+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1960103d80 con 0x7f19601039a0 2024-08-20T01:04:59.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.590+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f1950004ba0 con 0x7f19601039a0 2024-08-20T01:04:59.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.591+0000 7f195cc00640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 0x7f19440b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:04:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.591+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f19500e3890 con 0x7f19601039a0 2024-08-20T01:04:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.591+0000 7f195f400640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 0x7f19440b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:04:59.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.592+0000 7f195f400640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 0x7f19440b6160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f194c0096c0 tx=0x7f194c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:04:59.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.595+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f19500ab210 con 0x7f19601039a0 2024-08-20T01:04:59.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:59.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6836/2270224611,v1:172.21.15.167:6837/2270224611] up:boot 2024-08-20T01:04:59.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: fsmap 7 up:standby 2024-08-20T01:04:59.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:59.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:04:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6836/2270224611,v1:172.21.15.167:6837/2270224611] up:boot 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: fsmap 7 up:standby 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:59.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:04:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:04:59.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.818+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19601a7f50 con 0x7f19601039a0 2024-08-20T01:04:59.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.819+0000 7f195cc00640 1 -- 172.21.15.138:0/4236182201 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6403 (secure 0 0 0) 0x7f19500aab10 con 0x7f19601039a0 2024-08-20T01:04:59.819 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:04:59.819 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":8,"btime":"2024-08-20T01:04:58:558228+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-08-20T01:04:59.819 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 8 2024-08-20T01:04:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 msgr2=0x7f19440b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 0x7f19440b6160 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f194c0096c0 tx=0x7f194c009260 comp rx=0 tx=0).stop 2024-08-20T01:04:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 msgr2=0x7f1960101650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:04:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 0x7f1960101650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f195000e9f0 tx=0x7f195000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 shutdown_connections 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19440b3ca0 0x7f19440b6160 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f196010ce00 0x7f19601020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19601042c0 0x7f1960101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 --2- 172.21.15.138:0/4236182201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19601039a0 0x7f1960101650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 >> 172.21.15.138:0/4236182201 conn(0x7f19600fd400 msgr2=0x7f19600fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 shutdown_connections 2024-08-20T01:04:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:04:59.822+0000 7f1967200640 1 -- 172.21.15.138:0/4236182201 wait complete. 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6836/2270224611,v1:172.21.15.167:6837/2270224611] up:boot 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: fsmap 7 up:standby 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:04:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:04:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:04:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:04:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:00 smithi142 ceph-mon[21799]: Deploying daemon mds.h on smithi142 2024-08-20T01:05:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:00 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/4236182201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:00.741 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:00 smithi142 systemd[1]: Starting Ceph mds.h for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:05:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:00 smithi167 ceph-mon[21965]: Deploying daemon mds.h on smithi142 2024-08-20T01:05:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:00 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/4236182201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:01.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:00 smithi138 ceph-mon[18076]: Deploying daemon mds.h on smithi142 2024-08-20T01:05:01.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:00 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/4236182201' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:01.099 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:00 smithi142 podman[52665]: 2024-08-20 01:05:00.732666949 +0000 UTC m=+0.016265412 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:05:01.099 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:00 smithi142 podman[52665]: 2024-08-20 01:05:00.848370502 +0000 UTC m=+0.131968941 container create 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:05:01.099 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 podman[52665]: 2024-08-20 01:05:01.101362054 +0000 UTC m=+0.384960493 container init 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD) 2024-08-20T01:05:01.491 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 podman[52665]: 2024-08-20 01:05:01.104437933 +0000 UTC m=+0.388036373 container start 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-20T01:05:01.491 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mds[52684]: -- 172.21.15.142:0/500883120 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ad339f860 con 0x557ad337c800 2024-08-20T01:05:01.491 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: starting mds.h at 2024-08-20T01:05:01.491 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 bash[52665]: 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 2024-08-20T01:05:01.491 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:05:01 smithi142 systemd[1]: Started Ceph mds.h for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:05:01.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:01.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.829 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:01.830 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:02.768 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:02.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:02 smithi138 ceph-mon[18076]: Deploying daemon mds.i on smithi167 2024-08-20T01:05:02.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:02 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:boot 2024-08-20T01:05:02.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:02 smithi138 ceph-mon[18076]: fsmap 8 up:standby 2024-08-20T01:05:02.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:05:02.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:02 smithi167 ceph-mon[21965]: Deploying daemon mds.i on smithi167 2024-08-20T01:05:02.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:02 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:boot 2024-08-20T01:05:02.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:02 smithi167 ceph-mon[21965]: fsmap 8 up:standby 2024-08-20T01:05:02.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:05:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:02 smithi142 ceph-mon[21799]: Deploying daemon mds.i on smithi167 2024-08-20T01:05:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:02 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:boot 2024-08-20T01:05:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:02 smithi142 ceph-mon[21799]: fsmap 8 up:standby 2024-08-20T01:05:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:05:03.148 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:03.417 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 systemd[1]: Starting Ceph mds.i for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:05:03.700 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:03 smithi167 ceph-mon[21965]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:03.700 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 podman[53550]: 2024-08-20 01:05:03.321365428 +0000 UTC m=+0.015714346 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:05:03.700 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 podman[53550]: 2024-08-20 01:05:03.457857982 +0000 UTC m=+0.152206886 container create b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-20T01:05:03.765 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:03 smithi138 ceph-mon[18076]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 -- 172.21.15.138:0/1751495215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d010b040 msgr2=0x7fc6d0111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 --2- 172.21.15.138:0/1751495215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d010b040 0x7fc6d0111840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc6c8009ca0 tx=0x7fc6c802a8b0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 -- 172.21.15.138:0/1751495215 shutdown_connections 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 --2- 172.21.15.138:0/1751495215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d010b040 0x7fc6d0111840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 --2- 172.21.15.138:0/1751495215 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010a6c0 0x7fc6d010ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 --2- 172.21.15.138:0/1751495215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d0104cb0 0x7fc6d0105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 -- 172.21.15.138:0/1751495215 >> 172.21.15.138:0/1751495215 conn(0x7fc6d0100720 msgr2=0x7fc6d0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 -- 172.21.15.138:0/1751495215 shutdown_connections 2024-08-20T01:05:03.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.765+0000 7fc6d6400640 1 -- 172.21.15.138:0/1751495215 wait complete. 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 Processor -- start 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 -- start start 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 0x7fc6d019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010b040 0x7fc6d01a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0114d50 con 0x7fc6d0104cb0 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0114bd0 con 0x7fc6d010a6c0 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.766+0000 7fc6d6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0114ed0 con 0x7fc6d010b040 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33272/0 (socket says 172.21.15.138:33272) 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 -- 172.21.15.138:0/4217436143 learned_addr learned my addr 172.21.15.138:0/4217436143 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:03.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cfe00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010b040 0x7fc6d01a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6c7e00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 0x7fc6d019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 -- 172.21.15.138:0/4217436143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010b040 msgr2=0x7fc6d01a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010b040 0x7fc6d01a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 -- 172.21.15.138:0/4217436143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 msgr2=0x7fc6d019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 0x7fc6d019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 -- 172.21.15.138:0/4217436143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d01a11d0 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6c7e00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 0x7fc6d019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.767+0000 7fc6cf400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e9f0 tx=0x7fc6c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.768+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000cd80 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.768+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c00108e0 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.768+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d01a14c0 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.768+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0018980 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.768+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a7b80 con 0x7fc6d0104cb0 2024-08-20T01:05:03.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.769+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d0105090 con 0x7fc6d0104cb0 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.771+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fc6c0010430 con 0x7fc6d0104cb0 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.773+0000 7fc6ccc00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 0x7fc6b40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.774+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fc6c00e3c30 con 0x7fc6d0104cb0 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.774+0000 7fc6c7e00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 0x7fc6b40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.774+0000 7fc6c7e00640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 0x7fc6b40b60e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6d010a110 tx=0x7fc6b8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:03.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:03.777+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc6c00aaf60 con 0x7fc6d0104cb0 2024-08-20T01:05:03.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 podman[53550]: 2024-08-20 01:05:03.702505213 +0000 UTC m=+0.396854119 container init b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T01:05:03.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 podman[53550]: 2024-08-20 01:05:03.705383956 +0000 UTC m=+0.399732867 container start b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T01:05:03.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 ceph-mds[53569]: -- 172.21.15.167:0/1883386299 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555bda02d860 con 0x555bda00b000 2024-08-20T01:05:03.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: starting mds.i at 2024-08-20T01:05:03.963 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 bash[53550]: b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b 2024-08-20T01:05:03.964 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:05:03 smithi167 systemd[1]: Started Ceph mds.i for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:05:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:03 smithi142 ceph-mon[21799]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:04.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.020+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6d01a7f40 con 0x7fc6d0104cb0 2024-08-20T01:05:04.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.020+0000 7fc6ccc00640 1 -- 172.21.15.138:0/4217436143 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7231 (secure 0 0 0) 0x7fc6c00af4b0 con 0x7fc6d0104cb0 2024-08-20T01:05:04.020 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:04.020 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":9,"btime":"2024-08-20T01:05:01:572741+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-08-20T01:05:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 9 2024-08-20T01:05:04.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 msgr2=0x7fc6b40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 0x7fc6b40b60e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc6d010a110 tx=0x7fc6b8005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 msgr2=0x7fc6d019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6c000e9f0 tx=0x7fc6c000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 shutdown_connections 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6b40b3c20 0x7fc6b40b60e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6d010b040 0x7fc6d01a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6d010a6c0 0x7fc6d019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 --2- 172.21.15.138:0/4217436143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6d0104cb0 0x7fc6d019b120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 >> 172.21.15.138:0/4217436143 conn(0x7fc6d0100720 msgr2=0x7fc6d0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 shutdown_connections 2024-08-20T01:05:04.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:04.023+0000 7fc6d6400640 1 -- 172.21.15.138:0/4217436143 wait complete. 2024-08-20T01:05:05.144 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: Deploying daemon mds.g on smithi138 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/4217436143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6838/2113916191,v1:172.21.15.167:6839/2113916191] up:boot 2024-08-20T01:05:05.145 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: fsmap 9 up:standby 2024-08-20T01:05:05.146 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:04 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: Deploying daemon mds.g on smithi138 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/4217436143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6838/2113916191,v1:172.21.15.167:6839/2113916191] up:boot 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: fsmap 9 up:standby 2024-08-20T01:05:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:04 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:05:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: Deploying daemon mds.g on smithi138 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/4217436143' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6838/2113916191,v1:172.21.15.167:6839/2113916191] up:boot 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: fsmap 9 up:standby 2024-08-20T01:05:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:04 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:05:06.851 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:07.476 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 podman[63106]: 2024-08-20 01:05:07.241372158 +0000 UTC m=+0.183723338 container create 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-20T01:05:07.476 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 podman[63106]: 2024-08-20 01:05:07.47838418 +0000 UTC m=+0.420735361 container init 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T01:05:07.666 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:07.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:07 smithi138 ceph-mon[18076]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:07.791 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 podman[63106]: 2024-08-20 01:05:07.486922704 +0000 UTC m=+0.429273886 container start 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:05:07.791 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 ceph-mds[63150]: -- 172.21.15.138:0/3720460143 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b09adfd860 con 0x55b09addb000 2024-08-20T01:05:07.791 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: starting mds.g at 2024-08-20T01:05:07.791 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 bash[63106]: 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 2024-08-20T01:05:07.791 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:05:07 smithi138 systemd[1]: Started Ceph mds.g for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:05:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:07 smithi167 ceph-mon[21965]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:07 smithi142 ceph-mon[21799]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:08.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 -- 172.21.15.138:0/412136684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 msgr2=0x7fe6f8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:08.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 --2- 172.21.15.138:0/412136684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8108900 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4009900 tx=0x7fe6e402a410 comp rx=0 tx=0).stop 2024-08-20T01:05:08.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 -- 172.21.15.138:0/412136684 shutdown_connections 2024-08-20T01:05:08.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 --2- 172.21.15.138:0/412136684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8108e40 0x7fe6f810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 --2- 172.21.15.138:0/412136684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8108900 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 --2- 172.21.15.138:0/412136684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8102ab0 0x7fe6f8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 -- 172.21.15.138:0/412136684 >> 172.21.15.138:0/412136684 conn(0x7fe6f80fe500 msgr2=0x7fe6f8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 -- 172.21.15.138:0/412136684 shutdown_connections 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.252+0000 7fe6fda00640 1 -- 172.21.15.138:0/412136684 wait complete. 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 Processor -- start 2024-08-20T01:05:08.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 -- start start 2024-08-20T01:05:08.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 0x7fe6f80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 0x7fe6f8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8112ae0 con 0x7fe6f81084c0 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8112960 con 0x7fe6f8108e40 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.253+0000 7fe6fda00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f8112c60 con 0x7fe6f8102ab0 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 0x7fe6f80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40200/0 (socket says 172.21.15.138:40200) 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 0x7fe6f8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f6000640 1 -- 172.21.15.138:0/2842705905 learned_addr learned my addr 172.21.15.138:0/2842705905 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 -- 172.21.15.138:0/2842705905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 msgr2=0x7fe6f80751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:08.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 0x7fe6f80751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 -- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 msgr2=0x7fe6f8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f7400640 1 -- 172.21.15.138:0/2842705905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f81a73b0 con 0x7fe6f8108e40 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.254+0000 7fe6f6a00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 0x7fe6f80751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6f7400640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 0x7fe6f8075c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000ec50 tx=0x7fe6f000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0003b70 con 0x7fe6f8108e40 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f81a76a0 con 0x7fe6f8108e40 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6f0003d10 con 0x7fe6f8108e40 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0010830 con 0x7fe6f8108e40 2024-08-20T01:05:08.256 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.255+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f81a7be0 con 0x7fe6f8108e40 2024-08-20T01:05:08.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.256+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f8102e90 con 0x7fe6f8108e40 2024-08-20T01:05:08.257 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.258+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fe6f00109d0 con 0x7fe6f8108e40 2024-08-20T01:05:08.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.260+0000 7fe6ef400640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 0x7fe6dc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:08.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.260+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fe6f00e3e90 con 0x7fe6f8108e40 2024-08-20T01:05:08.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.260+0000 7fe6f6a00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 0x7fe6dc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:08.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.260+0000 7fe6f6a00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 0x7fe6dc0b6160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009fc0 tx=0x7fe6e8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:08.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.261+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe6f00ab8a0 con 0x7fe6f8108e40 2024-08-20T01:05:08.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.526+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6f80764d0 con 0x7fe6f8108e40 2024-08-20T01:05:08.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.527+0000 7fe6ef400640 1 -- 172.21.15.138:0/2842705905 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8890 (secure 0 0 0) 0x7fe6f00ab1a0 con 0x7fe6f8108e40 2024-08-20T01:05:08.526 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:08.526 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":11,"btime":"2024-08-20T01:05:08:514102+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24706,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-08-20T01:05:08.526 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 11 2024-08-20T01:05:08.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.529+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 msgr2=0x7fe6dc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:08.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.529+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 0x7fe6dc0b6160 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009fc0 tx=0x7fe6e8005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.529+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 msgr2=0x7fe6f8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.529+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 0x7fe6f8075c50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6f000ec50 tx=0x7fe6f000c6a0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 shutdown_connections 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6dc0b3ca0 0x7fe6dc0b6160 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f8108e40 0x7fe6f8075c50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6f81084c0 0x7fe6f8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 --2- 172.21.15.138:0/2842705905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6f8102ab0 0x7fe6f80751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 >> 172.21.15.138:0/2842705905 conn(0x7fe6f80fe500 msgr2=0x7fe6f8105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 shutdown_connections 2024-08-20T01:05:08.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:08.530+0000 7fe6fda00640 1 -- 172.21.15.138:0/2842705905 wait complete. 2024-08-20T01:05:08.777 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.777 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.777 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.777 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: Deploying daemon mds.a on smithi138 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:boot 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: fsmap 10 up:standby 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:05:08.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:08 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2842705905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: Deploying daemon mds.a on smithi138 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:boot 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: fsmap 10 up:standby 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:05:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:08 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2842705905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: Deploying daemon mds.a on smithi138 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:boot 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: fsmap 10 up:standby 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:05:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:08 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2842705905' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:11.016 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:10 smithi138 systemd[1]: Starting Ceph mds.a for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:05:11.291 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 podman[63502]: 2024-08-20 01:05:10.921931731 +0000 UTC m=+0.027049540 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:05:11.291 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 podman[63502]: 2024-08-20 01:05:11.092790767 +0000 UTC m=+0.197908566 container create c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:11 smithi138 ceph-mon[18076]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 podman[63502]: 2024-08-20 01:05:11.474834131 +0000 UTC m=+0.579951941 container init c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 podman[63502]: 2024-08-20 01:05:11.481240413 +0000 UTC m=+0.586358212 container start c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 ceph-mds[63522]: -- 172.21.15.138:0/149964765 <== mon.0 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559241083c20 con 0x559241061000 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a[63517]: starting mds.a at 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 bash[63502]: c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf 2024-08-20T01:05:11.791 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:05:11 smithi138 systemd[1]: Started Ceph mds.a for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:05:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:11 smithi167 ceph-mon[21965]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:11.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:11 smithi142 ceph-mon[21799]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:12.541 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: Deploying daemon mds.b on smithi142 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6840/3860533151,v1:172.21.15.138:6841/3860533151] up:boot 2024-08-20T01:05:12.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:12 smithi142 ceph-mon[21799]: fsmap 11 up:standby 2024-08-20T01:05:12.920 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: Deploying daemon mds.b on smithi142 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T01:05:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6840/3860533151,v1:172.21.15.138:6841/3860533151] up:boot 2024-08-20T01:05:12.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:12 smithi167 ceph-mon[21965]: fsmap 11 up:standby 2024-08-20T01:05:13.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: Deploying daemon mds.b on smithi142 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6840/3860533151,v1:172.21.15.138:6841/3860533151] up:boot 2024-08-20T01:05:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:12 smithi138 ceph-mon[18076]: fsmap 11 up:standby 2024-08-20T01:05:13.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 -- 172.21.15.138:0/2082880831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d410f470 msgr2=0x7fa2d4115c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:13.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 --2- 172.21.15.138:0/2082880831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d410f470 0x7fa2d4115c70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009ca0 tx=0x7fa2cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 -- 172.21.15.138:0/2082880831 shutdown_connections 2024-08-20T01:05:13.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 --2- 172.21.15.138:0/2082880831 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d410f470 0x7fa2d4115c70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 --2- 172.21.15.138:0/2082880831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410eaf0 0x7fa2d410ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 --2- 172.21.15.138:0/2082880831 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d41090e0 0x7fa2d41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 -- 172.21.15.138:0/2082880831 >> 172.21.15.138:0/2082880831 conn(0x7fa2d4078760 msgr2=0x7fa2d407ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 -- 172.21.15.138:0/2082880831 shutdown_connections 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.498+0000 7fa2db800640 1 -- 172.21.15.138:0/2082880831 wait complete. 2024-08-20T01:05:13.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.499+0000 7fa2db800640 1 Processor -- start 2024-08-20T01:05:13.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.499+0000 7fa2db800640 1 -- start start 2024-08-20T01:05:13.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d410eaf0 0x7fa2d419faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410f470 0x7fa2d41a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d41190f0 con 0x7fa2d41090e0 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4118f70 con 0x7fa2d410f470 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2db800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d4119270 con 0x7fa2d410eaf0 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410f470 0x7fa2d41a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40234/0 (socket says 172.21.15.138:40234) 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 -- 172.21.15.138:0/145027572 learned_addr learned my addr 172.21.15.138:0/145027572 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d3e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d410eaf0 0x7fa2d419faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 -- 172.21.15.138:0/145027572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d410eaf0 msgr2=0x7fa2d419faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:13.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d410eaf0 0x7fa2d419faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 -- 172.21.15.138:0/145027572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410f470 msgr2=0x7fa2d41a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.500+0000 7fa2d8e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410f470 0x7fa2d41a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2d8e00640 1 -- 172.21.15.138:0/145027572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d41a5610 con 0x7fa2d41090e0 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2d8e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c800cd80 con 0x7fa2d41090e0 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2c80054d0 con 0x7fa2d41090e0 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d41a5900 con 0x7fa2d41090e0 2024-08-20T01:05:13.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8010690 con 0x7fa2d41090e0 2024-08-20T01:05:13.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.501+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41abfc0 con 0x7fa2d41090e0 2024-08-20T01:05:13.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.502+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d41094c0 con 0x7fa2d41090e0 2024-08-20T01:05:13.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.505+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fa2c8004e30 con 0x7fa2d41090e0 2024-08-20T01:05:13.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.507+0000 7fa2d1600640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 0x7fa2bc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:13.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.507+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa2c80e3ec0 con 0x7fa2d41090e0 2024-08-20T01:05:13.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.507+0000 7fa2d3e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 0x7fa2bc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:13.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.508+0000 7fa2d3e00640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 0x7fa2bc0b60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4108a20 tx=0x7fa2c0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:13.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.508+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa2c8055e20 con 0x7fa2d41090e0 2024-08-20T01:05:13.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.730+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2d41a0690 con 0x7fa2d41090e0 2024-08-20T01:05:13.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.731+0000 7fa2d1600640 1 -- 172.21.15.138:0/145027572 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9719 (secure 0 0 0) 0x7fa2c80ab840 con 0x7fa2d41090e0 2024-08-20T01:05:13.730 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:13.731 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":12,"btime":"2024-08-20T01:05:12:515251+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24706,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[]} 2024-08-20T01:05:13.731 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 12 2024-08-20T01:05:13.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 msgr2=0x7fa2bc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:13.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 0x7fa2bc0b60e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4108a20 tx=0x7fa2c0005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:13.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 msgr2=0x7fa2d419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:13.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800e9f0 tx=0x7fa2c800edf0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 shutdown_connections 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2bc0b3c20 0x7fa2bc0b60e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2d410f470 0x7fa2d41a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2d410eaf0 0x7fa2d419faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 --2- 172.21.15.138:0/145027572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2d41090e0 0x7fa2d419f560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:13.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.733+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 >> 172.21.15.138:0/145027572 conn(0x7fa2d4078760 msgr2=0x7fa2d407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.734+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 shutdown_connections 2024-08-20T01:05:13.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:13.734+0000 7fa2db800640 1 -- 172.21.15.138:0/145027572 wait complete. 2024-08-20T01:05:13.741 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:13 smithi142 systemd[1]: Starting Ceph mds.b for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:05:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:13 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/145027572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:14.041 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:13 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/145027572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:14.041 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:13 smithi142 podman[52964]: 2024-08-20 01:05:13.791366923 +0000 UTC m=+0.015760188 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:05:14.041 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:13 smithi142 podman[52964]: 2024-08-20 01:05:13.920506569 +0000 UTC m=+0.144899872 container create 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:05:14.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:13 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/145027572' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 podman[52964]: 2024-08-20 01:05:14.181263538 +0000 UTC m=+0.405656796 container init 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 podman[52964]: 2024-08-20 01:05:14.184549906 +0000 UTC m=+0.408943167 container start 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 ceph-mds[52983]: -- 172.21.15.142:0/3364863498 <== mon.2 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a1b4701860 con 0x55a1b46de800 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: starting mds.b at 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 bash[52964]: 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 2024-08-20T01:05:14.491 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:14 smithi142 systemd[1]: Started Ceph mds.b for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:05:15.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.354 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:boot 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: fsmap 12 up:standby 2024-08-20T01:05:15.355 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:05:15.575 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.575 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:boot 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: fsmap 12 up:standby 2024-08-20T01:05:15.576 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:05:15.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.585 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:boot 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: fsmap 12 up:standby 2024-08-20T01:05:15.586 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:05:16.501 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:16.898 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:17.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:17.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 -- 172.21.15.138:0/29910612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 msgr2=0x7f7614104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 --2- 172.21.15.138:0/29910612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f7614104ef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7604009900 tx=0x7f760402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 -- 172.21.15.138:0/29910612 shutdown_connections 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 --2- 172.21.15.138:0/29910612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 --2- 172.21.15.138:0/29910612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 --2- 172.21.15.138:0/29910612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f7614104ef0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 -- 172.21.15.138:0/29910612 >> 172.21.15.138:0/29910612 conn(0x7f7614100540 msgr2=0x7f7614102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 -- 172.21.15.138:0/29910612 shutdown_connections 2024-08-20T01:05:17.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.501+0000 7f7619600640 1 -- 172.21.15.138:0/29910612 wait complete. 2024-08-20T01:05:17.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.502+0000 7f7619600640 1 Processor -- start 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.502+0000 7f7619600640 1 -- start start 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f761419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7614114b70 con 0x7f7614104b10 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f76141149f0 con 0x7f761410aea0 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7619600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7614114cf0 con 0x7f761410a520 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7613e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f761419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7613400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7613400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58960/0 (socket says 172.21.15.138:58960) 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7613400640 1 -- 172.21.15.138:0/83399938 learned_addr learned my addr 172.21.15.138:0/83399938 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.503+0000 7f7618c00640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:17.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 -- 172.21.15.138:0/83399938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 msgr2=0x7f76141a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 -- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 msgr2=0x7f761419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f761419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 -- 172.21.15.138:0/83399938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76141a0ff0 con 0x7f761410a520 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7618c00640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7613400640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f760000ea60 tx=0x7f760000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760000cd20 con 0x7f761410a520 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76000035f0 con 0x7f761410a520 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76141a12e0 con 0x7f761410a520 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.504+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600010690 con 0x7f761410a520 2024-08-20T01:05:17.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.505+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76141a7b70 con 0x7f761410a520 2024-08-20T01:05:17.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.506+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7614104ef0 con 0x7f761410a520 2024-08-20T01:05:17.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.507+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f76000037a0 con 0x7f761410a520 2024-08-20T01:05:17.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.510+0000 7f7610c00640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 0x7f75f80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:17.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.510+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f76000e3b70 con 0x7f761410a520 2024-08-20T01:05:17.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.510+0000 7f7613e00640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 0x7f75f80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:17.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.511+0000 7f7613e00640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 0x7f75f80b6200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7604005ab0 tx=0x7f7604005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:17.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.512+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f76000ab4f0 con 0x7f761410a520 2024-08-20T01:05:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.758+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f761410cce0 con 0x7f761410a520 2024-08-20T01:05:17.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.759+0000 7f7610c00640 1 -- 172.21.15.138:0/83399938 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10542 (secure 0 0 0) 0x7f76000aadf0 con 0x7f761410a520 2024-08-20T01:05:17.758 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:17.758 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":13,"btime":"2024-08-20T01:05:14:784267+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24706,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24743,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-08-20T01:05:17.758 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 13 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 msgr2=0x7f75f80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 0x7f75f80b6200 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7604005ab0 tx=0x7f7604005a20 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 msgr2=0x7f761419b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f760000ea60 tx=0x7f760000ee60 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 shutdown_connections 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f75f80b3d40 0x7f75f80b6200 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f761410aea0 0x7f76141a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f761410a520 0x7f761419b480 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 --2- 172.21.15.138:0/83399938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7614104b10 0x7f761419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 >> 172.21.15.138:0/83399938 conn(0x7f7614100540 msgr2=0x7f7614102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.761+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 shutdown_connections 2024-08-20T01:05:17.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:17.762+0000 7f7619600640 1 -- 172.21.15.138:0/83399938 wait complete. 2024-08-20T01:05:18.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:18 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/83399938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:18.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:18 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/83399938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:18.490 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:18 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/83399938' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:19.205 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-08-20T01:05:19.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:19 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:19 smithi142 ceph-mon[21799]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:19.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:19 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:19 smithi138 ceph-mon[18076]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:19 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:19.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:19 smithi167 ceph-mon[21965]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 -- 172.21.15.138:0/1342963120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c10eaf0 msgr2=0x7fa49c10ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1342963120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c10eaf0 0x7fa49c10ef30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa488009900 tx=0x7fa48802a410 comp rx=0 tx=0).stop 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 -- 172.21.15.138:0/1342963120 shutdown_connections 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1342963120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10f470 0x7fa49c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1342963120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c10eaf0 0x7fa49c10ef30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1342963120 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c1090e0 0x7fa49c1094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 -- 172.21.15.138:0/1342963120 >> 172.21.15.138:0/1342963120 conn(0x7fa49c078760 msgr2=0x7fa49c07ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 -- 172.21.15.138:0/1342963120 shutdown_connections 2024-08-20T01:05:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.234+0000 7fa4a2000640 1 -- 172.21.15.138:0/1342963120 wait complete. 2024-08-20T01:05:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.235+0000 7fa4a2000640 1 Processor -- start 2024-08-20T01:05:20.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.235+0000 7fa4a2000640 1 -- start start 2024-08-20T01:05:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:20.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 0x7fa49c19fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c10f470 0x7fa49c1a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1190c0 con 0x7fa49c1090e0 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c118f40 con 0x7fa49c10eaf0 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa4a2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c119240 con 0x7fa49c10f470 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 0x7fa49c19fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35490/0 (socket says 172.21.15.138:35490) 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 -- 172.21.15.138:0/1443084379 learned_addr learned my addr 172.21.15.138:0/1443084379 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49be00640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c10f470 0x7fa49c1a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 -- 172.21.15.138:0/1443084379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c10f470 msgr2=0x7fa49c1a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c10f470 0x7fa49c1a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 -- 172.21.15.138:0/1443084379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 msgr2=0x7fa49c19fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:20.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 0x7fa49c19fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49b400640 1 -- 172.21.15.138:0/1443084379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c1a5640 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.236+0000 7fa49aa00640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 0x7fa49c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa49b400640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e9f0 tx=0x7fa48c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c00cd80 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa48c0108e0 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c018980 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c1a5930 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.237+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1ac1c0 con 0x7fa49c1090e0 2024-08-20T01:05:20.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.238+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa49c10a390 con 0x7fa49c1090e0 2024-08-20T01:05:20.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.240+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fa48c010430 con 0x7fa49c1090e0 2024-08-20T01:05:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.242+0000 7fa493e00640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 0x7fa4800b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.242+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7fa48c0e3bc0 con 0x7fa49c1090e0 2024-08-20T01:05:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.242+0000 7fa49aa00640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 0x7fa4800b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:20.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.243+0000 7fa49aa00640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 0x7fa4800b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa488005ab0 tx=0x7fa488005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.246+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa48c0aaef0 con 0x7fa49c1090e0 2024-08-20T01:05:20.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.452+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa49c110cc0 con 0x7fa49c1090e0 2024-08-20T01:05:20.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.453+0000 7fa493e00640 1 -- 172.21.15.138:0/1443084379 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7fa48c0af440 con 0x7fa49c1090e0 2024-08-20T01:05:20.451 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout:[ 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout: "poolnum": 1, 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout: "poolname": ".mgr" 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:05:20.452 INFO:teuthology.orchestra.run.smithi138.stdout:] 2024-08-20T01:05:20.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 msgr2=0x7fa4800b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:20.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 0x7fa4800b6160 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa488005ab0 tx=0x7fa488005a20 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 msgr2=0x7fa49c19f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa48c00e9f0 tx=0x7fa48c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 shutdown_connections 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4800b3ca0 0x7fa4800b6160 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa49c10f470 0x7fa49c1a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa49c10eaf0 0x7fa49c19fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 --2- 172.21.15.138:0/1443084379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa49c1090e0 0x7fa49c19f590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 >> 172.21.15.138:0/1443084379 conn(0x7fa49c078760 msgr2=0x7fa49c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 shutdown_connections 2024-08-20T01:05:20.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:20.456+0000 7fa4a2000640 1 -- 172.21.15.138:0/1443084379 wait complete. 2024-08-20T01:05:20.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:20 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1443084379' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T01:05:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:20 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1443084379' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T01:05:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:20 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1443084379' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-08-20T01:05:21.285 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-08-20T01:05:21.285 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-08-20T01:05:21.656 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:21.794 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:21.794 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.794 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.794 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:21.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:21.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:21 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:21 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:05:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:21 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:05:22.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.224+0000 7f754d800640 1 -- 172.21.15.138:0/2824347151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7548075270 msgr2=0x7f7548075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:22.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.224+0000 7f754d800640 1 --2- 172.21.15.138:0/2824347151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7548075270 0x7f7548075650 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7538009900 tx=0x7f753802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.223 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 -- 172.21.15.138:0/2824347151 shutdown_connections 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 --2- 172.21.15.138:0/2824347151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f754810f0d0 0x7f75481114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 --2- 172.21.15.138:0/2824347151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075c70 0x7f754810eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 --2- 172.21.15.138:0/2824347151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7548075270 0x7f7548075650 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 -- 172.21.15.138:0/2824347151 >> 172.21.15.138:0/2824347151 conn(0x7f75480fe500 msgr2=0x7f7548100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 -- 172.21.15.138:0/2824347151 shutdown_connections 2024-08-20T01:05:22.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.225+0000 7f754d800640 1 -- 172.21.15.138:0/2824347151 wait complete. 2024-08-20T01:05:22.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 Processor -- start 2024-08-20T01:05:22.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 -- start start 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075270 0x7f754819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 0x7f754819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7548114cd0 con 0x7f754810f0d0 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7548114b50 con 0x7f7548075c70 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.226+0000 7f754d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7548114e50 con 0x7f7548075270 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f7547400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 0x7f754819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f7546a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075270 0x7f754819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:22.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f7547400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35504/0 (socket says 172.21.15.138:35504) 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f7547400640 1 -- 172.21.15.138:0/3624736222 learned_addr learned my addr 172.21.15.138:0/3624736222 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 -- 172.21.15.138:0/3624736222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075270 msgr2=0x7f754819b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075270 0x7f754819b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 -- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 msgr2=0x7f75481a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 -- 172.21.15.138:0/3624736222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75481a1270 con 0x7f7548075c70 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f7547400640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:22.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.227+0000 7f753e000640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 0x7f754819b700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f753000d720 tx=0x7f753000dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.228+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7530017070 con 0x7f7548075c70 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.228+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f753000de30 con 0x7f7548075c70 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.228+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75481a1560 con 0x7f7548075c70 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.228+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75300047a0 con 0x7f7548075c70 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.228+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75481a7c20 con 0x7f7548075c70 2024-08-20T01:05:22.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.229+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7548075650 con 0x7f7548075c70 2024-08-20T01:05:22.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.231+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f75300099d0 con 0x7f7548075c70 2024-08-20T01:05:22.230 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.233+0000 7f753fe00640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 0x7f752c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:22.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.233+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7342+0+0 (secure 0 0 0) 0x7f75300e38e0 con 0x7f7548075c70 2024-08-20T01:05:22.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.233+0000 7f7546a00640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 0x7f752c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:22.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.233+0000 7f7546a00640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 0x7f752c0b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7538005ab0 tx=0x7f7538005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:22.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.235+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7530020020 con 0x7f7548075c70 2024-08-20T01:05:22.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.418+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f75481a7f50 con 0x7f7548075c70 2024-08-20T01:05:22.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.558+0000 7f753fe00640 1 -- 172.21.15.138:0/3624736222 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f75300ab2e0 con 0x7f7548075c70 2024-08-20T01:05:22.557 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' created 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 msgr2=0x7f752c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 0x7f752c0b6160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7538005ab0 tx=0x7f7538005a20 comp rx=0 tx=0).stop 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 msgr2=0x7f754819b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 0x7f754819b700 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f753000d720 tx=0x7f753000dbe0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 shutdown_connections 2024-08-20T01:05:22.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f752c0b3ca0 0x7f752c0b6160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f754810f0d0 0x7f75481a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7548075c70 0x7f754819b700 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 --2- 172.21.15.138:0/3624736222 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7548075270 0x7f754819b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 >> 172.21.15.138:0/3624736222 conn(0x7f75480fe500 msgr2=0x7f75481112b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 shutdown_connections 2024-08-20T01:05:22.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:22.562+0000 7f754d800640 1 -- 172.21.15.138:0/3624736222 wait complete. 2024-08-20T01:05:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:22 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3624736222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:22 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:22.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:22 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3624736222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:22.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:22 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:22 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3624736222' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:22 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-08-20T01:05:23.395 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-08-20T01:05:23.777 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:23 smithi138 ceph-mon[18076]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:23 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T01:05:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:23 smithi138 ceph-mon[18076]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T01:05:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:23 smithi167 ceph-mon[21965]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:23 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T01:05:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:23 smithi167 ceph-mon[21965]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T01:05:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:23 smithi142 ceph-mon[21799]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:23 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-08-20T01:05:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:23 smithi142 ceph-mon[21799]: osdmap e73: 12 total, 12 up, 12 in 2024-08-20T01:05:24.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 -- 172.21.15.138:0/580160407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 msgr2=0x7f4c2010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:24.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 --2- 172.21.15.138:0/580160407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2010aaf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009900 tx=0x7f4c0c02a410 comp rx=0 tx=0).stop 2024-08-20T01:05:24.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 -- 172.21.15.138:0/580160407 shutdown_connections 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 --2- 172.21.15.138:0/580160407 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c2010b030 0x7f4c20111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 --2- 172.21.15.138:0/580160407 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2010aaf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 --2- 172.21.15.138:0/580160407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c20104ca0 0x7f4c20105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 -- 172.21.15.138:0/580160407 >> 172.21.15.138:0/580160407 conn(0x7f4c20100730 msgr2=0x7f4c20102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 -- 172.21.15.138:0/580160407 shutdown_connections 2024-08-20T01:05:24.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.371+0000 7f4c28400640 1 -- 172.21.15.138:0/580160407 wait complete. 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 Processor -- start 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 -- start start 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 0x7f4c201a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20114cd0 con 0x7f4c2010a6b0 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20114b50 con 0x7f4c2010b030 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.372+0000 7f4c28400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4c20114e50 con 0x7f4c20104ca0 2024-08-20T01:05:24.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37962/0 (socket says 172.21.15.138:37962) 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 -- 172.21.15.138:0/1074668013 learned_addr learned my addr 172.21.15.138:0/1074668013 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25000640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c26400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 0x7f4c201a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 -- 172.21.15.138:0/1074668013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 msgr2=0x7f4c201a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 0x7f4c201a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 -- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 msgr2=0x7f4c2019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 -- 172.21.15.138:0/1074668013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c201a1230 con 0x7f4c20104ca0 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c26400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 0x7f4c201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.373+0000 7f4c25a00640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d50 tx=0x7f4c1400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:24.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.374+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003b70 con 0x7f4c20104ca0 2024-08-20T01:05:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.374+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c201a1520 con 0x7f4c20104ca0 2024-08-20T01:05:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.374+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c14003d10 con 0x7f4c20104ca0 2024-08-20T01:05:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.374+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140108b0 con 0x7f4c20104ca0 2024-08-20T01:05:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.374+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c201a7db0 con 0x7f4c20104ca0 2024-08-20T01:05:24.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.375+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c20105080 con 0x7f4c20104ca0 2024-08-20T01:05:24.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.377+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f4c14004000 con 0x7f4c20104ca0 2024-08-20T01:05:24.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.379+0000 7f4c1e000640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 0x7f4c080b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.379+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7712+0+0 (secure 0 0 0) 0x7f4c140e4280 con 0x7f4c20104ca0 2024-08-20T01:05:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.379+0000 7f4c25000640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 0x7f4c080b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:24.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.379+0000 7f4c25000640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 0x7f4c080b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005ab0 tx=0x7f4c0c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:24.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.380+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4c140abb40 con 0x7f4c20104ca0 2024-08-20T01:05:24.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:24.569+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f4c2019c2b0 con 0x7f4c20104ca0 2024-08-20T01:05:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:24 smithi167 ceph-mon[21965]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T01:05:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:24 smithi142 ceph-mon[21799]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T01:05:25.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:24 smithi138 ceph-mon[18076]: osdmap e74: 12 total, 12 up, 12 in 2024-08-20T01:05:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.584+0000 7f4c1e000640 1 -- 172.21.15.138:0/1074668013 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f4c140ab440 con 0x7f4c20104ca0 2024-08-20T01:05:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' created 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.591+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 msgr2=0x7f4c080b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.591+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 0x7f4c080b60e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c005ab0 tx=0x7f4c0c005a20 comp rx=0 tx=0).stop 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 msgr2=0x7f4c2019b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009d50 tx=0x7f4c1400c6a0 comp rx=0 tx=0).stop 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 shutdown_connections 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4c080b3c20 0x7f4c080b60e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:25.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4c2010b030 0x7f4c201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2010a6b0 0x7f4c2019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 --2- 172.21.15.138:0/1074668013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4c20104ca0 0x7f4c2019b180 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 >> 172.21.15.138:0/1074668013 conn(0x7f4c20100730 msgr2=0x7f4c20107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 shutdown_connections 2024-08-20T01:05:25.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:25.592+0000 7f4c28400640 1 -- 172.21.15.138:0/1074668013 wait complete. 2024-08-20T01:05:25.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:25 smithi167 ceph-mon[21965]: pgmap v267: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:25.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:25 smithi167 ceph-mon[21965]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:25.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:25 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1074668013' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:25.857 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:25 smithi167 ceph-mon[21965]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T01:05:25.858 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:25 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:25 smithi142 ceph-mon[21799]: pgmap v267: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:25 smithi142 ceph-mon[21799]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:25 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1074668013' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:25 smithi142 ceph-mon[21799]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T01:05:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:25 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:26.017 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:25 smithi138 ceph-mon[18076]: pgmap v267: 33 pgs: 27 unknown, 6 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:26.018 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:25 smithi138 ceph-mon[18076]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:26.018 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:25 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1074668013' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:26.018 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:25 smithi138 ceph-mon[18076]: osdmap e75: 12 total, 12 up, 12 in 2024-08-20T01:05:26.018 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:25 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} : dispatch 2024-08-20T01:05:26.415 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-08-20T01:05:26.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:26 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-20T01:05:26.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:26 smithi167 ceph-mon[21965]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T01:05:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:26 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-20T01:05:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:26 smithi138 ceph-mon[18076]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T01:05:26.806 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:26 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-08-20T01:05:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:26 smithi142 ceph-mon[21799]: osdmap e76: 12 total, 12 up, 12 in 2024-08-20T01:05:27.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 -- 172.21.15.138:0/466291343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe010a6a0 msgr2=0x7febe010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 --2- 172.21.15.138:0/466291343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe010a6a0 0x7febe010aae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7febc8009900 tx=0x7febc802a410 comp rx=0 tx=0).stop 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 -- 172.21.15.138:0/466291343 shutdown_connections 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 --2- 172.21.15.138:0/466291343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 0x7febe0111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 --2- 172.21.15.138:0/466291343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe010a6a0 0x7febe010aae0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 --2- 172.21.15.138:0/466291343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe0104c90 0x7febe0105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 -- 172.21.15.138:0/466291343 >> 172.21.15.138:0/466291343 conn(0x7febe01006c0 msgr2=0x7febe0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 -- 172.21.15.138:0/466291343 shutdown_connections 2024-08-20T01:05:27.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.412+0000 7febe5200640 1 -- 172.21.15.138:0/466291343 wait complete. 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 Processor -- start 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 -- start start 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 0x7febe019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 0x7febe019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 0x7febe01a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114ce0 con 0x7febe0104c90 2024-08-20T01:05:27.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114b60 con 0x7febe010b020 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.413+0000 7febe5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7febe0114e60 con 0x7febe010a6a0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febdde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 0x7febe019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 0x7febe019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febdde00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 0x7febe019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35542/0 (socket says 172.21.15.138:35542) 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febdde00640 1 -- 172.21.15.138:0/387790263 learned_addr learned my addr 172.21.15.138:0/387790263 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febde800640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 0x7febe01a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 -- 172.21.15.138:0/387790263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 msgr2=0x7febe01a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 0x7febe01a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 -- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 msgr2=0x7febe019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 0x7febe019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 -- 172.21.15.138:0/387790263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01a1220 con 0x7febe010a6a0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.414+0000 7febd7e00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 0x7febe019b6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc8004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:27.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.415+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc803f070 con 0x7febe010a6a0 2024-08-20T01:05:27.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.415+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01a14b0 con 0x7febe010a6a0 2024-08-20T01:05:27.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.415+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febc8003ca0 con 0x7febe010a6a0 2024-08-20T01:05:27.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.415+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8035070 con 0x7febe010a6a0 2024-08-20T01:05:27.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.415+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01a7bd0 con 0x7febe010a6a0 2024-08-20T01:05:27.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.416+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febe0105070 con 0x7febe010a6a0 2024-08-20T01:05:27.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.418+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7febc803b460 con 0x7febe010a6a0 2024-08-20T01:05:27.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.420+0000 7febd6a00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 0x7febc40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:27.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.420+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8094+0+0 (secure 0 0 0) 0x7febc8104310 con 0x7febe010a6a0 2024-08-20T01:05:27.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.420+0000 7febdde00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 0x7febc40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:27.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.420+0000 7febdde00640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 0x7febc40b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febd0009cd0 tx=0x7febd0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:27.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.421+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7febc80cb350 con 0x7febe010a6a0 2024-08-20T01:05:27.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:27.646+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7febe019c2a0 con 0x7febe010a6a0 2024-08-20T01:05:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:27 smithi167 ceph-mon[21965]: pgmap v270: 97 pgs: 4 creating+peering, 67 unknown, 26 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:27.973 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:27 smithi167 ceph-mon[21965]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T01:05:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:27 smithi142 ceph-mon[21799]: pgmap v270: 97 pgs: 4 creating+peering, 67 unknown, 26 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:27 smithi142 ceph-mon[21799]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T01:05:28.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:27 smithi138 ceph-mon[18076]: pgmap v270: 97 pgs: 4 creating+peering, 67 unknown, 26 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:28.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:27 smithi138 ceph-mon[18076]: osdmap e77: 12 total, 12 up, 12 in 2024-08-20T01:05:28.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.605+0000 7febd6a00640 1 -- 172.21.15.138:0/387790263 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7febc80cf8a0 con 0x7febe010a6a0 2024-08-20T01:05:28.604 INFO:teuthology.orchestra.run.smithi138.stderr:new fs with metadata pool 2 and data pool 3 2024-08-20T01:05:28.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 msgr2=0x7febc40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:28.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 0x7febc40b60e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7febd0009cd0 tx=0x7febd0005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:28.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 msgr2=0x7febe019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:28.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 0x7febe019b6b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc8004060 comp rx=0 tx=0).stop 2024-08-20T01:05:28.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 shutdown_connections 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7febc40b3c20 0x7febc40b60e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febe010b020 0x7febe01a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7febe010a6a0 0x7febe019b6b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 --2- 172.21.15.138:0/387790263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7febe0104c90 0x7febe019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 >> 172.21.15.138:0/387790263 conn(0x7febe01006c0 msgr2=0x7febe0102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 shutdown_connections 2024-08-20T01:05:28.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:28.608+0000 7febe5200640 1 -- 172.21.15.138:0/387790263 wait complete. 2024-08-20T01:05:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:28 smithi167 ceph-mon[21965]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T01:05:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:28 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/387790263' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:28 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:28 smithi167 ceph-mon[21965]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T01:05:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:28 smithi142 ceph-mon[21799]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T01:05:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:28 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/387790263' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:28 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:28 smithi142 ceph-mon[21799]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T01:05:29.029 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:28 smithi138 ceph-mon[18076]: osdmap e78: 12 total, 12 up, 12 in 2024-08-20T01:05:29.030 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:28 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/387790263' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:29.030 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:28 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-08-20T01:05:29.030 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:28 smithi138 ceph-mon[18076]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-08-20T01:05:29.556 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-08-20T01:05:29.556 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-08-20T01:05:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: pgmap v273: 97 pgs: 4 creating+activating, 4 creating+peering, 50 unknown, 39 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T01:05:29.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T01:05:29.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T01:05:29.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T01:05:29.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:active 2024-08-20T01:05:29.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:29 smithi138 ceph-mon[18076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:29.927 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: pgmap v273: 97 pgs: 4 creating+activating, 4 creating+peering, 50 unknown, 39 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:active 2024-08-20T01:05:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:29 smithi167 ceph-mon[21965]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: 2024-08-20T01:05:29.792+0000 7f8921da7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: pgmap v273: 97 pgs: 4 creating+activating, 4 creating+peering, 50 unknown, 39 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: osdmap e79: 12 total, 12 up, 12 in 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] up:active 2024-08-20T01:05:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:29 smithi142 ceph-mon[21799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 -- 172.21.15.138:0/1094437823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10a6b0 msgr2=0x7fd82c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 --2- 172.21.15.138:0/1094437823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10a6b0 0x7fd82c10aaf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd814009900 tx=0x7fd81402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 -- 172.21.15.138:0/1094437823 shutdown_connections 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 --2- 172.21.15.138:0/1094437823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c10b030 0x7fd82c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 --2- 172.21.15.138:0/1094437823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10a6b0 0x7fd82c10aaf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 --2- 172.21.15.138:0/1094437823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c104ca0 0x7fd82c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 -- 172.21.15.138:0/1094437823 >> 172.21.15.138:0/1094437823 conn(0x7fd82c100730 msgr2=0x7fd82c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.484+0000 7fd830a00640 1 -- 172.21.15.138:0/1094437823 shutdown_connections 2024-08-20T01:05:30.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.485+0000 7fd830a00640 1 -- 172.21.15.138:0/1094437823 wait complete. 2024-08-20T01:05:30.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.485+0000 7fd830a00640 1 Processor -- start 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 -- start start 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 0x7fd82c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c10a6b0 0x7fd82c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 0x7fd82c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c114d40 con 0x7fd82c10b030 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c114bc0 con 0x7fd82c104ca0 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd830a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c114ec0 con 0x7fd82c10a6b0 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd82a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 0x7fd82c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd82a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 0x7fd82c1a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40204/0 (socket says 172.21.15.138:40204) 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd829e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 0x7fd82c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd82a800640 1 -- 172.21.15.138:0/3443553988 learned_addr learned my addr 172.21.15.138:0/3443553988 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:30.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.486+0000 7fd823e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c10a6b0 0x7fd82c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 -- 172.21.15.138:0/3443553988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c10a6b0 msgr2=0x7fd82c19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c10a6b0 0x7fd82c19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 -- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 msgr2=0x7fd82c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 0x7fd82c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:30.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 -- 172.21.15.138:0/3443553988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c1a1240 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd829e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 0x7fd82c19b190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e9f0 tx=0x7fd81c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00ccc0 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c1a1530 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c0108e0 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c018880 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.487+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a7dc0 con 0x7fd82c104ca0 2024-08-20T01:05:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.488+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c105f50 con 0x7fd82c104ca0 2024-08-20T01:05:30.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.491+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fd81c010430 con 0x7fd82c104ca0 2024-08-20T01:05:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.492+0000 7fd822a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 0x7fd8100b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:30.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.492+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8194+0+0 (secure 0 0 0) 0x7fd81c0e44c0 con 0x7fd82c104ca0 2024-08-20T01:05:30.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.492+0000 7fd823e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 0x7fd8100b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:30.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.493+0000 7fd823e00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 0x7fd8100b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd814000c00 tx=0x7fd814031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:30.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.494+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd81c0abad0 con 0x7fd82c104ca0 2024-08-20T01:05:30.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:30.691+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fd82c10ce10 con 0x7fd82c104ca0 2024-08-20T01:05:30.690 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:30 smithi138 ceph-mon[18076]: pgmap v275: 97 pgs: 4 creating+activating, 4 creating+peering, 9 unknown, 80 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-20T01:05:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:30 smithi167 ceph-mon[21965]: pgmap v275: 97 pgs: 4 creating+activating, 4 creating+peering, 9 unknown, 80 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-20T01:05:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:30 smithi142 ceph-mon[21799]: pgmap v275: 97 pgs: 4 creating+activating, 4 creating+peering, 9 unknown, 80 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-08-20T01:05:31.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.619+0000 7fd822a00640 1 -- 172.21.15.138:0/3443553988 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7fd81c0ab3d0 con 0x7fd82c104ca0 2024-08-20T01:05:31.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 msgr2=0x7fd8100b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 0x7fd8100b60e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd814000c00 tx=0x7fd814031400 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 msgr2=0x7fd82c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 0x7fd82c19b190 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00e9f0 tx=0x7fd81c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 shutdown_connections 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd8100b3c20 0x7fd8100b60e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd82c10b030 0x7fd82c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd82c10a6b0 0x7fd82c19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 --2- 172.21.15.138:0/3443553988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd82c104ca0 0x7fd82c19b190 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 >> 172.21.15.138:0/3443553988 conn(0x7fd82c100730 msgr2=0x7fd82c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:31.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 shutdown_connections 2024-08-20T01:05:31.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:31.622+0000 7fd830a00640 1 -- 172.21.15.138:0/3443553988 wait complete. 2024-08-20T01:05:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:31 smithi167 ceph-mon[21965]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T01:05:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:31 smithi167 ceph-mon[21965]: Cluster is now healthy 2024-08-20T01:05:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:31 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3443553988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:31 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:31 smithi142 ceph-mon[21799]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T01:05:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:31 smithi142 ceph-mon[21799]: Cluster is now healthy 2024-08-20T01:05:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:31 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3443553988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:31 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:32.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:31 smithi138 ceph-mon[18076]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-08-20T01:05:32.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:31 smithi138 ceph-mon[18076]: Cluster is now healthy 2024-08-20T01:05:32.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:31 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3443553988' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:32.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:31 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} : dispatch 2024-08-20T01:05:32.678 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-08-20T01:05:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:32 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-20T01:05:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:32 smithi167 ceph-mon[21965]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T01:05:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:32 smithi167 ceph-mon[21965]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-08-20T01:05:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:32 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-20T01:05:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:32 smithi142 ceph-mon[21799]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T01:05:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:32 smithi142 ceph-mon[21799]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-08-20T01:05:33.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:32 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-08-20T01:05:33.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:32 smithi138 ceph-mon[18076]: osdmap e80: 12 total, 12 up, 12 in 2024-08-20T01:05:33.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:32 smithi138 ceph-mon[18076]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s wr, 7 op/s 2024-08-20T01:05:33.050 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:33.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 -- 172.21.15.138:0/1696550998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98113670 msgr2=0x7f4e98115a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 --2- 172.21.15.138:0/1696550998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98113670 0x7f4e98115a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009ca0 tx=0x7f4e9002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 -- 172.21.15.138:0/1696550998 shutdown_connections 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 --2- 172.21.15.138:0/1696550998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98113670 0x7f4e98115a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 --2- 172.21.15.138:0/1696550998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075b90 0x7f4e98113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 --2- 172.21.15.138:0/1696550998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98075270 0x7f4e98075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 -- 172.21.15.138:0/1696550998 >> 172.21.15.138:0/1696550998 conn(0x7f4e980fe3c0 msgr2=0x7f4e981007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 -- 172.21.15.138:0/1696550998 shutdown_connections 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.680+0000 7f4e97e00640 1 -- 172.21.15.138:0/1696550998 wait complete. 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.681+0000 7f4e97e00640 1 Processor -- start 2024-08-20T01:05:33.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.681+0000 7f4e97e00640 1 -- start start 2024-08-20T01:05:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 0x7f4e9819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98113670 0x7f4e981a4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98118f40 con 0x7f4e98075270 2024-08-20T01:05:33.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e98118dc0 con 0x7f4e98075b90 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e981190c0 con 0x7f4e98113670 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e96a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e96000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 0x7f4e9819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e96a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40228/0 (socket says 172.21.15.138:40228) 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e96a00640 1 -- 172.21.15.138:0/3162983605 learned_addr learned my addr 172.21.15.138:0/3162983605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.682+0000 7f4e97400640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98113670 0x7f4e981a4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 -- 172.21.15.138:0/3162983605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98113670 msgr2=0x7f4e981a4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98113670 0x7f4e981a4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 -- 172.21.15.138:0/3162983605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 msgr2=0x7f4e9819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 0x7f4e9819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 -- 172.21.15.138:0/3162983605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e981a5380 con 0x7f4e98075270 2024-08-20T01:05:33.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96000640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 0x7f4e9819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e96a00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9f0 tx=0x7f4e8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8800cd80 con 0x7f4e98075270 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e981a5670 con 0x7f4e98075270 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e880108e0 con 0x7f4e98075270 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.683+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e88018980 con 0x7f4e98075270 2024-08-20T01:05:33.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.684+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e981abf00 con 0x7f4e98075270 2024-08-20T01:05:33.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.685+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e98076780 con 0x7f4e98075270 2024-08-20T01:05:33.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.687+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f4e88010430 con 0x7f4e98075270 2024-08-20T01:05:33.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.688+0000 7f4e9ca00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 0x7f4e7c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:33.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.689+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8492+0+0 (secure 0 0 0) 0x7f4e880e4510 con 0x7f4e98075270 2024-08-20T01:05:33.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.689+0000 7f4e96000640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 0x7f4e7c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:33.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.689+0000 7f4e96000640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 0x7f4e7c0b6200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4e98115a40 tx=0x7f4e84005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:33.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.691+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4e880ab3b0 con 0x7f4e98075270 2024-08-20T01:05:33.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:33.884+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f4e981ac1e0 con 0x7f4e98075270 2024-08-20T01:05:34.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:33 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T01:05:34.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:33 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T01:05:34.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:33 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-08-20T01:05:34.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.939+0000 7f4e9ca00640 1 -- 172.21.15.138:0/3162983605 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f4e880af900 con 0x7f4e98075270 2024-08-20T01:05:34.938 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data_ec' created 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 msgr2=0x7f4e7c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 0x7f4e7c0b6200 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4e98115a40 tx=0x7f4e84005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 msgr2=0x7f4e9819f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4e8800e9f0 tx=0x7f4e8800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 shutdown_connections 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e7c0b3d40 0x7f4e7c0b6200 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e98113670 0x7f4e981a4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e98075b90 0x7f4e9819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:34.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 --2- 172.21.15.138:0/3162983605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e98075270 0x7f4e9819f2d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:34.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.942+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 >> 172.21.15.138:0/3162983605 conn(0x7f4e980fe3c0 msgr2=0x7f4e980fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:34.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.944+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 shutdown_connections 2024-08-20T01:05:34.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:34.944+0000 7f4e97e00640 1 -- 172.21.15.138:0/3162983605 wait complete. 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.193 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-20T01:05:35.194 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:34 smithi138 ceph-mon[18076]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-20T01:05:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:34 smithi167 ceph-mon[21965]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T01:05:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: osdmap e81: 12 total, 12 up, 12 in 2024-08-20T01:05:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]} : dispatch 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]} : dispatch 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 7 op/s 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3162983605' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 4]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 6]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-08-20T01:05:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:34 smithi142 ceph-mon[21799]: osdmap e82: 12 total, 12 up, 12 in 2024-08-20T01:05:35.750 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-08-20T01:05:36.141 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:36.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 -- 172.21.15.138:0/3817298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4075b30 msgr2=0x7f6eb410e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3817298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4075b30 0x7f6eb410e9d0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0009860 tx=0x7f6ea002a340 comp rx=0 tx=0).stop 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 -- 172.21.15.138:0/3817298761 shutdown_connections 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3817298761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb410ef10 0x7f6eb4111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3817298761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb4075b30 0x7f6eb410e9d0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3817298761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075210 0x7f6eb40755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 -- 172.21.15.138:0/3817298761 >> 172.21.15.138:0/3817298761 conn(0x7f6eb40fe350 msgr2=0x7f6eb4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 -- 172.21.15.138:0/3817298761 shutdown_connections 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.784+0000 7f6eb9200640 1 -- 172.21.15.138:0/3817298761 wait complete. 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.785+0000 7f6eb9200640 1 Processor -- start 2024-08-20T01:05:36.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 -- start start 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 0x7f6eb419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075b30 0x7f6eb419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4114af0 con 0x7f6eb410ef10 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4114970 con 0x7f6eb4075b30 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb9200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb4114c70 con 0x7f6eb4075210 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40238/0 (socket says 172.21.15.138:40238) 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb3e00640 1 -- 172.21.15.138:0/3037458431 learned_addr learned my addr 172.21.15.138:0/3037458431 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.786+0000 7f6eb3400640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 0x7f6eb419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb2a00640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075b30 0x7f6eb419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 -- 172.21.15.138:0/3037458431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 msgr2=0x7f6eb419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:36.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 0x7f6eb419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 -- 172.21.15.138:0/3037458431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075b30 msgr2=0x7f6eb419b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075b30 0x7f6eb419b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 -- 172.21.15.138:0/3037458431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb41a1030 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3400640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 0x7f6eb419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb3e00640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009fa0 tx=0x7f6eac00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac004b20 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb41a1320 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eac003e20 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.787+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00b570 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.788+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb41a7bb0 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.789+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb4076720 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.791+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f6eac01d020 con 0x7f6eb410ef10 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.792+0000 7f6eabe00640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 0x7f6e980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:36.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.793+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 9217+0+0 (secure 0 0 0) 0x7f6eac0e4990 con 0x7f6eb410ef10 2024-08-20T01:05:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.793+0000 7f6eb3400640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 0x7f6e980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.793+0000 7f6eb3400640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 0x7f6e980b6200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40095f0 tx=0x7f6ea4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:36.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.795+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6eac024030 con 0x7f6eb410ef10 2024-08-20T01:05:36.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:36.981+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f6eb41a7d80 con 0x7f6eb410ef10 2024-08-20T01:05:37.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:36 smithi138 ceph-mon[18076]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T01:05:37.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:36 smithi138 ceph-mon[18076]: pgmap v282: 129 pgs: 13 creating+peering, 2 peering, 15 unknown, 99 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-20T01:05:37.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:36 smithi167 ceph-mon[21965]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T01:05:37.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:36 smithi167 ceph-mon[21965]: pgmap v282: 129 pgs: 13 creating+peering, 2 peering, 15 unknown, 99 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-20T01:05:37.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:36 smithi142 ceph-mon[21799]: osdmap e83: 12 total, 12 up, 12 in 2024-08-20T01:05:37.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:36 smithi142 ceph-mon[21799]: pgmap v282: 129 pgs: 13 creating+peering, 2 peering, 15 unknown, 99 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-08-20T01:05:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.955+0000 7f6eabe00640 1 -- 172.21.15.138:0/3037458431 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f6eac0b2e20 con 0x7f6eb410ef10 2024-08-20T01:05:37.954 INFO:teuthology.orchestra.run.smithi138.stderr:set pool 4 allow_ec_overwrites to true 2024-08-20T01:05:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.958+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 msgr2=0x7f6e980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.958+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 0x7f6e980b6200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ea40095f0 tx=0x7f6ea4005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.958+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 msgr2=0x7f6eb41a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:37.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.958+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6eac009fa0 tx=0x7f6eac00c800 comp rx=0 tx=0).stop 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 shutdown_connections 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e980b3d40 0x7f6e980b6200 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb410ef10 0x7f6eb41a0860 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6eb4075b30 0x7f6eb419b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 --2- 172.21.15.138:0/3037458431 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6eb4075210 0x7f6eb419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 >> 172.21.15.138:0/3037458431 conn(0x7f6eb40fe350 msgr2=0x7f6eb40fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 shutdown_connections 2024-08-20T01:05:37.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:37.959+0000 7f6eb9200640 1 -- 172.21.15.138:0/3037458431 wait complete. 2024-08-20T01:05:38.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:37 smithi167 ceph-mon[21965]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:38.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:37 smithi167 ceph-mon[21965]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T01:05:38.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:37 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-20T01:05:38.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:37 smithi142 ceph-mon[21799]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:38.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:37 smithi142 ceph-mon[21799]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T01:05:38.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:37 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-20T01:05:38.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:37 smithi138 ceph-mon[18076]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-08-20T01:05:38.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:37 smithi138 ceph-mon[18076]: osdmap e84: 12 total, 12 up, 12 in 2024-08-20T01:05:38.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:37 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd={"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} : dispatch 2024-08-20T01:05:38.728 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-08-20T01:05:39.100 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:39.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:38 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-20T01:05:39.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:38 smithi167 ceph-mon[21965]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T01:05:39.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:38 smithi167 ceph-mon[21965]: pgmap v285: 129 pgs: 13 creating+peering, 2 peering, 13 unknown, 101 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:39.238 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:38 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-20T01:05:39.238 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:38 smithi138 ceph-mon[18076]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T01:05:39.238 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:38 smithi138 ceph-mon[18076]: pgmap v285: 129 pgs: 13 creating+peering, 2 peering, 13 unknown, 101 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:39.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:38 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3037458431' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-08-20T01:05:39.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:38 smithi142 ceph-mon[21799]: osdmap e85: 12 total, 12 up, 12 in 2024-08-20T01:05:39.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:38 smithi142 ceph-mon[21799]: pgmap v285: 129 pgs: 13 creating+peering, 2 peering, 13 unknown, 101 active+clean; 579 KiB data, 343 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 -- 172.21.15.138:0/1044302299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b810ae90 msgr2=0x7fe7b8111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 --2- 172.21.15.138:0/1044302299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b810ae90 0x7fe7b8111690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009ca0 tx=0x7fe7b002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 -- 172.21.15.138:0/1044302299 shutdown_connections 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 --2- 172.21.15.138:0/1044302299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b810ae90 0x7fe7b8111690 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 --2- 172.21.15.138:0/1044302299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810a510 0x7fe7b810a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 --2- 172.21.15.138:0/1044302299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b8104b00 0x7fe7b8104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 -- 172.21.15.138:0/1044302299 >> 172.21.15.138:0/1044302299 conn(0x7fe7b8100550 msgr2=0x7fe7b8102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 -- 172.21.15.138:0/1044302299 shutdown_connections 2024-08-20T01:05:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.701+0000 7fe7b7200640 1 -- 172.21.15.138:0/1044302299 wait complete. 2024-08-20T01:05:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.702+0000 7fe7b7200640 1 Processor -- start 2024-08-20T01:05:39.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.702+0000 7fe7b7200640 1 -- start start 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8104b00 0x7fe7b819af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b810a510 0x7fe7b819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114b60 con 0x7fe7b8104b00 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b81149e0 con 0x7fe7b810ae90 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b8114ce0 con 0x7fe7b810a510 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8104b00 0x7fe7b819af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56736/0 (socket says 172.21.15.138:56736) 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 -- 172.21.15.138:0/3422441000 learned_addr learned my addr 172.21.15.138:0/3422441000 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7afe00640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b810a510 0x7fe7b819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:39.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 -- 172.21.15.138:0/3422441000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b810a510 msgr2=0x7fe7b819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b810a510 0x7fe7b819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 -- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8104b00 msgr2=0x7fe7b819af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8104b00 0x7fe7b819af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.703+0000 7fe7b6800640 1 -- 172.21.15.138:0/3422441000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b81a0fc0 con 0x7fe7b810ae90 2024-08-20T01:05:39.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7b6800640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002f540 tx=0x7fe7b00316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0003b70 con 0x7fe7b810ae90 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7b0003d10 con 0x7fe7b810ae90 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b81a1250 con 0x7fe7b810ae90 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b002f990 con 0x7fe7b810ae90 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.704+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b81a7970 con 0x7fe7b810ae90 2024-08-20T01:05:39.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.705+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b8104ee0 con 0x7fe7b810ae90 2024-08-20T01:05:39.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.707+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fe7b002fb30 con 0x7fe7b810ae90 2024-08-20T01:05:39.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.709+0000 7fe7aea00640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 0x7fe79c0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:39.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.709+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 9217+0+0 (secure 0 0 0) 0x7fe7b0105590 con 0x7fe7b810ae90 2024-08-20T01:05:39.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.709+0000 7fe7b5e00640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 0x7fe79c0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:39.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.710+0000 7fe7b5e00640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 0x7fe79c0b6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009bd0 tx=0x7fe7a8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:39.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.711+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe7b00cc7a0 con 0x7fe7b810ae90 2024-08-20T01:05:39.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.933+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe7b81a7c30 con 0x7fe7b810ae90 2024-08-20T01:05:39.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.973+0000 7fe7aea00640 1 -- 172.21.15.138:0/3422441000 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v16) ==== 137+0+0 (secure 0 0 0) 0x7fe7b00cc0a0 con 0x7fe7b810ae90 2024-08-20T01:05:39.971 INFO:teuthology.orchestra.run.smithi138.stderr:added data pool 4 to fsmap 2024-08-20T01:05:39.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 msgr2=0x7fe79c0b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 0x7fe79c0b6180 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009bd0 tx=0x7fe7a8005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 msgr2=0x7fe7b81a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe7b002f540 tx=0x7fe7b00316f0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 shutdown_connections 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe79c0b3cc0 0x7fe79c0b6180 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe7b810ae90 0x7fe7b81a07f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe7b810a510 0x7fe7b819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 --2- 172.21.15.138:0/3422441000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7b8104b00 0x7fe7b819af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 >> 172.21.15.138:0/3422441000 conn(0x7fe7b8100550 msgr2=0x7fe7b8100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:39.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 shutdown_connections 2024-08-20T01:05:39.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:39.975+0000 7fe7b7200640 1 -- 172.21.15.138:0/3422441000 wait complete. 2024-08-20T01:05:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:39 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3422441000' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:39 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:39 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3422441000' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:39 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:39 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3422441000' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:39 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} : dispatch 2024-08-20T01:05:40.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:40 smithi142 ceph-mon[21799]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T01:05:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:40 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-20T01:05:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:40 smithi142 ceph-mon[21799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:40 smithi142 ceph-mon[21799]: pgmap v287: 129 pgs: 10 creating+peering, 2 peering, 117 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:41.272 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:41.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:40 smithi138 ceph-mon[18076]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T01:05:41.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:40 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-20T01:05:41.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:40 smithi138 ceph-mon[18076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:41.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:40 smithi138 ceph-mon[18076]: pgmap v287: 129 pgs: 10 creating+peering, 2 peering, 117 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:41.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:40 smithi167 ceph-mon[21965]: osdmap e86: 12 total, 12 up, 12 in 2024-08-20T01:05:41.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:40 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-08-20T01:05:41.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:40 smithi167 ceph-mon[21965]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:05:41.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:40 smithi167 ceph-mon[21965]: pgmap v287: 129 pgs: 10 creating+peering, 2 peering, 117 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:41.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 -- 172.21.15.138:0/1232157924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 msgr2=0x7f5d8010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 --2- 172.21.15.138:0/1232157924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d8010c620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5d68009900 tx=0x7f5d6802a410 comp rx=0 tx=0).stop 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 -- 172.21.15.138:0/1232157924 shutdown_connections 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 --2- 172.21.15.138:0/1232157924 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d8010cc70 0x7f5d8010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 --2- 172.21.15.138:0/1232157924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d8010c620 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 --2- 172.21.15.138:0/1232157924 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d80103810 0x7f5d80103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 -- 172.21.15.138:0/1232157924 >> 172.21.15.138:0/1232157924 conn(0x7f5d800fd2d0 msgr2=0x7f5d800ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 -- 172.21.15.138:0/1232157924 shutdown_connections 2024-08-20T01:05:41.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.836+0000 7f5d7f200640 1 -- 172.21.15.138:0/1232157924 wait complete. 2024-08-20T01:05:41.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.837+0000 7f5d7f200640 1 Processor -- start 2024-08-20T01:05:41.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.837+0000 7f5d7f200640 1 -- start start 2024-08-20T01:05:41.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:41.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d80199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010cc70 0x7f5d8019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80112930 con 0x7f5d80104130 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d801127b0 con 0x7f5d80103810 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d80112ab0 con 0x7f5d8010cc70 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d77e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d80199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56746/0 (socket says 172.21.15.138:56746) 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 -- 172.21.15.138:0/3792997804 learned_addr learned my addr 172.21.15.138:0/3792997804 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7e800640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010cc70 0x7f5d8019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 -- 172.21.15.138:0/3792997804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010cc70 msgr2=0x7f5d8019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010cc70 0x7f5d8019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:41.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 -- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 msgr2=0x7f5d80199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d80199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.838+0000 7f5d7de00640 1 -- 172.21.15.138:0/3792997804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8019ee70 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d7de00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d950 tx=0x7f5d7000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d70017030 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d70004020 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d700047a0 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8019f160 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.839+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d801a5820 con 0x7f5d80103810 2024-08-20T01:05:41.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.840+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d80103bf0 con 0x7f5d80103810 2024-08-20T01:05:41.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.842+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f5d70009a30 con 0x7f5d80103810 2024-08-20T01:05:41.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.844+0000 7f5d76a00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 0x7f5d640b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:41.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.844+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f5d700e40f0 con 0x7f5d80103810 2024-08-20T01:05:41.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.844+0000 7f5d77e00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 0x7f5d640b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:41.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.845+0000 7f5d77e00640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 0x7f5d640b6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d68031440 tx=0x7f5d68005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:41.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:41.846+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5d700ab2e0 con 0x7f5d80103810 2024-08-20T01:05:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:41 smithi138 ceph-mon[18076]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-20T01:05:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:41 smithi138 ceph-mon[18076]: Cluster is now healthy 2024-08-20T01:05:42.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.066+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d80199ef0 con 0x7f5d80103810 2024-08-20T01:05:42.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.067+0000 7f5d76a00640 1 -- 172.21.15.138:0/3792997804 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11860 (secure 0 0 0) 0x7f5d80199ef0 con 0x7f5d80103810 2024-08-20T01:05:42.065 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:42.066 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"btime":"2024-08-20T01:05:39:962817+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24706,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:05:39.935680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24743,"qdb_cluster":[24743]},"id":1}]} 2024-08-20T01:05:42.066 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 16 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 msgr2=0x7f5d640b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 0x7f5d640b6180 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5d68031440 tx=0x7f5d68005a20 comp rx=0 tx=0).stop 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 msgr2=0x7f5d80198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000d950 tx=0x7f5d7000de10 comp rx=0 tx=0).stop 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 shutdown_connections 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d640b3cc0 0x7f5d640b6180 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d8010cc70 0x7f5d8019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d80104130 0x7f5d80199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:42.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 --2- 172.21.15.138:0/3792997804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d80103810 0x7f5d80198dc0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 >> 172.21.15.138:0/3792997804 conn(0x7f5d800fd2d0 msgr2=0x7f5d800fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.069+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 shutdown_connections 2024-08-20T01:05:42.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:42.070+0000 7f5d7f200640 1 -- 172.21.15.138:0/3792997804 wait complete. 2024-08-20T01:05:42.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:41 smithi142 ceph-mon[21799]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-20T01:05:42.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:41 smithi142 ceph-mon[21799]: Cluster is now healthy 2024-08-20T01:05:42.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:41 smithi167 ceph-mon[21965]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-08-20T01:05:42.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:41 smithi167 ceph-mon[21965]: Cluster is now healthy 2024-08-20T01:05:42.966 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:05:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:42 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3792997804' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:42 smithi142 ceph-mon[21799]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:43.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:42 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3792997804' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:43.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:42 smithi138 ceph-mon[18076]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:43.334 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:43.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:42 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3792997804' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:43.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:42 smithi167 ceph-mon[21965]: pgmap v288: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:43.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 -- 172.21.15.138:0/2974960805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 msgr2=0x7ff7400777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 --2- 172.21.15.138:0/2974960805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff7400777e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff734009900 tx=0x7ff73402a410 comp rx=0 tx=0).stop 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 -- 172.21.15.138:0/2974960805 shutdown_connections 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 --2- 172.21.15.138:0/2974960805 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740113500 0x7ff7401158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 --2- 172.21.15.138:0/2974960805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740077d20 0x7ff740112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 --2- 172.21.15.138:0/2974960805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff7400777e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 -- 172.21.15.138:0/2974960805 >> 172.21.15.138:0/2974960805 conn(0x7ff740100540 msgr2=0x7ff740102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 -- 172.21.15.138:0/2974960805 shutdown_connections 2024-08-20T01:05:43.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.948+0000 7ff746800640 1 -- 172.21.15.138:0/2974960805 wait complete. 2024-08-20T01:05:43.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.949+0000 7ff746800640 1 Processor -- start 2024-08-20T01:05:43.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.949+0000 7ff746800640 1 -- start start 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.949+0000 7ff746800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.949+0000 7ff746800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 0x7ff74019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.949+0000 7ff746800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740113500 0x7ff7401a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff746800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff740118f10 con 0x7ff740077400 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff746800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff740118d90 con 0x7ff740113500 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff746800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff740119090 con 0x7ff740077d20 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44044/0 (socket says 172.21.15.138:44044) 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 -- 172.21.15.138:0/98759521 learned_addr learned my addr 172.21.15.138:0/98759521 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff744a00640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 0x7ff74019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:43.951 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745e00640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740113500 0x7ff7401a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 -- 172.21.15.138:0/98759521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 msgr2=0x7ff74019f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 0x7ff74019f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 -- 172.21.15.138:0/98759521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740113500 msgr2=0x7ff7401a4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740113500 0x7ff7401a4c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff745400640 1 -- 172.21.15.138:0/98759521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7401a53f0 con 0x7ff740077400 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.950+0000 7ff744a00640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 0x7ff74019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff745400640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff7340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73403f030 con 0x7ff740077400 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff734003cb0 con 0x7ff740077400 2024-08-20T01:05:43.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff734035070 con 0x7ff740077400 2024-08-20T01:05:43.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7401a5680 con 0x7ff740077400 2024-08-20T01:05:43.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.951+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7401abda0 con 0x7ff740077400 2024-08-20T01:05:43.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.952+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7400777e0 con 0x7ff740077400 2024-08-20T01:05:43.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.954+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7ff734042030 con 0x7ff740077400 2024-08-20T01:05:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.957+0000 7ff73e000640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 0x7ff7280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.957+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7ff734104b60 con 0x7ff740077400 2024-08-20T01:05:43.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.957+0000 7ff744a00640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 0x7ff7280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:43.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.958+0000 7ff744a00640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 0x7ff7280b6200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0058c0 tx=0x7ff72c005850 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:43.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:43.959+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff7340cbd50 con 0x7ff740077400 2024-08-20T01:05:44.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.147+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7401ac000 con 0x7ff740077400 2024-08-20T01:05:44.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.149+0000 7ff73e000640 1 -- 172.21.15.138:0/98759521 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25045 (secure 0 0 0) 0x7ff7340cb650 con 0x7ff740077400 2024-08-20T01:05:44.148 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:44.149 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:05:39.935470+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.151+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 msgr2=0x7ff7280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 0x7ff7280b6200 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0058c0 tx=0x7ff72c005850 comp rx=0 tx=0).stop 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 msgr2=0x7ff74019f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff734005940 tx=0x7ff7340042f0 comp rx=0 tx=0).stop 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 shutdown_connections 2024-08-20T01:05:44.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7280b3d40 0x7ff7280b6200 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff740113500 0x7ff7401a4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff740077d20 0x7ff74019f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 --2- 172.21.15.138:0/98759521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff740077400 0x7ff74019f340 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 >> 172.21.15.138:0/98759521 conn(0x7ff740100540 msgr2=0x7ff740102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 shutdown_connections 2024-08-20T01:05:44.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:44.152+0000 7ff746800640 1 -- 172.21.15.138:0/98759521 wait complete. 2024-08-20T01:05:44.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:44 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/98759521' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:44.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:44 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/98759521' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:44.506 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:44 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/98759521' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:44.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:05:45.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:45 smithi138 ceph-mon[18076]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:45.371 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:45.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:45 smithi167 ceph-mon[21965]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:45.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:45 smithi142 ceph-mon[21799]: pgmap v289: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:45.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.951+0000 7f1a6b200640 1 -- 172.21.15.138:0/4101874488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108330 msgr2=0x7f1a6c108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:45.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.951+0000 7f1a6b200640 1 --2- 172.21.15.138:0/4101874488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108330 0x7f1a6c108770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009900 tx=0x7f1a5402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 -- 172.21.15.138:0/4101874488 shutdown_connections 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 --2- 172.21.15.138:0/4101874488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108cb0 0x7f1a6c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 --2- 172.21.15.138:0/4101874488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108330 0x7f1a6c108770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 --2- 172.21.15.138:0/4101874488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 0x7f1a6c102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 -- 172.21.15.138:0/4101874488 >> 172.21.15.138:0/4101874488 conn(0x7f1a6c0fe350 msgr2=0x7f1a6c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.952+0000 7f1a6b200640 1 -- 172.21.15.138:0/4101874488 shutdown_connections 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.953+0000 7f1a6b200640 1 -- 172.21.15.138:0/4101874488 wait complete. 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.953+0000 7f1a6b200640 1 Processor -- start 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 -- start start 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 0x7f1a6c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:45.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108cb0 0x7f1a6c19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c1129c0 con 0x7f1a6c108cb0 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c112840 con 0x7f1a6c102920 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a6c112b40 con 0x7f1a6c108330 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a61400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a69e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 0x7f1a6c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a61400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54266/0 (socket says 172.21.15.138:54266) 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a61400640 1 -- 172.21.15.138:0/3382705564 learned_addr learned my addr 172.21.15.138:0/3382705564 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.954+0000 7f1a6a800640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108cb0 0x7f1a6c19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 -- 172.21.15.138:0/3382705564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 msgr2=0x7f1a6c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 0x7f1a6c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 -- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108cb0 msgr2=0x7f1a6c19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:45.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108cb0 0x7f1a6c19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 -- 172.21.15.138:0/3382705564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6c19ee20 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a61400640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a54031440 tx=0x7f1a54003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5403f070 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a54031b60 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6c19f0b0 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a54035030 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.955+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a6c1a57d0 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.956+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6c102d00 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.958+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f1a54042070 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.960+0000 7f1a63400640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 0x7f1a500b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.960+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f1a54104a10 con 0x7f1a6c108330 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.960+0000 7f1a69e00640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 0x7f1a500b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.960+0000 7f1a69e00640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 0x7f1a500b6200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009bd0 tx=0x7f1a5c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:45.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:45.962+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1a540cbc00 con 0x7f1a6c108330 2024-08-20T01:05:46.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.145+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1a6c199ea0 con 0x7f1a6c108330 2024-08-20T01:05:46.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.147+0000 7f1a63400640 1 -- 172.21.15.138:0/3382705564 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25045 (secure 0 0 0) 0x7f1a6c199ea0 con 0x7f1a6c108330 2024-08-20T01:05:46.146 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:46.147 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:05:39.935470+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 msgr2=0x7f1a500b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 0x7f1a500b6200 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c009bd0 tx=0x7f1a5c005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 msgr2=0x7f1a6c1992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1a54031440 tx=0x7f1a54003c10 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 shutdown_connections 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a500b3d40 0x7f1a500b6200 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a6c108cb0 0x7f1a6c19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a6c108330 0x7f1a6c1992b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 --2- 172.21.15.138:0/3382705564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a6c102920 0x7f1a6c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.149+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 >> 172.21.15.138:0/3382705564 conn(0x7f1a6c0fe350 msgr2=0x7f1a6c101880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.150+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 shutdown_connections 2024-08-20T01:05:46.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:46.150+0000 7f1a6b200640 1 -- 172.21.15.138:0/3382705564 wait complete. 2024-08-20T01:05:46.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:46 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3382705564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:46.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:46 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3382705564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:46.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:46 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3382705564' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:47.086 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:05:47.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:47 smithi138 ceph-mon[18076]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:47.462 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:47.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:47 smithi142 ceph-mon[21799]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:47.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:47 smithi167 ceph-mon[21965]: pgmap v290: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 -- 172.21.15.138:0/1407787626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc102900 msgr2=0x7ff1cc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 --2- 172.21.15.138:0/1407787626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc102900 0x7ff1cc102ce0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0009900 tx=0x7ff1c002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 -- 172.21.15.138:0/1407787626 shutdown_connections 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 --2- 172.21.15.138:0/1407787626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc108c90 0x7ff1cc10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 --2- 172.21.15.138:0/1407787626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 0x7ff1cc108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 --2- 172.21.15.138:0/1407787626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc102900 0x7ff1cc102ce0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 -- 172.21.15.138:0/1407787626 >> 172.21.15.138:0/1407787626 conn(0x7ff1cc0fe390 msgr2=0x7ff1cc1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 -- 172.21.15.138:0/1407787626 shutdown_connections 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.089+0000 7ff1d2600640 1 -- 172.21.15.138:0/1407787626 wait complete. 2024-08-20T01:05:48.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 Processor -- start 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 -- start start 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 0x7ff1cc199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc1129a0 con 0x7ff1cc108c90 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc112820 con 0x7ff1cc108310 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.091+0000 7ff1d2600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc112b20 con 0x7ff1cc102900 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:51634/0 (socket says 172.21.15.138:51634) 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 -- 172.21.15.138:0/2512250317 learned_addr learned my addr 172.21.15.138:0/2512250317 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:48.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42220/0 (socket says 172.21.15.138:42220) 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1cbe00640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 0x7ff1cc199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 -- 172.21.15.138:0/2512250317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 msgr2=0x7ff1cc199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 0x7ff1cc199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 -- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 msgr2=0x7ff1cc19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 -- 172.21.15.138:0/2512250317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc19ede0 con 0x7ff1cc102900 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1c00640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.092+0000 7ff1d1200640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0005ab0 tx=0x7ff1c0003cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.093+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c003f070 con 0x7ff1cc102900 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.093+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc19f070 con 0x7ff1cc102900 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.093+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1c0031d60 con 0x7ff1cc102900 2024-08-20T01:05:48.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.093+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c0035030 con 0x7ff1cc102900 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.093+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1a5790 con 0x7ff1cc102900 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.094+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1cc102ce0 con 0x7ff1cc102900 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.096+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7ff1c0042070 con 0x7ff1cc102900 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.098+0000 7ff1c9600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 0x7ff1b40b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.098+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7ff1c0104f20 con 0x7ff1cc102900 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.098+0000 7ff1cbe00640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 0x7ff1b40b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.098+0000 7ff1cbe00640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 0x7ff1b40b6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1017d0 tx=0x7ff1b800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:48.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.100+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff1c00cbac0 con 0x7ff1cc102900 2024-08-20T01:05:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.320+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1cc1a59f0 con 0x7ff1cc102900 2024-08-20T01:05:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.321+0000 7ff1c9600640 1 -- 172.21.15.138:0/2512250317 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+11860 (secure 0 0 0) 0x7ff1c00d0010 con 0x7ff1cc102900 2024-08-20T01:05:48.321 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:48.321 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":16,"btime":"2024-08-20T01:05:39:962817+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24674,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24679,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24694,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24704,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24706,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:05:39.935680+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24743,"qdb_cluster":[24743]},"id":1}]} 2024-08-20T01:05:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 16 2024-08-20T01:05:48.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 msgr2=0x7ff1b40b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 0x7ff1b40b6180 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc1017d0 tx=0x7ff1b800b390 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 msgr2=0x7ff1cc198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0005ab0 tx=0x7ff1c0003cd0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 shutdown_connections 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1b40b3cc0 0x7ff1b40b6180 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1cc108c90 0x7ff1cc19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1cc108310 0x7ff1cc199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 --2- 172.21.15.138:0/2512250317 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1cc102900 0x7ff1cc198d30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 >> 172.21.15.138:0/2512250317 conn(0x7ff1cc0fe390 msgr2=0x7ff1cc0ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 shutdown_connections 2024-08-20T01:05:48.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:48.324+0000 7ff1d2600640 1 -- 172.21.15.138:0/2512250317 wait complete. 2024-08-20T01:05:48.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:48 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2512250317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:48.718 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:48 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2512250317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:48.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:48 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2512250317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:05:49.100 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:05:49.469 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:49.645 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:49 smithi138 ceph-mon[18076]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:49.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:49 smithi167 ceph-mon[21965]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:49.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:49 smithi142 ceph-mon[21799]: pgmap v291: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:50.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 -- 172.21.15.138:0/848171710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 msgr2=0x7f30e010f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:50.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 --2- 172.21.15.138:0/848171710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e010f4b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30d8009ca0 tx=0x7f30d802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 -- 172.21.15.138:0/848171710 shutdown_connections 2024-08-20T01:05:50.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 --2- 172.21.15.138:0/848171710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e010f4b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 --2- 172.21.15.138:0/848171710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0108330 0x7f30e0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 --2- 172.21.15.138:0/848171710 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0102920 0x7f30e0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 -- 172.21.15.138:0/848171710 >> 172.21.15.138:0/848171710 conn(0x7f30e00fe350 msgr2=0x7f30e0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:50.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 -- 172.21.15.138:0/848171710 shutdown_connections 2024-08-20T01:05:50.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.042+0000 7f30e5e00640 1 -- 172.21.15.138:0/848171710 wait complete. 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.043+0000 7f30e5e00640 1 Processor -- start 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.043+0000 7f30e5e00640 1 -- start start 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0108330 0x7f30e019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e01a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0112980 con 0x7f30e0108cb0 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0112800 con 0x7f30e0102920 2024-08-20T01:05:50.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f30e0112b00 con 0x7f30e0108330 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55348/0 (socket says 172.21.15.138:55348) 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 -- 172.21.15.138:0/3417187904 learned_addr learned my addr 172.21.15.138:0/3417187904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e5400640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e01a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30dfe00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0108330 0x7f30e019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 -- 172.21.15.138:0/3417187904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0108330 msgr2=0x7f30e019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0108330 0x7f30e019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 -- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 msgr2=0x7f30e01a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.044+0000 7f30e4a00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e01a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30e4a00640 1 -- 172.21.15.138:0/3417187904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e01a0fb0 con 0x7f30e0102920 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30e4a00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30d400d770 tx=0x7f30d400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:50.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4017070 con 0x7f30e0102920 2024-08-20T01:05:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e01a12a0 con 0x7f30e0102920 2024-08-20T01:05:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30d40054d0 con 0x7f30e0102920 2024-08-20T01:05:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30d4004d40 con 0x7f30e0102920 2024-08-20T01:05:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.045+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e01a7960 con 0x7f30e0102920 2024-08-20T01:05:50.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.046+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e0102d00 con 0x7f30e0102920 2024-08-20T01:05:50.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.049+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f30d4004ee0 con 0x7f30e0102920 2024-08-20T01:05:50.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.050+0000 7f30dd600640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 0x7f30c80b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:50.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.051+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f30d40e3ff0 con 0x7f30e0102920 2024-08-20T01:05:50.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.051+0000 7f30dfe00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 0x7f30c80b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:50.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.051+0000 7f30dfe00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 0x7f30c80b6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30e00771b0 tx=0x7f30cc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:50.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.052+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f30d40ab240 con 0x7f30e0102920 2024-08-20T01:05:50.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.238+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30e01a7c80 con 0x7f30e0102920 2024-08-20T01:05:50.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.239+0000 7f30dd600640 1 -- 172.21.15.138:0/3417187904 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25045 (secure 0 0 0) 0x7f30d40aab40 con 0x7f30e0102920 2024-08-20T01:05:50.239 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:50.239 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:05:39.935470+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:05:50.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 msgr2=0x7f30c80b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:50.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 0x7f30c80b6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f30e00771b0 tx=0x7f30cc005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:50.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 msgr2=0x7f30e019af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f30d400d770 tx=0x7f30d400dc30 comp rx=0 tx=0).stop 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 shutdown_connections 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30c80b3cc0 0x7f30c80b6180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30e0108cb0 0x7f30e01a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f30e0108330 0x7f30e019b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 --2- 172.21.15.138:0/3417187904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30e0102920 0x7f30e019af00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:50.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.242+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 >> 172.21.15.138:0/3417187904 conn(0x7f30e00fe350 msgr2=0x7f30e00750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:50.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.243+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 shutdown_connections 2024-08-20T01:05:50.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:50.243+0000 7f30e5e00640 1 -- 172.21.15.138:0/3417187904 wait complete. 2024-08-20T01:05:50.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3417187904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:50.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:50 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3417187904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:50.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:50 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3417187904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:05:51.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-08-20T01:05:27.648119+0000', 'modified': '2024-08-20T01:05:39.935680+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.142:6839/80285616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 80285616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 80285616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24743, 'qdb_cluster': [24743]} 2024-08-20T01:05:51.072 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-08-20T01:05:51.072 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T01:05:51.443 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:51.665 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:51 smithi138 ceph-mon[18076]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:51.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:51 smithi167 ceph-mon[21965]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:51.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:51 smithi142 ceph-mon[21799]: pgmap v292: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:52.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 -- 172.21.15.138:0/2118814926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 msgr2=0x7fbd04108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 --2- 172.21.15.138:0/2118814926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd04108760 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009900 tx=0x7fbcec02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 -- 172.21.15.138:0/2118814926 shutdown_connections 2024-08-20T01:05:52.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 --2- 172.21.15.138:0/2118814926 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04108ca0 0x7fbd0410f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 --2- 172.21.15.138:0/2118814926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd04108760 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 --2- 172.21.15.138:0/2118814926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04102910 0x7fbd04102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 -- 172.21.15.138:0/2118814926 >> 172.21.15.138:0/2118814926 conn(0x7fbd040fe3c0 msgr2=0x7fbd041007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 -- 172.21.15.138:0/2118814926 shutdown_connections 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.075+0000 7fbd03200640 1 -- 172.21.15.138:0/2118814926 wait complete. 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 Processor -- start 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 -- start start 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 0x7fbd0419af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04108ca0 0x7fbd041a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbd04112970 con 0x7fbd04108320 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbd041127f0 con 0x7fbd04108ca0 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.076+0000 7fbd03200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbd04112af0 con 0x7fbd04102910 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42250/0 (socket says 172.21.15.138:42250) 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 0x7fbd0419af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 -- 172.21.15.138:0/2739807462 learned_addr learned my addr 172.21.15.138:0/2739807462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbd02800640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04108ca0 0x7fbd041a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 -- 172.21.15.138:0/2739807462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 msgr2=0x7fbd0419af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 0x7fbd0419af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 -- 172.21.15.138:0/2739807462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04108ca0 msgr2=0x7fbd041a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04108ca0 0x7fbd041a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbcfbe00640 1 -- 172.21.15.138:0/2739807462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd041a1030 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.077+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 0x7fbd0419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005ab0 tx=0x7fbcec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbcfaa00640 1 -- 172.21.15.138:0/2739807462 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec03f070 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbcfaa00640 1 -- 172.21.15.138:0/2739807462 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec031b60 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbcfaa00640 1 -- 172.21.15.138:0/2739807462 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec035030 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd041a12c0 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.078+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd041a79e0 con 0x7fbd04108320 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.082+0000 7fbd0b40e740 1 Processor -- start 2024-08-20T01:05:52.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.082+0000 7fbd0b40e740 1 -- start start 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.082+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.082+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.082+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0164a20 0x5610b0166e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610b00ce840 con 0x5610b0158170 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5610b00bdd40 con 0x5610b0158a90 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5610b00c1d80 con 0x5610b0164a20 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbcfaa00640 1 -- 172.21.15.138:0/2739807462 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fbcec042070 con 0x7fbd04108320 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd02800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd02800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42260/0 (socket says 172.21.15.138:42260) 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55366/0 (socket says 172.21.15.138:55366) 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd02800640 1 -- 172.21.15.138:0/2445423839 learned_addr learned my addr 172.21.15.138:0/2445423839 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0164a20 0x5610b0166e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 -- 172.21.15.138:0/2445423839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0164a20 msgr2=0x5610b0166e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0164a20 0x5610b0166e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 -- 172.21.15.138:0/2445423839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 msgr2=0x5610b0158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd01e00640 1 -- 172.21.15.138:0/2445423839 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b00641c0 con 0x5610b0158a90 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.083+0000 7fbd02800640 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.084+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009fd0 tx=0x7fbcf40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d7569e0d928bbc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.084+0000 7fbcf3000640 1 -- 172.21.15.138:0/2445423839 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004020 con 0x5610b0158a90 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.084+0000 7fbcf3000640 1 -- 172.21.15.138:0/2445423839 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf40054d0 con 0x5610b0158a90 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.084+0000 7fbcf3000640 1 -- 172.21.15.138:0/2445423839 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4004c00 con 0x5610b0158a90 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.084+0000 7fbcfaa00640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 0x7fbce80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2445423839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 msgr2=0x5610b01644e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009fd0 tx=0x7fbcf40118d0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbcfaa00640 1 -- 172.21.15.138:0/2739807462 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fbcec104710 con 0x7fbd04108320 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd02800640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 0x7fbce80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2445423839 shutdown_connections 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0164a20 0x5610b0166e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b01644e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2445423839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158170 0x5610b0158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2445423839 >> 172.21.15.138:0/2445423839 conn(0x5610b0155fa0 msgr2=0x5610b0157140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2445423839 shutdown_connections 2024-08-20T01:05:52.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2445423839 wait complete. 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd02800640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 0x7fbce80b6200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009de0 tx=0x7fbcfc009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.085+0000 7fbd0b40e740 1 -- start start 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 Processor -- start 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b0161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b0161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610aff205f0 con 0x5610b0164a20 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5610b0059170 con 0x5610b0158a90 2024-08-20T01:05:52.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5610b01676d0 con 0x5610b0158170 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd0b40e740 10 client.0 _ll_register_callbacks cb 0x5610b00b88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:51684/0 (socket says 172.21.15.138:51684) 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd01e00640 1 -- 172.21.15.138:0/1082748578 learned_addr learned my addr 172.21.15.138:0/1082748578 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.086+0000 7fbd02800640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b0161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b0161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 -- 172.21.15.138:0/1082748578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 msgr2=0x5610b01641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 -- 172.21.15.138:0/1082748578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 msgr2=0x5610b0161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b0161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 -- 172.21.15.138:0/1082748578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610afe1d540 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbd01e00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b0161620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5610b0158120 tx=0x7fbcec009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd7ebf4ac289ce4f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbce7400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec049070 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf40054d0 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.087+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fbcec104370 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fbcec0cb2b0 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 10 client.24839 fetch_fsmap learned FSMap version 16 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5610b0172ef0 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcf40054d0 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7fbcec0e2ce0 con 0x5610b0158a90 2024-08-20T01:05:52.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 10 client.24839 fetch_fsmap finished waiting for FSMap version 16 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 10 client.24839 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 20 client.24839 populate_metadata read hostname 'smithi138' 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01754f0 0x5610b0176960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 4 client.24839 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610b00d0790 con 0x5610b01754f0 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.088+0000 7fbd01e00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01754f0 0x5610b0176960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.089+0000 7fbd01e00640 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01754f0 0x5610b0176960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13921884eac77b2c server_cookie=e006618605c37aaa in_seq=0 out_seq=0 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.089+0000 7fbce7400640 10 client.24839 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbce7400640 1 -- 172.21.15.138:0/1082748578 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5610b00d0790 con 0x5610b01754f0 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbce7400640 10 client.24839 handle_command_reply: tid=0 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 client.24839 shutdown 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 msgr2=0x5610b0161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b0161620 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x5610b0158120 tx=0x7fbcec009260 comp rx=0 tx=0).stop 2024-08-20T01:05:52.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 shutdown_connections 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01754f0 0x5610b0176960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b0161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158a90 0x5610b0161620 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbce7400640 0 client.24839 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/1082748578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b01641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.090+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 >> 172.21.15.138:0/1082748578 conn(0x5610b0155fa0 msgr2=0x5610b015bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.091+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 shutdown_connections 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.091+0000 7fbd0b40e740 1 -- 172.21.15.138:0/1082748578 wait complete. 2024-08-20T01:05:52.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.091+0000 7fbd0b40e740 20 client.24839 trim_cache size 0 max 16384 2024-08-20T01:05:52.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.109+0000 7fbd0b40e740 1 Processor -- start 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.109+0000 7fbd0b40e740 1 -- start start 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b0160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 0x5610b0161910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610b0174ca0 con 0x5610b0158a90 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5610b0175370 con 0x5610b0164a20 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5610b0175a40 con 0x5610b0158170 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42272/0 (socket says 172.21.15.138:42272) 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 -- 172.21.15.138:0/105609901 learned_addr learned my addr 172.21.15.138:0/105609901 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd01e00640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b0160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd02800640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 0x5610b0161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 -- 172.21.15.138:0/105609901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 msgr2=0x5610b0160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b0160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 -- 172.21.15.138:0/105609901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 msgr2=0x5610b0161910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 0x5610b0161910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbcfbe00640 1 -- 172.21.15.138:0/105609901 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610b0156a20 con 0x5610b0158a90 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.110+0000 7fbd02800640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 0x5610b0161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.111+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0cb240 tx=0x7fbcec0cb740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cae41ed5643221 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.111+0000 7fbcf2600640 1 -- 172.21.15.138:0/105609901 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec049070 con 0x5610b0158a90 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.111+0000 7fbcf2600640 1 -- 172.21.15.138:0/105609901 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec0e9020 con 0x5610b0158a90 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.111+0000 7fbcf2600640 1 -- 172.21.15.138:0/105609901 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec104370 con 0x5610b0158a90 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- 172.21.15.138:0/105609901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 msgr2=0x5610b01613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0cb240 tx=0x7fbcec0cb740 comp rx=0 tx=0).stop 2024-08-20T01:05:52.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- 172.21.15.138:0/105609901 shutdown_connections 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0164a20 0x5610b0161910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0158a90 0x5610b01613d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/105609901 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158170 0x5610b0160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- 172.21.15.138:0/105609901 >> 172.21.15.138:0/105609901 conn(0x5610b0155fa0 msgr2=0x5610b015fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- 172.21.15.138:0/105609901 shutdown_connections 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- 172.21.15.138:0/105609901 wait complete. 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 -- start start 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.112+0000 7fbd0b40e740 1 Processor -- start 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 0x5610b017a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b017ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5610b01767e0 con 0x5610b0164a20 2024-08-20T01:05:52.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5610b0176110 con 0x5610b0158170 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5610b0176eb0 con 0x5610b0158a90 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd0b40e740 10 client.0 _ll_register_callbacks cb 0x7fbcec005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd02800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b017ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd01e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55400/0 (socket says 172.21.15.138:55400) 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbd01e00640 1 -- 172.21.15.138:0/2734108761 learned_addr learned my addr 172.21.15.138:0/2734108761 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.113+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 0x5610b017a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 -- 172.21.15.138:0/2734108761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 msgr2=0x5610b017a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 0x5610b017a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 -- 172.21.15.138:0/2734108761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 msgr2=0x5610b017ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b017ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 -- 172.21.15.138:0/2734108761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610b017b1b0 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 0x5610b017a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd01e00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4004610 tx=0x7fbcf4005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ec0c4798845a7a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbce7400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4032060 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fbcf4016040 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcec0e9020 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fbcf4039050 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd0b40e740 10 client.24845 fetch_fsmap learned FSMap version 16 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.114+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=4+}) -- 0x5610b01dfa90 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec0e9020 con 0x5610b0158170 2024-08-20T01:05:52.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 16) ==== 8350+0+0 (secure 0 0 0) 0x7fbcf4026b10 con 0x5610b0158170 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 10 client.24845 fetch_fsmap finished waiting for FSMap version 16 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 10 client.24845 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 20 client.24845 populate_metadata read hostname 'smithi138' 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01e44f0 0x5610b01e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 4 client.24845 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "status"}) -- 0x5610b00cc230 con 0x5610b01e44f0 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.115+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01e44f0 0x5610b01e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbcfbe00640 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01e44f0 0x5610b01e3200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bfc973098638e97 server_cookie=40628c7d0a625f2 in_seq=0 out_seq=0 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbce7400640 10 client.24845 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbce7400640 1 -- 172.21.15.138:0/2734108761 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5610b00cc230 con 0x5610b01e44f0 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbce7400640 10 client.24845 handle_command_reply: tid=0 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 client.24845 shutdown 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 msgr2=0x5610b017f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4004610 tx=0x7fbcf4005db0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 shutdown_connections 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5610b01e44f0 0x5610b01e3200 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5610b0164a20 0x5610b017ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5610b0158a90 0x5610b017a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 --2- 172.21.15.138:0/2734108761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610b0158170 0x5610b017f0e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbce7400640 0 client.24845 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 >> 172.21.15.138:0/2734108761 conn(0x5610b0155fa0 msgr2=0x5610b01613d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 shutdown_connections 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 1 -- 172.21.15.138:0/2734108761 wait complete. 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.116+0000 7fbd0b40e740 20 client.24845 trim_cache size 0 max 16384 2024-08-20T01:05:52.133 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24743, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 23.511737607000001, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 16, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 86, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 37.907628181 2024-08-20T01:05:52.134 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 msgr2=0x7fbce80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 0x7fbce80b6200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc009de0 tx=0x7fbcfc009500 comp rx=0 tx=0).stop 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 msgr2=0x7fbd0419b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbcec005ab0 tx=0x7fbcec003c10 comp rx=0 tx=0).stop 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 shutdown_connections 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbce80b3d40 0x7fbce80b6200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd04108ca0 0x7fbd041a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd04108320 0x7fbd0419b4c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 --2- 172.21.15.138:0/2739807462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbd04102910 0x7fbd0419af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:52.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 >> 172.21.15.138:0/2739807462 conn(0x7fbd040fe3c0 msgr2=0x7fbd04105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:52.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 shutdown_connections 2024-08-20T01:05:52.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:52.117+0000 7fbd03200640 1 -- 172.21.15.138:0/2739807462 wait complete. 2024-08-20T01:05:53.067 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-20T01:05:53.067 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:53.068 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-20T01:05:53.435 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:05:53.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:53 smithi138 ceph-mon[18076]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:53 smithi167 ceph-mon[21965]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:53 smithi142 ceph-mon[21799]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 -- 172.21.15.138:0/3148165463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d810a6b0 msgr2=0x7ff3d810aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3148165463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d810a6b0 0x7ff3d810aaf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 -- 172.21.15.138:0/3148165463 shutdown_connections 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3148165463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3148165463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d810a6b0 0x7ff3d810aaf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3148165463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d8104ca0 0x7ff3d8105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 -- 172.21.15.138:0/3148165463 >> 172.21.15.138:0/3148165463 conn(0x7ff3d8100730 msgr2=0x7ff3d8102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 -- 172.21.15.138:0/3148165463 shutdown_connections 2024-08-20T01:05:54.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.056+0000 7ff3dee00640 1 -- 172.21.15.138:0/3148165463 wait complete. 2024-08-20T01:05:54.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 Processor -- start 2024-08-20T01:05:54.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 -- start start 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d810a6b0 0x7ff3d819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d81a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8114cf0 con 0x7ff3d8104ca0 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8114b70 con 0x7ff3d810b030 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.058+0000 7ff3dee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d8114e70 con 0x7ff3d810a6b0 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42282/0 (socket says 172.21.15.138:42282) 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d81a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:54.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3d7e00640 1 -- 172.21.15.138:0/3707791558 learned_addr learned my addr 172.21.15.138:0/3707791558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:05:54.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3cfe00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d810a6b0 0x7ff3d819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:54.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 -- 172.21.15.138:0/3707791558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d810a6b0 msgr2=0x7ff3d819b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:54.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d810a6b0 0x7ff3d819b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 -- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 msgr2=0x7ff3d819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 -- 172.21.15.138:0/3707791558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d81a1250 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3d7e00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.059+0000 7ff3dce00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d81a0a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009fd0 tx=0x7ff3d000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.060+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000ce20 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.060+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d81a1540 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.060+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3d0010430 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.060+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000f620 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.060+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d81a7c00 con 0x7ff3d810b030 2024-08-20T01:05:54.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.061+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d8105080 con 0x7ff3d810b030 2024-08-20T01:05:54.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.063+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7ff3d00108e0 con 0x7ff3d810b030 2024-08-20T01:05:54.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.064+0000 7ff3d5600640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 0x7ff3bc0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:05:54.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.064+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7ff3d00e4810 con 0x7ff3d810b030 2024-08-20T01:05:54.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.065+0000 7ff3d7e00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 0x7ff3bc0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:05:54.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.065+0000 7ff3d7e00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 0x7ff3bc0b60e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8011500 tx=0x7ff3c8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:05:54.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.066+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff3d00aba00 con 0x7ff3d810b030 2024-08-20T01:05:54.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.288+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3d819c2d0 con 0x7ff3d810b030 2024-08-20T01:05:54.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.289+0000 7ff3d5600640 1 -- 172.21.15.138:0/3707791558 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+88 (secure 0 0 0) 0x7ff3d00ab300 con 0x7ff3d810b030 2024-08-20T01:05:54.288 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.291+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 msgr2=0x7ff3bc0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.291+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 0x7ff3bc0b60e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8011500 tx=0x7ff3c8005f90 comp rx=0 tx=0).stop 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 msgr2=0x7ff3d81a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d81a0a80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009fd0 tx=0x7ff3d000ef10 comp rx=0 tx=0).stop 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 shutdown_connections 2024-08-20T01:05:54.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3bc0b3c20 0x7ff3bc0b60e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff3d810b030 0x7ff3d81a0a80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff3d810a6b0 0x7ff3d819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 --2- 172.21.15.138:0/3707791558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3d8104ca0 0x7ff3d819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 >> 172.21.15.138:0/3707791558 conn(0x7ff3d8100730 msgr2=0x7ff3d8102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 shutdown_connections 2024-08-20T01:05:54.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:54.292+0000 7ff3dee00640 1 -- 172.21.15.138:0/3707791558 wait complete. 2024-08-20T01:05:54.778 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:54 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3707791558' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:05:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:54 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3707791558' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:05:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:54 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3707791558' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:05:55.193 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-08-20T01:05:55.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-20T01:05:55.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-08-20T01:05:55.193 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-08-20T01:05:55.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:55.194 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-08-20T01:05:55.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-20T01:05:55.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-20T01:05:55.194 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-08-20T01:05:55.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:55.194 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:05:55.231 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42273sec preferred_lft 42273sec 2024-08-20T01:05:55.232 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-08-20T01:05:55.233 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:05:55.233 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-20T01:05:55.233 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:05:55.233 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:05:55.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-08-20T01:05:55.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-08-20T01:05:55.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-08-20T01:05:55.234 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-20T01:05:55.234 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:05:55.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:05:55.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:55.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:55.663 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-20T01:05:55.718 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-08-20T01:05:55.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:55.722 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-08-20T01:05:55.764 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-08-20T01:05:55.764 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-20T01:05:55.764 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-20T01:05:55.764 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-20T01:05:55.765 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-20T01:05:55.765 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:05:55.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:55 smithi138 ceph-mon[18076]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:55.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:05:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:55 smithi167 ceph-mon[21965]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:55.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:55 smithi142 ceph-mon[21799]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:56.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:56.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:56.427 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-08-20T01:05:56.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:56.463 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-08-20T01:05:56.539 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:05:56.539 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:05:56.539 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:56.539 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-08-20T01:05:56.539 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:05:56.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:05:56.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:56.718 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-08-20T01:05:56.718 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:05:56.718 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:05:56.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-08-20T01:05:56.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-20T01:05:56.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-08-20T01:05:56.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-08-20T01:05:56.719 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-08-20T01:05:56.719 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:05:56.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:05:56.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:56.988 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:05:56.988 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:05:56.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-08-20T01:05:56.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-08-20T01:05:56.988 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:05:57.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:05:57.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:57.154 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-08-20T01:05:57.154 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T01:05:57.154 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:57.189 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-08-20T01:05:57.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T01:05:57.190 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:57.265 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-08-20T01:05:57.363 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-08-20T01:05:57.399 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T01:05:57.400 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/18f6ff62fcdfdb79a5d0c30f1e856912cf77ee1e0ee3d8d884421e18158b0e78/merged 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/4568db797d37429f01e646a419a41870422bbe380a92bfd07f892c9a79981862/merged 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cde4c2c6df86ef46ee8e8652aa29a3f8a4fc1c6f486594c967793a60d3b24213/merged 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/f06bd98fbb975b0d7504127c79eead766c35d866cbdbf6f113f0b52189f7094c/merged 2024-08-20T01:05:57.401 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cd35298e3ad7ed753ec742f7e023025da2139a729f917a9a5ad5c48fad09e289/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/157c5e29dfd19dffbab1ccc58a8a71d8a9746153a85878d1a431f82abd5adfa8/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/583d6c65cd79218ee97d8af8abde56940e3f4e7532bfdbc257562dd97d44f1f0/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cf1cc85bd01986fe32663c3b32ee2f5ff036b45479efa1928036dae1b7262326/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/6923df3f1873a6ff64588e8156ec5e2c99e6eb23e6f8fb72a33ea4acd81a3419/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/b4998db6ac61c4ccb2179407c008a9aac7e9a6db7b3f2bdd54f48897874aa2d5/merged 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:57.402 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:57.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:57.403 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-08-20T01:05:57.479 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-08-20T01:05:57.479 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-08-20T01:05:57.523 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-08-20T01:05:57.581 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T01:05:57.618 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-08-20T01:05:57.618 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-08-20T01:05:57.618 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-08-20T01:05:57.618 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-08-20T01:05:57.619 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/18f6ff62fcdfdb79a5d0c30f1e856912cf77ee1e0ee3d8d884421e18158b0e78/merged 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/4568db797d37429f01e646a419a41870422bbe380a92bfd07f892c9a79981862/merged 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-08-20T01:05:57.620 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cde4c2c6df86ef46ee8e8652aa29a3f8a4fc1c6f486594c967793a60d3b24213/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/f06bd98fbb975b0d7504127c79eead766c35d866cbdbf6f113f0b52189f7094c/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cd35298e3ad7ed753ec742f7e023025da2139a729f917a9a5ad5c48fad09e289/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/157c5e29dfd19dffbab1ccc58a8a71d8a9746153a85878d1a431f82abd5adfa8/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/583d6c65cd79218ee97d8af8abde56940e3f4e7532bfdbc257562dd97d44f1f0/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cf1cc85bd01986fe32663c3b32ee2f5ff036b45479efa1928036dae1b7262326/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/6923df3f1873a6ff64588e8156ec5e2c99e6eb23e6f8fb72a33ea4acd81a3419/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/b4998db6ac61c4ccb2179407c008a9aac7e9a6db7b3f2bdd54f48897874aa2d5/merged 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:57.621 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:57.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:57.622 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-08-20T01:05:57.696 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi138.stderr:2024-08-20T01:05:57.697+0000 7fb8f8d07480 -1 init, newargv = 0x561b44fd7f60 newargc=15 2024-08-20T01:05:57.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi138.stderr:ceph-fuse[66673]: starting ceph client 2024-08-20T01:05:57.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi138.stderr:ceph-fuse[66673]: starting fuse 2024-08-20T01:05:57.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:57 smithi138 ceph-mon[18076]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:57 smithi167 ceph-mon[21965]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:57 smithi142 ceph-mon[21799]: pgmap v295: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:58.803 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-08-20T01:05:58.895 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-08-20T01:05:58.933 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-08-20T01:05:58.934 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/18f6ff62fcdfdb79a5d0c30f1e856912cf77ee1e0ee3d8d884421e18158b0e78/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/4568db797d37429f01e646a419a41870422bbe380a92bfd07f892c9a79981862/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cde4c2c6df86ef46ee8e8652aa29a3f8a4fc1c6f486594c967793a60d3b24213/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/f06bd98fbb975b0d7504127c79eead766c35d866cbdbf6f113f0b52189f7094c/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cd35298e3ad7ed753ec742f7e023025da2139a729f917a9a5ad5c48fad09e289/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/157c5e29dfd19dffbab1ccc58a8a71d8a9746153a85878d1a431f82abd5adfa8/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/583d6c65cd79218ee97d8af8abde56940e3f4e7532bfdbc257562dd97d44f1f0/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/cf1cc85bd01986fe32663c3b32ee2f5ff036b45479efa1928036dae1b7262326/merged 2024-08-20T01:05:58.935 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/6923df3f1873a6ff64588e8156ec5e2c99e6eb23e6f8fb72a33ea4acd81a3419/merged 2024-08-20T01:05:58.936 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/b4998db6ac61c4ccb2179407c008a9aac7e9a6db7b3f2bdd54f48897874aa2d5/merged 2024-08-20T01:05:58.936 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-08-20T01:05:58.936 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:58.936 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:05:58.936 INFO:teuthology.orchestra.run.smithi138.stdout:/home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:58.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:58.937 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-08-20T01:05:59.012 INFO:teuthology.orchestra.run.smithi138.stdout:120 2024-08-20T01:05:59.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [120] 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> 2024-08-20T01:05:59.013 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> 2024-08-20T01:05:59.014 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.admin")) 2024-08-20T01:05:59.015 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-08-20T01:05:59.177 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.admin.66673.asok 2024-08-20T01:05:59.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:59.192 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.66673.asok 2024-08-20T01:05:59.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:59.192 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.66673.asok status 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "ef4275410d8483698016fe484b2a3c664080b4c1", 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 19.0.0-5872-gef427541 (ef4275410d8483698016fe484b2a3c664080b4c1) squid (dev)", 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "admin", 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-08-20T01:05:59.376 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "66673", 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24838, 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 24838 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-08-20T01:05:59.377 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1324178474 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1324178474 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.24838 172.21.15.138:0/1324178474", 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "172.21.15.138:0/1324178474", 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 16, 2024-08-20T01:05:59.378 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 86, 2024-08-20T01:05:59.379 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-08-20T01:05:59.379 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-08-20T01:05:59.379 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-08-20T01:05:59.379 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:05:59.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:59.389 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-08-20T01:05:59.389 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:59.427 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-08-20T01:05:59.428 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:59.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:05:59.428 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:05:59.527 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-08-20T01:05:59.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:59 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T01:05:59.640 INFO:teuthology.orchestra.run.smithi138.stdout:# file: . 2024-08-20T01:05:59.640 INFO:teuthology.orchestra.run.smithi138.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-08-20T01:05:59.641 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:05:59.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:05:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:05:59.650 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:05:59.650 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T01:05:59.687 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T01:05:59.775 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:05:59 smithi138 ceph-mon[18076]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:05:59 smithi167 ceph-mon[21965]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:05:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:05:59 smithi142 ceph-mon[21799]: pgmap v296: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:06:00.098 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 -- 172.21.15.138:0/3454941400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4108e40 msgr2=0x7fbaa410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 --2- 172.21.15.138:0/3454941400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4108e40 0x7fbaa410f640 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ca0 tx=0x7fba9c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 -- 172.21.15.138:0/3454941400 shutdown_connections 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 --2- 172.21.15.138:0/3454941400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4108e40 0x7fbaa410f640 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 --2- 172.21.15.138:0/3454941400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa41084c0 0x7fbaa4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 --2- 172.21.15.138:0/3454941400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 -- 172.21.15.138:0/3454941400 >> 172.21.15.138:0/3454941400 conn(0x7fbaa40fe500 msgr2=0x7fbaa4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 -- 172.21.15.138:0/3454941400 shutdown_connections 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.726+0000 7fbaac400640 1 -- 172.21.15.138:0/3454941400 wait complete. 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 Processor -- start 2024-08-20T01:06:00.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 -- start start 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4108e40 0x7fbaa419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.727+0000 7fbaac400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4112b50 con 0x7fbaa41084c0 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaac400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41129d0 con 0x7fbaa4102ab0 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaac400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4112cd0 con 0x7fbaa4108e40 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42580/0 (socket says 172.21.15.138:42580) 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 -- 172.21.15.138:0/1432621986 learned_addr learned my addr 172.21.15.138:0/1432621986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaaa400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4108e40 0x7fbaa419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 -- 172.21.15.138:0/1432621986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4108e40 msgr2=0x7fbaa419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:00.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4108e40 0x7fbaa419e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 -- 172.21.15.138:0/1432621986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 msgr2=0x7fbaa4198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9000640 1 -- 172.21.15.138:0/1432621986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa419f010 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.728+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaa9000640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e9f0 tx=0x7fba9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9000cd80 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba900054d0 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa419f300 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90010690 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.729+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a59c0 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.730+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaa4102e90 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.732+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fba90004e90 con 0x7fbaa41084c0 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.734+0000 7fbaa2000640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 0x7fba8c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:00.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.734+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fba900e45b0 con 0x7fbaa41084c0 2024-08-20T01:06:00.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.734+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 0x7fba8c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:00.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.735+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 0x7fba8c0b60e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba980095f0 tx=0x7fba98005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:00.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.736+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fba900ab7a0 con 0x7fbaa41084c0 2024-08-20T01:06:00.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.932+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbaa41a5db0 con 0x7fbaa41084c0 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.932+0000 7fbaa2000640 1 -- 172.21.15.138:0/1432621986 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+897 (secure 0 0 0) 0x7fba900ab0a0 con 0x7fbaa41084c0 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2534070123 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3607751007 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:00.932 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3948920804 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/64005733 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/1354849701 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2736003984 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3346890941 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/4262320942 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3120862368 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:00.933 INFO:teuthology.orchestra.run.smithi138.stderr:listed 15 entries 2024-08-20T01:06:00.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 msgr2=0x7fba8c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:00.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 0x7fba8c0b60e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba980095f0 tx=0x7fba98005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:00.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 msgr2=0x7fbaa41994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:00.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fba9000e9f0 tx=0x7fba9000edf0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 shutdown_connections 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3c20 0x7fba8c0b60e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4108e40 0x7fbaa419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa41084c0 0x7fbaa41994a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 --2- 172.21.15.138:0/1432621986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa4102ab0 0x7fbaa4198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 >> 172.21.15.138:0/1432621986 conn(0x7fbaa40fe500 msgr2=0x7fbaa41008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 shutdown_connections 2024-08-20T01:06:00.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:00.936+0000 7fbaac400640 1 -- 172.21.15.138:0/1432621986 wait complete. 2024-08-20T01:06:01.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:01 smithi138 ceph-mon[18076]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:01.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:01 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1432621986' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:01.858 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:06:01.858 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T01:06:01.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T01:06:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:01 smithi167 ceph-mon[21965]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:01 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1432621986' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:01 smithi142 ceph-mon[21799]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:01 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1432621986' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:02.304 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:02.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 -- 172.21.15.138:0/1464826836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 msgr2=0x7fbcb8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:02.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1464826836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb8102e90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbca8009860 tx=0x7fbca802a340 comp rx=0 tx=0).stop 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 -- 172.21.15.138:0/1464826836 shutdown_connections 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1464826836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 0x7fbcb810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1464826836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1464826836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb8102e90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 -- 172.21.15.138:0/1464826836 >> 172.21.15.138:0/1464826836 conn(0x7fbcb80fe500 msgr2=0x7fbcb8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:02.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 -- 172.21.15.138:0/1464826836 shutdown_connections 2024-08-20T01:06:02.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.887+0000 7fbcbe600640 1 -- 172.21.15.138:0/1464826836 wait complete. 2024-08-20T01:06:02.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.888+0000 7fbcbe600640 1 Processor -- start 2024-08-20T01:06:02.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.888+0000 7fbcbe600640 1 -- start start 2024-08-20T01:06:02.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:02.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:02.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 0x7fbcb8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:02.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112b00 con 0x7fbcb81084c0 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112980 con 0x7fbcb8102ab0 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcbe600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbcb8112c80 con 0x7fbcb8108e40 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 0x7fbcb8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42710/0 (socket says 172.21.15.138:42710) 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 -- 172.21.15.138:0/1856500588 learned_addr learned my addr 172.21.15.138:0/1856500588 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 -- 172.21.15.138:0/1856500588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 msgr2=0x7fbcb8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 0x7fbcb8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 -- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 msgr2=0x7fbcb8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:02.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb7400640 1 -- 172.21.15.138:0/1856500588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcb81a73b0 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.889+0000 7fbcb6a00640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcb7400640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbca8035040 tx=0x7fbca8032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8037040 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbca8041070 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcb81a7640 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbca8003850 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.890+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcb81a7ac0 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.891+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcb8102e90 con 0x7fbcb8102ab0 2024-08-20T01:06:02.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.893+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fbca80039f0 con 0x7fbcb8102ab0 2024-08-20T01:06:02.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.895+0000 7fbcafe00640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 0x7fbc9c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:02.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.895+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fbca8104560 con 0x7fbcb8102ab0 2024-08-20T01:06:02.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.895+0000 7fbcb6a00640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 0x7fbc9c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:02.893 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.895+0000 7fbcb6a00640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 0x7fbc9c0b60e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbca40096c0 tx=0x7fbca4009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:02.894 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:02.896+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbca80cb750 con 0x7fbcb8102ab0 2024-08-20T01:06:03.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.093+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbcb802e340 con 0x7fbcb8102ab0 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.094+0000 7fbcafe00640 1 -- 172.21.15.138:0/1856500588 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v86) ==== 81+0+897 (secure 0 0 0) 0x7fbca80cb050 con 0x7fbcb8102ab0 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2534070123 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:03.092 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3607751007 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3948920804 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/64005733 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/1354849701 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2736003984 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3346890941 2024-08-21T00:58:34.377803+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/4262320942 2024-08-21T00:57:02.889250+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3120862368 2024-08-21T00:57:30.262737+0000 2024-08-20T01:06:03.093 INFO:teuthology.orchestra.run.smithi138.stderr:listed 15 entries 2024-08-20T01:06:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 msgr2=0x7fbc9c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 0x7fbc9c0b60e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbca40096c0 tx=0x7fbca4009260 comp rx=0 tx=0).stop 2024-08-20T01:06:03.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 msgr2=0x7fbcb80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbca8035040 tx=0x7fbca8032d50 comp rx=0 tx=0).stop 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 shutdown_connections 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc9c0b3c20 0x7fbc9c0b60e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcb8108e40 0x7fbcb8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcb81084c0 0x7fbcb8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 --2- 172.21.15.138:0/1856500588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbcb8102ab0 0x7fbcb80751d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 >> 172.21.15.138:0/1856500588 conn(0x7fbcb80fe500 msgr2=0x7fbcb81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 shutdown_connections 2024-08-20T01:06:03.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:03.096+0000 7fbcbe600640 1 -- 172.21.15.138:0/1856500588 wait complete. 2024-08-20T01:06:03.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:03 smithi138 ceph-mon[18076]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:03.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:03 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1856500588' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:03.903 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi138.front.sepia.ceph.com... 2024-08-20T01:06:03.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:06:03.903 DEBUG:teuthology.orchestra.run.smithi138:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-08-20T01:06:03.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi138.stderr:ceph-fuse[66673]: fuse finished with error 0 and tester_r 0 2024-08-20T01:06:03.962 INFO:teuthology.orchestra.run:waiting for 300 2024-08-20T01:06:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:03 smithi167 ceph-mon[21965]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:03.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:03 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1856500588' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:03 smithi142 ceph-mon[21799]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:03 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1856500588' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:06:05.679 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:05 smithi138 ceph-mon[18076]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:05 smithi167 ceph-mon[21965]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:05 smithi142 ceph-mon[21799]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-08-20T01:06:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:07 smithi167 ceph-mon[21965]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:07.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:07 smithi142 ceph-mon[21799]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:08.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:07 smithi138 ceph-mon[18076]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:09 smithi167 ceph-mon[21965]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:09 smithi142 ceph-mon[21799]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:10.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:09 smithi138 ceph-mon[18076]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:10.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:06:10.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:06:10.063 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-08-20T01:06:10.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:06:10.494 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 -- 172.21.15.138:0/2826587319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34113660 msgr2=0x7fef34115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 --2- 172.21.15.138:0/2826587319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34113660 0x7fef34115a50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fef2c009ca0 tx=0x7fef2c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 -- 172.21.15.138:0/2826587319 shutdown_connections 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 --2- 172.21.15.138:0/2826587319 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34113660 0x7fef34115a50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 --2- 172.21.15.138:0/2826587319 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075b90 0x7fef34113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 --2- 172.21.15.138:0/2826587319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34075270 0x7fef34075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 -- 172.21.15.138:0/2826587319 >> 172.21.15.138:0/2826587319 conn(0x7fef340fe390 msgr2=0x7fef341007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 -- 172.21.15.138:0/2826587319 shutdown_connections 2024-08-20T01:06:11.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.132+0000 7fef39400640 1 -- 172.21.15.138:0/2826587319 wait complete. 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.133+0000 7fef39400640 1 Processor -- start 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.133+0000 7fef39400640 1 -- start start 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 0x7fef3419f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34113660 0x7fef341a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fef34118f30 con 0x7fef34075b90 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fef34118db0 con 0x7fef34113660 2024-08-20T01:06:11.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef39400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fef341190b0 con 0x7fef34075270 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35394/0 (socket says 172.21.15.138:35394) 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef2b400640 1 -- 172.21.15.138:0/1280228638 learned_addr learned my addr 172.21.15.138:0/1280228638 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef33e00640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 0x7fef3419f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.134+0000 7fef38a00640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34113660 0x7fef341a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 -- 172.21.15.138:0/1280228638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 msgr2=0x7fef3419f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 0x7fef3419f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 -- 172.21.15.138:0/1280228638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34113660 msgr2=0x7fef341a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34113660 0x7fef341a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 -- 172.21.15.138:0/1280228638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef341a53b0 con 0x7fef34075b90 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef33e00640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 0x7fef3419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:11.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef2b400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00d950 tx=0x7fef1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c017070 con 0x7fef34075b90 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef1c009c30 con 0x7fef34075b90 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef341a56a0 con 0x7fef34075b90 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.135+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c00f620 con 0x7fef34075b90 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.136+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef341abf30 con 0x7fef34075b90 2024-08-20T01:06:11.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.136+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef34076780 con 0x7fef34075b90 2024-08-20T01:06:11.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.139+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fef1c010460 con 0x7fef34075b90 2024-08-20T01:06:11.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.140+0000 7fef31600640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 0x7fef180b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:11.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.140+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fef1c0e3bd0 con 0x7fef34075b90 2024-08-20T01:06:11.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.140+0000 7fef33e00640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 0x7fef180b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:11.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.141+0000 7fef33e00640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 0x7fef180b6200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef240095f0 tx=0x7fef24005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:11.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.144+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fef1c009de0 con 0x7fef34075b90 2024-08-20T01:06:11.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.328+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef341ac100 con 0x7fef34075b90 2024-08-20T01:06:11.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.329+0000 7fef31600640 1 -- 172.21.15.138:0/1280228638 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25045 (secure 0 0 0) 0x7fef1c0aa770 con 0x7fef34075b90 2024-08-20T01:06:11.328 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:11.329 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:05:39.935470+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:06:11.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 msgr2=0x7fef180b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:11.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 0x7fef180b6200 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef240095f0 tx=0x7fef24005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:11.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 msgr2=0x7fef3419f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fef1c00d950 tx=0x7fef1c00de10 comp rx=0 tx=0).stop 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 shutdown_connections 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fef180b3d40 0x7fef180b6200 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fef34113660 0x7fef341a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef34075b90 0x7fef3419f840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 --2- 172.21.15.138:0/1280228638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fef34075270 0x7fef3419f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 >> 172.21.15.138:0/1280228638 conn(0x7fef340fe390 msgr2=0x7fef340fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 shutdown_connections 2024-08-20T01:06:11.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:11.332+0000 7fef39400640 1 -- 172.21.15.138:0/1280228638 wait complete. 2024-08-20T01:06:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:11 smithi167 ceph-mon[21965]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:11 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1280228638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:11 smithi142 ceph-mon[21799]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:11 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1280228638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:12.001 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:11 smithi138 ceph-mon[18076]: pgmap v302: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-08-20T01:06:12.001 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:11 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1280228638' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:12.038 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:06:12.408 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:13.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 -- 172.21.15.138:0/3000590899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108310 msgr2=0x7fddec108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 --2- 172.21.15.138:0/3000590899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108310 0x7fddec108750 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009900 tx=0x7fddd402a400 comp rx=0 tx=0).stop 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 -- 172.21.15.138:0/3000590899 shutdown_connections 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 --2- 172.21.15.138:0/3000590899 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec108c90 0x7fddec10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 --2- 172.21.15.138:0/3000590899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108310 0x7fddec108750 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 --2- 172.21.15.138:0/3000590899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec102900 0x7fddec102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 -- 172.21.15.138:0/3000590899 >> 172.21.15.138:0/3000590899 conn(0x7fddec0fe390 msgr2=0x7fddec1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 -- 172.21.15.138:0/3000590899 shutdown_connections 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.991+0000 7fddebe00640 1 -- 172.21.15.138:0/3000590899 wait complete. 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 Processor -- start 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 -- start start 2024-08-20T01:06:13.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec102900 0x7fddec0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 0x7fddec075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fddec1129a0 con 0x7fddec108c90 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fddec112820 con 0x7fddec108310 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.992+0000 7fddebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fddec112b20 con 0x7fddec102900 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec102900 0x7fddec0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35412/0 (socket says 172.21.15.138:35412) 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 -- 172.21.15.138:0/8348696 learned_addr learned my addr 172.21.15.138:0/8348696 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fdde3e00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 0x7fddec075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 -- 172.21.15.138:0/8348696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec102900 msgr2=0x7fddec0751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec102900 0x7fddec0751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 -- 172.21.15.138:0/8348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 msgr2=0x7fddec075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 0x7fddec075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 -- 172.21.15.138:0/8348696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddec1a7230 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fdde3e00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 0x7fddec075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.993+0000 7fddeb400640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdde400eaa0 tx=0x7fdde400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.994+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde400cdb0 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.994+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdde4005020 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.994+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddec1a7520 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.994+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde40116d0 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.994+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddec1a7a30 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.995+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddec102ce0 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.997+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fdde40051d0 con 0x7fddec108c90 2024-08-20T01:06:13.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.998+0000 7fdde3400640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 0x7fddd00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:13.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.998+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fdde40e4160 con 0x7fddec108c90 2024-08-20T01:06:13.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.999+0000 7fddeaa00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 0x7fddd00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:13.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:12.999+0000 7fddeaa00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 0x7fddd00b6200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00bbe0 tx=0x7fdddc00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:13.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.002+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdde4016030 con 0x7fddec108c90 2024-08-20T01:06:13.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.186+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fddec0764d0 con 0x7fddec108c90 2024-08-20T01:06:13.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.189+0000 7fdde3400640 1 -- 172.21.15.138:0/8348696 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+25045 (secure 0 0 0) 0x7fdde40cd070 con 0x7fddec108c90 2024-08-20T01:06:13.187 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:13.187 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":86,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:05:39.935470+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 msgr2=0x7fddd00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 0x7fddd00b6200 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdddc00bbe0 tx=0x7fdddc00b390 comp rx=0 tx=0).stop 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 msgr2=0x7fddec075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdde400eaa0 tx=0x7fdde400ef60 comp rx=0 tx=0).stop 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 shutdown_connections 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fddd00b3d40 0x7fddd00b6200 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fddec108c90 0x7fddec075c50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fddec108310 0x7fddec075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 --2- 172.21.15.138:0/8348696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fddec102900 0x7fddec0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:13.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 >> 172.21.15.138:0/8348696 conn(0x7fddec0fe390 msgr2=0x7fddec1048c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:13.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 shutdown_connections 2024-08-20T01:06:13.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:13.192+0000 7fddebe00640 1 -- 172.21.15.138:0/8348696 wait complete. 2024-08-20T01:06:13.962 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:13 smithi138 ceph-mon[18076]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:13.962 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:13 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/8348696' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:13 smithi167 ceph-mon[21965]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:13 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/8348696' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:13 smithi142 ceph-mon[21799]: pgmap v303: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:13 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/8348696' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:13.999 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-08-20T01:06:14.368 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 -- 172.21.15.138:0/3556836742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10a6b0 msgr2=0x7f8b4c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:14.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 --2- 172.21.15.138:0/3556836742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c10aaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8b38009900 tx=0x7f8b3802a410 comp rx=0 tx=0).stop 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 -- 172.21.15.138:0/3556836742 shutdown_connections 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 --2- 172.21.15.138:0/3556836742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10b030 0x7f8b4c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 --2- 172.21.15.138:0/3556836742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c10aaf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 --2- 172.21.15.138:0/3556836742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 -- 172.21.15.138:0/3556836742 >> 172.21.15.138:0/3556836742 conn(0x7f8b4c1006f0 msgr2=0x7f8b4c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 -- 172.21.15.138:0/3556836742 shutdown_connections 2024-08-20T01:06:14.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.998+0000 7f8b52a00640 1 -- 172.21.15.138:0/3556836742 wait complete. 2024-08-20T01:06:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 Processor -- start 2024-08-20T01:06:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 -- start start 2024-08-20T01:06:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c19b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:14.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10b030 0x7f8b4c1a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114c80 con 0x7f8b4c10b030 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114b00 con 0x7f8b4c104ca0 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:14.999+0000 7f8b52a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8b4c114e00 con 0x7f8b4c10a6b0 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b50a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10b030 0x7f8b4c1a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:36516/0 (socket says 172.21.15.138:36516) 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 -- 172.21.15.138:0/2362151358 learned_addr learned my addr 172.21.15.138:0/2362151358 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4be00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c19b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 -- 172.21.15.138:0/2362151358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 msgr2=0x7f8b4c19b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c19b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 -- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10b030 msgr2=0x7f8b4c1a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10b030 0x7f8b4c1a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:14.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4b400640 1 -- 172.21.15.138:0/2362151358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b4c1a1200 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.000+0000 7f8b4be00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b4b400640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b38005940 tx=0x7f8b38004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3803f030 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b4c1a1490 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b4c1a7bb0 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b38003aa0 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.001+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b38049060 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.002+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b4c105080 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.005+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f8b38042030 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.006+0000 7f8b48c00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 0x7f8b300b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:15.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.006+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f8b38104590 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.006+0000 7f8b4be00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 0x7f8b300b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:15.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.007+0000 7f8b4be00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 0x7f8b300b60e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0095f0 tx=0x7f8b3c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:15.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.008+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8b380cb130 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.228+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f8b4c19c280 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.574+0000 7f8b48c00640 1 -- 172.21.15.138:0/2362151358 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f8b380cf680 con 0x7f8b4c10a6b0 2024-08-20T01:06:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 msgr2=0x7f8b300b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 0x7f8b300b60e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0095f0 tx=0x7f8b3c005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 msgr2=0x7f8b4c19b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8b38005940 tx=0x7f8b38004270 comp rx=0 tx=0).stop 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 shutdown_connections 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b300b3c20 0x7f8b300b60e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b4c10b030 0x7f8b4c1a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8b4c10a6b0 0x7f8b4c19b690 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 --2- 172.21.15.138:0/2362151358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b4c104ca0 0x7f8b4c19b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 >> 172.21.15.138:0/2362151358 conn(0x7f8b4c1006f0 msgr2=0x7f8b4c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 shutdown_connections 2024-08-20T01:06:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:15.577+0000 7f8b52a00640 1 -- 172.21.15.138:0/2362151358 wait complete. 2024-08-20T01:06:15.922 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:15 smithi138 ceph-mon[18076]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:15.923 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:15 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2362151358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:15.923 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:15 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:15 smithi167 ceph-mon[21965]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:15 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2362151358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:15 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:15 smithi142 ceph-mon[21799]: pgmap v304: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:15 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2362151358' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:15 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-08-20T01:06:16.352 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-08-20T01:06:16.352 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-08-20T01:06:16.733 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:16.871 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:16 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T01:06:16.871 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:16 smithi138 ceph-mon[18076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:16 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T01:06:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:16 smithi167 ceph-mon[21965]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:16.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:16 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-08-20T01:06:16.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:16 smithi142 ceph-mon[21799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 -- 172.21.15.138:0/3515038554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f434410ef30 msgr2=0x7f4344111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 --2- 172.21.15.138:0/3515038554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f434410ef30 0x7f4344111320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f433c009ca0 tx=0x7f433c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 -- 172.21.15.138:0/3515038554 shutdown_connections 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 --2- 172.21.15.138:0/3515038554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f434410ef30 0x7f4344111320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 --2- 172.21.15.138:0/3515038554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4344075b10 0x7f434410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 --2- 172.21.15.138:0/3515038554 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f43440751f0 0x7f43440755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 -- 172.21.15.138:0/3515038554 >> 172.21.15.138:0/3515038554 conn(0x7f43440fe350 msgr2=0x7f4344100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 -- 172.21.15.138:0/3515038554 shutdown_connections 2024-08-20T01:06:17.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.388+0000 7f4343200640 1 -- 172.21.15.138:0/3515038554 wait complete. 2024-08-20T01:06:17.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 Processor -- start 2024-08-20T01:06:17.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 -- start start 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 0x7f434419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4344075b10 0x7f434419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 0x7f43441a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4344114b10 con 0x7f4344075b10 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4344114990 con 0x7f43440751f0 2024-08-20T01:06:17.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.390+0000 7f4343200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4344114c90 con 0x7f434410ef30 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f433be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4344075b10 0x7f434419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4342800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 0x7f43441a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 0x7f434419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4342800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 0x7f43441a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:36538/0 (socket says 172.21.15.138:36538) 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4342800640 1 -- 172.21.15.138:0/216278260 learned_addr learned my addr 172.21.15.138:0/216278260 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 -- 172.21.15.138:0/216278260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 msgr2=0x7f43441a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 0x7f43441a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 -- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4344075b10 msgr2=0x7f434419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4344075b10 0x7f434419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:17.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.391+0000 7f4341e00640 1 -- 172.21.15.138:0/216278260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43441a0ff0 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f4341e00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 0x7f434419af40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f433400e9f0 tx=0x7f433400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433400ccc0 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43441a12e0 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4334005020 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4334004a00 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.392+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43441a79a0 con 0x7f43440751f0 2024-08-20T01:06:17.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.393+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43440755d0 con 0x7f43440751f0 2024-08-20T01:06:17.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.395+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7f4334004ba0 con 0x7f43440751f0 2024-08-20T01:06:17.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.397+0000 7f433aa00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 0x7f43280b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:17.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.397+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f43340e40d0 con 0x7f43440751f0 2024-08-20T01:06:17.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.397+0000 7f433be00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 0x7f43280b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:17.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.398+0000 7f433be00640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 0x7f43280b6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4344100b20 tx=0x7f432c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:17.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.398+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f43340ab350 con 0x7f43440751f0 2024-08-20T01:06:17.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:17.620+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f43441a7c90 con 0x7f43440751f0 2024-08-20T01:06:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:17 smithi167 ceph-mon[21965]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:17 smithi142 ceph-mon[21799]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:17 smithi138 ceph-mon[18076]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-08-20T01:06:18.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.588+0000 7f433aa00640 1 -- 172.21.15.138:0/216278260 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f43340aac50 con 0x7f43440751f0 2024-08-20T01:06:18.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.592+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 msgr2=0x7f43280b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.592+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 0x7f43280b6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4344100b20 tx=0x7f432c005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.592+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 msgr2=0x7f434419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.592+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 0x7f434419af40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f433400e9f0 tx=0x7f433400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 shutdown_connections 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f43280b3cc0 0x7f43280b6180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f434410ef30 0x7f43441a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:18.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4344075b10 0x7f434419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:18.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 --2- 172.21.15.138:0/216278260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f43440751f0 0x7f434419af40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:18.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 >> 172.21.15.138:0/216278260 conn(0x7f43440fe350 msgr2=0x7f43441110e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:18.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 shutdown_connections 2024-08-20T01:06:18.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:18.593+0000 7f4343200640 1 -- 172.21.15.138:0/216278260 wait complete. 2024-08-20T01:06:18.944 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:18 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/216278260' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:18.944 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:18 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:18.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:18 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/216278260' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:18.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:18 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:18.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:18 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/216278260' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:18.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:18 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} : dispatch 2024-08-20T01:06:19.379 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-08-20T01:06:19.758 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:19.898 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:06:19.898 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T01:06:19.898 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T01:06:19.899 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T01:06:19.899 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:19.899 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-20T01:06:19.899 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:19 smithi138 ceph-mon[18076]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-20T01:06:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:19 smithi167 ceph-mon[21965]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: 2024-08-20T01:06:19.796+0000 7f8921da7640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: daemon mds.g assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-08-20T01:06:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-08-20T01:06:19.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: fsmap cephfs:2 {0=b=up:active,1=g=up:creating} 10 up:standby 2024-08-20T01:06:19.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:19 smithi142 ceph-mon[21799]: daemon mds.g is now active in filesystem cephfs as rank 1 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 -- 172.21.15.138:0/1507445975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10aea0 msgr2=0x7ff7ec1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/1507445975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1116a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009ca0 tx=0x7ff7e402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 -- 172.21.15.138:0/1507445975 shutdown_connections 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/1507445975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1116a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/1507445975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec10a520 0x7ff7ec10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/1507445975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec104b10 0x7ff7ec104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 -- 172.21.15.138:0/1507445975 >> 172.21.15.138:0/1507445975 conn(0x7ff7ec100540 msgr2=0x7ff7ec102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 -- 172.21.15.138:0/1507445975 shutdown_connections 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.352+0000 7ff7ebe00640 1 -- 172.21.15.138:0/1507445975 wait complete. 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.353+0000 7ff7ebe00640 1 Processor -- start 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.353+0000 7ff7ebe00640 1 -- start start 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10a520 0x7ff7ec19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec114bb0 con 0x7ff7ec10a520 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec114a30 con 0x7ff7ec10aea0 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7ec114d30 con 0x7ff7ec104b10 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:20.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7ea000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10a520 0x7ff7ec19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:50232/0 (socket says 172.21.15.138:50232) 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 -- 172.21.15.138:0/686813504 learned_addr learned my addr 172.21.15.138:0/686813504 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eb400640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 -- 172.21.15.138:0/686813504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec10aea0 msgr2=0x7ff7ec1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.354+0000 7ff7eaa00640 1 -- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10a520 msgr2=0x7ff7ec19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7eaa00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10a520 0x7ff7ec19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7eaa00640 1 -- 172.21.15.138:0/686813504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7ec1a0fd0 con 0x7ff7ec104b10 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7eaa00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00cd80 con 0x7ff7ec104b10 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7ec1a12c0 con 0x7ff7ec104b10 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7dc0108e0 con 0x7ff7ec104b10 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc018980 con 0x7ff7ec104b10 2024-08-20T01:06:20.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.355+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7ec1a7b50 con 0x7ff7ec104b10 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.356+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7ec104ef0 con 0x7ff7ec104b10 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.359+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7ff7dc010430 con 0x7ff7ec104b10 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.360+0000 7ff7e3400640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 0x7ff7d00b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.360+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7ff7dc0e4890 con 0x7ff7ec104b10 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.360+0000 7ff7ea000640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 0x7ff7d00b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.361+0000 7ff7ea000640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 0x7ff7d00b6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec102de0 tx=0x7ff7d8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:20.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.362+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff7dc0ab430 con 0x7ff7ec104b10 2024-08-20T01:06:20.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.584+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7ff7ec1a7d20 con 0x7ff7ec104b10 2024-08-20T01:06:20.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.608+0000 7ff7e3400640 1 -- 172.21.15.138:0/686813504 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7ff7dc0af980 con 0x7ff7ec104b10 2024-08-20T01:06:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 msgr2=0x7ff7d00b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 0x7ff7d00b6180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec102de0 tx=0x7ff7d8005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:20.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 msgr2=0x7ff7ec19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 shutdown_connections 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7d00b3cc0 0x7ff7d00b6180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7ec10aea0 0x7ff7ec1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7ec10a520 0x7ff7ec19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 --2- 172.21.15.138:0/686813504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7ec104b10 0x7ff7ec19af20 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00e9f0 tx=0x7ff7dc00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 >> 172.21.15.138:0/686813504 conn(0x7ff7ec100540 msgr2=0x7ff7ec103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 shutdown_connections 2024-08-20T01:06:20.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:20.611+0000 7ff7ebe00640 1 -- 172.21.15.138:0/686813504 wait complete. 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:active 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: Cluster is now healthy 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/686813504' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:20 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:active 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: Cluster is now healthy 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/686813504' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:20 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] up:active 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: Cluster is now healthy 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: fsmap cephfs:2 {0=b=up:active,1=g=up:active} 10 up:standby 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:creating} 9 up:standby 2024-08-20T01:06:20.995 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-08-20T01:06:20.996 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/686813504' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:20.996 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:20 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-08-20T01:06:21.420 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-08-20T01:06:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: pgmap v307: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-08-20T01:06:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T01:06:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:active 2024-08-20T01:06:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-20T01:06:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:21.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:21 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: pgmap v307: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:active 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:21 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: pgmap v307: 129 pgs: 129 active+clean; 583 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 2 op/s 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] up:active 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 9 up:standby 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:21 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:06:22.455 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 -- 172.21.15.138:0/186060529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 msgr2=0x7fc5b4115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 --2- 172.21.15.138:0/186060529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b4115a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009ca0 tx=0x7fc5ac02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 -- 172.21.15.138:0/186060529 shutdown_connections 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 --2- 172.21.15.138:0/186060529 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b4115a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 --2- 172.21.15.138:0/186060529 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077d20 0x7fc5b4113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 --2- 172.21.15.138:0/186060529 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077400 0x7fc5b40777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 -- 172.21.15.138:0/186060529 >> 172.21.15.138:0/186060529 conn(0x7fc5b4100730 msgr2=0x7fc5b4102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 -- 172.21.15.138:0/186060529 shutdown_connections 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.071+0000 7fc5bae00640 1 -- 172.21.15.138:0/186060529 wait complete. 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 Processor -- start 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 -- start start 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077d20 0x7fc5b419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b41a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b41190b0 con 0x7fc5b4077400 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4118f30 con 0x7fc5b4113690 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5bae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b4119230 con 0x7fc5b4077d20 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41704/0 (socket says 172.21.15.138:41704) 2024-08-20T01:06:23.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5b3e00640 1 -- 172.21.15.138:0/348698581 learned_addr learned my addr 172.21.15.138:0/348698581 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.072+0000 7fc5b8e00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b41a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3400640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077d20 0x7fc5b419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 -- 172.21.15.138:0/348698581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077d20 msgr2=0x7fc5b419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077d20 0x7fc5b419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 -- 172.21.15.138:0/348698581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 msgr2=0x7fc5b41a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b41a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 -- 172.21.15.138:0/348698581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b41a56b0 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b3e00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400d950 tx=0x7fc5a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4017030 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a4004020 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40047a0 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.073+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b41a59a0 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.074+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b41ac060 con 0x7fc5b4077400 2024-08-20T01:06:23.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.075+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5b40777e0 con 0x7fc5b4077400 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.077+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fc5a4009a30 con 0x7fc5b4077400 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.078+0000 7fc5b0c00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 0x7fc5980b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.078+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fc5a40e40d0 con 0x7fc5b4077400 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.078+0000 7fc5b3400640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 0x7fc5980b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.079+0000 7fc5b3400640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 0x7fc5980b60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4115bd0 tx=0x7fc5a0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:23.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.080+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc5a40ab2c0 con 0x7fc5b4077400 2024-08-20T01:06:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:23 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:23 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:23 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:23 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:23 smithi138 ceph-mon[18076]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-08-20T01:06:23.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:23.366+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc5b41ac420 con 0x7fc5b4077400 2024-08-20T01:06:23.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:23 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:23 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:23 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:23 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:23 smithi167 ceph-mon[21965]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-08-20T01:06:23.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:23 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:23 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:23 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:23 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:23.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:23 smithi142 ceph-mon[21799]: pgmap v308: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s rd, 2.0 KiB/s wr, 5 op/s 2024-08-20T01:06:24.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.171+0000 7fc5b0c00640 1 -- 172.21.15.138:0/348698581 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fc5a40aabc0 con 0x7fc5b4077400 2024-08-20T01:06:24.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 msgr2=0x7fc5980b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:24.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 0x7fc5980b60e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4115bd0 tx=0x7fc5a0005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:24.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 msgr2=0x7fc5b419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400d950 tx=0x7fc5a400de10 comp rx=0 tx=0).stop 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 shutdown_connections 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc5980b3c20 0x7fc5980b60e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc5b4113690 0x7fc5b41a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc5b4077d20 0x7fc5b419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 --2- 172.21.15.138:0/348698581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5b4077400 0x7fc5b419f600 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.174+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 >> 172.21.15.138:0/348698581 conn(0x7fc5b4100730 msgr2=0x7fc5b4102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:24.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.175+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 shutdown_connections 2024-08-20T01:06:24.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:24.175+0000 7fc5bae00640 1 -- 172.21.15.138:0/348698581 wait complete. 2024-08-20T01:06:24.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:24 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T01:06:24.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:24 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:24 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T01:06:24.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:24 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:24 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-08-20T01:06:24.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:24 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:24.977 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-08-20T01:06:24.977 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-08-20T01:06:25.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T01:06:25.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:06:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:06:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:25.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:25 smithi138 ceph-mon[18076]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-08-20T01:06:25.347 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:25.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T01:06:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:06:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:06:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:25.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:25 smithi167 ceph-mon[21965]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/348698581' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:06:25.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:25 smithi142 ceph-mon[21799]: pgmap v309: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 2.0 KiB/s wr, 6 op/s 2024-08-20T01:06:26.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.023+0000 7fd092c00640 1 -- 172.21.15.138:0/122010665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c102ab0 msgr2=0x7fd08c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:26.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.023+0000 7fd092c00640 1 --2- 172.21.15.138:0/122010665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c102ab0 0x7fd08c102e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009860 tx=0x7fd07c02a340 comp rx=0 tx=0).stop 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 -- 172.21.15.138:0/122010665 shutdown_connections 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 --2- 172.21.15.138:0/122010665 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c108e40 0x7fd08c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 --2- 172.21.15.138:0/122010665 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 --2- 172.21.15.138:0/122010665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c102ab0 0x7fd08c102e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 -- 172.21.15.138:0/122010665 >> 172.21.15.138:0/122010665 conn(0x7fd08c0fe500 msgr2=0x7fd08c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 -- 172.21.15.138:0/122010665 shutdown_connections 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.024+0000 7fd092c00640 1 -- 172.21.15.138:0/122010665 wait complete. 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 Processor -- start 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 -- start start 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c108e40 0x7fd08c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:26.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112b00 con 0x7fd08c102ab0 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112980 con 0x7fd08c108e40 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.025+0000 7fd092c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112c80 con 0x7fd08c1084c0 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41720/0 (socket says 172.21.15.138:41720) 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 -- 172.21.15.138:0/1927504 learned_addr learned my addr 172.21.15.138:0/1927504 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd083e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd090c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c108e40 0x7fd08c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 -- 172.21.15.138:0/1927504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 msgr2=0x7fd08c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 -- 172.21.15.138:0/1927504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c108e40 msgr2=0x7fd08c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c108e40 0x7fd08c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 -- 172.21.15.138:0/1927504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c1a1330 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd083e00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.026+0000 7fd08be00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd07c035040 tx=0x7fd07c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.027+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c037040 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.027+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd07c041070 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.027+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003850 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.027+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c1a15c0 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.027+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1a7ce0 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.028+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c102e90 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.030+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) ==== 151077+0+0 (secure 0 0 0) 0x7fd07c0039f0 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.031+0000 7fd089600640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 0x7fd0700b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.032+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7fd07c103fd0 con 0x7fd08c102ab0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.032+0000 7fd083e00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 0x7fd0700b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:26.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.032+0000 7fd083e00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 0x7fd0700b60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd08c101ac0 tx=0x7fd074005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:26.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.034+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd07c074080 con 0x7fd08c102ab0 2024-08-20T01:06:26.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.246+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd08c100300 con 0x7fd0700b3c20 2024-08-20T01:06:26.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.273+0000 7fd089600640 1 -- 172.21.15.138:0/1927504 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd08c100300 con 0x7fd0700b3c20 2024-08-20T01:06:26.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.275+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 msgr2=0x7fd0700b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:26.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.275+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 0x7fd0700b60e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd08c101ac0 tx=0x7fd074005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:26.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.275+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 msgr2=0x7fd08c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:26.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.275+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd07c035040 tx=0x7fd07c032d50 comp rx=0 tx=0).stop 2024-08-20T01:06:26.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 shutdown_connections 2024-08-20T01:06:26.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd0700b3c20 0x7fd0700b60e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd08c108e40 0x7fd08c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd08c1084c0 0x7fd08c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 --2- 172.21.15.138:0/1927504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd08c102ab0 0x7fd08c19b280 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:26.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 >> 172.21.15.138:0/1927504 conn(0x7fd08c0fe500 msgr2=0x7fd08c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:26.275 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:06:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:06:26.253+0000 7f572ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:06:26.275 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:06:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:06:26.253+0000 7f572ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:06:26.275 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:06:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:06:26.253+0000 7f572ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:06:26.276 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:06:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:06:26.253+0000 7f572ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:06:26.276 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:06:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:06:26.253+0000 7f572ec00640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:06:26.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 shutdown_connections 2024-08-20T01:06:26.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:26.276+0000 7fd092c00640 1 -- 172.21.15.138:0/1927504 wait complete. 2024-08-20T01:06:27.064 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-08-20T01:06:27.431 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:27.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:27 smithi138 ceph-mon[18076]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:27 smithi138 ceph-mon[18076]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:27 smithi167 ceph-mon[21965]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:27 smithi167 ceph-mon[21965]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:27 smithi142 ceph-mon[21799]: from='client.14877 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:27 smithi142 ceph-mon[21799]: pgmap v310: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:28.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 -- 172.21.15.138:0/3044115296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 msgr2=0x7f491c105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 --2- 172.21.15.138:0/3044115296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c105080 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4910009900 tx=0x7f491002a410 comp rx=0 tx=0).stop 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 -- 172.21.15.138:0/3044115296 shutdown_connections 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 --2- 172.21.15.138:0/3044115296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 0x7f491c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 --2- 172.21.15.138:0/3044115296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 --2- 172.21.15.138:0/3044115296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c105080 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 -- 172.21.15.138:0/3044115296 >> 172.21.15.138:0/3044115296 conn(0x7f491c100730 msgr2=0x7f491c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 -- 172.21.15.138:0/3044115296 shutdown_connections 2024-08-20T01:06:28.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.088+0000 7f4924800640 1 -- 172.21.15.138:0/3044115296 wait complete. 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.089+0000 7f4924800640 1 Processor -- start 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.089+0000 7f4924800640 1 -- start start 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 0x7f491c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f491c114d40 con 0x7f491c104ca0 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f491c114bc0 con 0x7f491c10b030 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4924800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f491c114ec0 con 0x7f491c10a6b0 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:28.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40222/0 (socket says 172.21.15.138:40222) 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 -- 172.21.15.138:0/3325407005 learned_addr learned my addr 172.21.15.138:0/3325407005 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4922800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 0x7f491c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 -- 172.21.15.138:0/3325407005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 msgr2=0x7f491c19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 -- 172.21.15.138:0/3325407005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 msgr2=0x7f491c1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 0x7f491c1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921e00640 1 -- 172.21.15.138:0/3325407005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491c1a1260 con 0x7f491c104ca0 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.090+0000 7f4921400640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:28.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4921e00640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4910005ab0 tx=0x7f4910003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491003f070 con 0x7f491c104ca0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491c1a14f0 con 0x7f491c104ca0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4910031d60 con 0x7f491c104ca0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4910035030 con 0x7f491c104ca0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.091+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f491c1a7c10 con 0x7f491c104ca0 2024-08-20T01:06:28.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.093+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f491c105080 con 0x7f491c104ca0 2024-08-20T01:06:28.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.095+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f4910042070 con 0x7f491c104ca0 2024-08-20T01:06:28.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.096+0000 7f4916a00640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 0x7f49040b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:28.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.096+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f49101049d0 con 0x7f491c104ca0 2024-08-20T01:06:28.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.096+0000 7f4921400640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 0x7f49040b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:28.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.097+0000 7f4921400640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 0x7f49040b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f491c10a140 tx=0x7f490c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:28.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.100+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f49100cb410 con 0x7f491c104ca0 2024-08-20T01:06:28.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.309+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f491c10ce70 con 0x7f49040b3cb0 2024-08-20T01:06:28.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.336+0000 7f4916a00640 1 -- 172.21.15.138:0/3325407005 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f491c10ce70 con 0x7f49040b3cb0 2024-08-20T01:06:28.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.338+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 msgr2=0x7f49040b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:28.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.338+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 0x7f49040b6170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f491c10a140 tx=0x7f490c005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:28.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 msgr2=0x7f491c19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:28.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4910005ab0 tx=0x7f4910003c80 comp rx=0 tx=0).stop 2024-08-20T01:06:28.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 shutdown_connections 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49040b3cb0 0x7f49040b6170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f491c10b030 0x7f491c1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f491c10a6b0 0x7f491c19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 --2- 172.21.15.138:0/3325407005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f491c104ca0 0x7f491c19b1b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 >> 172.21.15.138:0/3325407005 conn(0x7f491c100730 msgr2=0x7f491c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 shutdown_connections 2024-08-20T01:06:28.339 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:28.339+0000 7f4924800640 1 -- 172.21.15.138:0/3325407005 wait complete. 2024-08-20T01:06:28.720 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:28 smithi138 ceph-mon[18076]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-20T01:06:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:28 smithi167 ceph-mon[21965]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-20T01:06:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:28 smithi142 ceph-mon[21799]: mgrmap e21: x(active, since 7m), standbys: z, y 2024-08-20T01:06:29.165 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-08-20T01:06:29.532 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:29.835 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:29 smithi138 ceph-mon[18076]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:29.835 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:29 smithi138 ceph-mon[18076]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:29 smithi167 ceph-mon[21965]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:29 smithi167 ceph-mon[21965]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:29 smithi142 ceph-mon[21799]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:29 smithi142 ceph-mon[21799]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-08-20T01:06:30.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 -- 172.21.15.138:0/550272221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 msgr2=0x7f8d10105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 --2- 172.21.15.138:0/550272221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d10105090 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8d00009ca0 tx=0x7f8d0002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 -- 172.21.15.138:0/550272221 shutdown_connections 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 --2- 172.21.15.138:0/550272221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010b040 0x7f8d10111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 --2- 172.21.15.138:0/550272221 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010a6c0 0x7f8d1010ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 --2- 172.21.15.138:0/550272221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d10105090 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 -- 172.21.15.138:0/550272221 >> 172.21.15.138:0/550272221 conn(0x7f8d10100720 msgr2=0x7f8d10102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 -- 172.21.15.138:0/550272221 shutdown_connections 2024-08-20T01:06:30.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.115+0000 7f8d16c00640 1 -- 172.21.15.138:0/550272221 wait complete. 2024-08-20T01:06:30.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.116+0000 7f8d16c00640 1 Processor -- start 2024-08-20T01:06:30.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.116+0000 7f8d16c00640 1 -- start start 2024-08-20T01:06:30.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d1019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010a6c0 0x7f8d1019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8d10114d50 con 0x7f8d10104cb0 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8d10114bd0 con 0x7f8d1010a6c0 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d16c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8d10114ed0 con 0x7f8d1010b040 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d0fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d1019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33554/0 (socket says 172.21.15.138:33554) 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 -- 172.21.15.138:0/713975801 learned_addr learned my addr 172.21.15.138:0/713975801 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d07400640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010a6c0 0x7f8d1019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 -- 172.21.15.138:0/713975801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010a6c0 msgr2=0x7f8d1019b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:30.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010a6c0 0x7f8d1019b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 -- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 msgr2=0x7f8d1019b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:30.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d1019b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.117+0000 7f8d14c00640 1 -- 172.21.15.138:0/713975801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d101a1270 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d14c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e9b0 tx=0x7f8d0800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0800cd80 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d101a1560 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d080035f0 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d08010690 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.118+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d101a7c20 con 0x7f8d1010b040 2024-08-20T01:06:30.119 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.119+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d10105090 con 0x7f8d1010b040 2024-08-20T01:06:30.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.122+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f8d08004000 con 0x7f8d1010b040 2024-08-20T01:06:30.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.123+0000 7f8d0d600640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 0x7f8cf40b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:30.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.123+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7f8d080e4630 con 0x7f8d1010b040 2024-08-20T01:06:30.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.123+0000 7f8d0fe00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 0x7f8cf40b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:30.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.124+0000 7f8d0fe00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 0x7f8cf40b6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d00009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:30.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.125+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8d080ab750 con 0x7f8d1010b040 2024-08-20T01:06:30.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.337+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f8d100632c0 con 0x7f8cf40b3cb0 2024-08-20T01:06:30.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.342+0000 7f8d0d600640 1 -- 172.21.15.138:0/713975801 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f8d100632c0 con 0x7f8cf40b3cb0 2024-08-20T01:06:30.341 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/qa/sv_0/4794f4a3-73b1-4071-b6db-d3166f941f96 2024-08-20T01:06:30.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 msgr2=0x7f8cf40b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:30.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 0x7f8cf40b6170 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8d00006010 tx=0x7f8d00009820 comp rx=0 tx=0).stop 2024-08-20T01:06:30.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 msgr2=0x7f8d101a0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:30.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800e9b0 tx=0x7f8d0800ee70 comp rx=0 tx=0).stop 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 shutdown_connections 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cf40b3cb0 0x7f8cf40b6170 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8d1010b040 0x7f8d101a0aa0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d1010a6c0 0x7f8d1019b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 --2- 172.21.15.138:0/713975801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8d10104cb0 0x7f8d1019b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 >> 172.21.15.138:0/713975801 conn(0x7f8d10100720 msgr2=0x7f8d10102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 shutdown_connections 2024-08-20T01:06:30.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:30.345+0000 7f8d16c00640 1 -- 172.21.15.138:0/713975801 wait complete. 2024-08-20T01:06:31.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-08-20T01:06:31.554 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:31.874 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:31 smithi138 ceph-mon[18076]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:31.874 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:31 smithi138 ceph-mon[18076]: pgmap v312: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-08-20T01:06:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:31 smithi167 ceph-mon[21965]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:31 smithi167 ceph-mon[21965]: pgmap v312: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-08-20T01:06:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:31 smithi142 ceph-mon[21799]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:31 smithi142 ceph-mon[21799]: pgmap v312: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 -- 172.21.15.138:0/2646122390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010a6a0 msgr2=0x7efe5010aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 --2- 172.21.15.138:0/2646122390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010a6a0 0x7efe5010aae0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efe38009900 tx=0x7efe3802a410 comp rx=0 tx=0).stop 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 -- 172.21.15.138:0/2646122390 shutdown_connections 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 --2- 172.21.15.138:0/2646122390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe5010b020 0x7efe50111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 --2- 172.21.15.138:0/2646122390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010a6a0 0x7efe5010aae0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 --2- 172.21.15.138:0/2646122390 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe50104c90 0x7efe50105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 -- 172.21.15.138:0/2646122390 >> 172.21.15.138:0/2646122390 conn(0x7efe501006c0 msgr2=0x7efe50102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 -- 172.21.15.138:0/2646122390 shutdown_connections 2024-08-20T01:06:32.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.193+0000 7efe55c00640 1 -- 172.21.15.138:0/2646122390 wait complete. 2024-08-20T01:06:32.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 Processor -- start 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 -- start start 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe50104c90 0x7efe5019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010b020 0x7efe501a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7efe50114cc0 con 0x7efe5010b020 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7efe50114b40 con 0x7efe50104c90 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.194+0000 7efe55c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7efe50114e40 con 0x7efe5010a6a0 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:32.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe4f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010b020 0x7efe501a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33578/0 (socket says 172.21.15.138:33578) 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 -- 172.21.15.138:0/3333390110 learned_addr learned my addr 172.21.15.138:0/3333390110 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe4ea00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe50104c90 0x7efe5019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 -- 172.21.15.138:0/3333390110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe50104c90 msgr2=0x7efe5019b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe50104c90 0x7efe5019b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 -- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010b020 msgr2=0x7efe501a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010b020 0x7efe501a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.195+0000 7efe46000640 1 -- 172.21.15.138:0/3333390110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe501a1220 con 0x7efe5010a6a0 2024-08-20T01:06:32.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe46000640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe38005ab0 tx=0x7efe38003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3803f070 con 0x7efe5010a6a0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe38031d60 con 0x7efe5010a6a0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe501a14b0 con 0x7efe5010a6a0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38035030 con 0x7efe5010a6a0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.196+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe501a7bd0 con 0x7efe5010a6a0 2024-08-20T01:06:32.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.197+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe50105070 con 0x7efe5010a6a0 2024-08-20T01:06:32.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.199+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7efe38042070 con 0x7efe5010a6a0 2024-08-20T01:06:32.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.201+0000 7efe47e00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 0x7efe340b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:32.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.201+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7efe38104e80 con 0x7efe5010a6a0 2024-08-20T01:06:32.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.201+0000 7efe4ea00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 0x7efe340b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:32.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.202+0000 7efe4ea00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 0x7efe340b6170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe40005fd0 tx=0x7efe40009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:32.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.203+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7efe380cb8c0 con 0x7efe5010a6a0 2024-08-20T01:06:32.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.410+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efe500632c0 con 0x7efe340b3cb0 2024-08-20T01:06:32.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.437+0000 7efe47e00640 1 -- 172.21.15.138:0/3333390110 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe500632c0 con 0x7efe340b3cb0 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.439+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 msgr2=0x7efe340b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.439+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 0x7efe340b6170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7efe40005fd0 tx=0x7efe40009760 comp rx=0 tx=0).stop 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.439+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 msgr2=0x7efe5019b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.439+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7efe38005ab0 tx=0x7efe38003c80 comp rx=0 tx=0).stop 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 shutdown_connections 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7efe340b3cb0 0x7efe340b6170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe5010b020 0x7efe501a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7efe5010a6a0 0x7efe5019b6b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 --2- 172.21.15.138:0/3333390110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe50104c90 0x7efe5019b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:32.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 >> 172.21.15.138:0/3333390110 conn(0x7efe501006c0 msgr2=0x7efe50102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:32.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 shutdown_connections 2024-08-20T01:06:32.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:32.440+0000 7efe55c00640 1 -- 172.21.15.138:0/3333390110 wait complete. 2024-08-20T01:06:33.363 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-08-20T01:06:33.731 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:33.869 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:33 smithi138 ceph-mon[18076]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:33.869 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:33 smithi138 ceph-mon[18076]: pgmap v313: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-20T01:06:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:33 smithi167 ceph-mon[21965]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:33 smithi167 ceph-mon[21965]: pgmap v313: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-20T01:06:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:33 smithi142 ceph-mon[21799]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:33 smithi142 ceph-mon[21799]: pgmap v313: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 4.2 KiB/s wr, 12 op/s 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 -- 172.21.15.138:0/1568159907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c40751f0 msgr2=0x7ff9c40755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 --2- 172.21.15.138:0/1568159907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c40751f0 0x7ff9c40755d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009900 tx=0x7ff9b402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 -- 172.21.15.138:0/1568159907 shutdown_connections 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 --2- 172.21.15.138:0/1568159907 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 0x7ff9c4111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 --2- 172.21.15.138:0/1568159907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c4075b10 0x7ff9c410e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 --2- 172.21.15.138:0/1568159907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c40751f0 0x7ff9c40755d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 -- 172.21.15.138:0/1568159907 >> 172.21.15.138:0/1568159907 conn(0x7ff9c40fe350 msgr2=0x7ff9c4100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 -- 172.21.15.138:0/1568159907 shutdown_connections 2024-08-20T01:06:34.367 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.366+0000 7ff9c9600640 1 -- 172.21.15.138:0/1568159907 wait complete. 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 Processor -- start 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 -- start start 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c40751f0 0x7ff9c419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 0x7ff9c41a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4114b60 con 0x7ff9c4075b10 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c41149e0 con 0x7ff9c40751f0 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.368+0000 7ff9c9600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4114ce0 con 0x7ff9c410ef30 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:34.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40284/0 (socket says 172.21.15.138:40284) 2024-08-20T01:06:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 -- 172.21.15.138:0/2177919685 learned_addr learned my addr 172.21.15.138:0/2177919685 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3e00640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c40751f0 0x7ff9c419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c8c00640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 0x7ff9c41a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:34.369 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 -- 172.21.15.138:0/2177919685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 msgr2=0x7ff9c41a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 0x7ff9c41a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 -- 172.21.15.138:0/2177919685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c40751f0 msgr2=0x7ff9c419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c40751f0 0x7ff9c419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.369+0000 7ff9c3400640 1 -- 172.21.15.138:0/2177919685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a1000 con 0x7ff9c4075b10 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c3400640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000d950 tx=0x7ff9b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0017030 con 0x7ff9c4075b10 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9b0004020 con 0x7ff9c4075b10 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a12f0 con 0x7ff9c4075b10 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00047a0 con 0x7ff9c4075b10 2024-08-20T01:06:34.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.370+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a79b0 con 0x7ff9c4075b10 2024-08-20T01:06:34.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.371+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c40755d0 con 0x7ff9c4075b10 2024-08-20T01:06:34.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.373+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff9b0009a30 con 0x7ff9c4075b10 2024-08-20T01:06:34.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.375+0000 7ff9c0c00640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 0x7ff9a80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:34.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.376+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 9249+0+0 (secure 0 0 0) 0x7ff9b00e3c80 con 0x7ff9c4075b10 2024-08-20T01:06:34.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.376+0000 7ff9c3e00640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 0x7ff9a80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:34.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.377+0000 7ff9c3e00640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 0x7ff9a80b6220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005ab0 tx=0x7ff9b4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:34.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.379+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff9b00aad10 con 0x7ff9c4075b10 2024-08-20T01:06:34.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.603+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff9c40632c0 con 0x7ff9a80b3d60 2024-08-20T01:06:34.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.610+0000 7ff9c0c00640 1 -- 172.21.15.138:0/2177919685 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff9c40632c0 con 0x7ff9a80b3d60 2024-08-20T01:06:34.609 INFO:teuthology.orchestra.run.smithi138.stdout:/volumes/qa/sv_1/61a922e4-5ca6-4b73-ad1d-5bdb03bd564e 2024-08-20T01:06:34.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.612+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 msgr2=0x7ff9a80b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:34.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.612+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 0x7ff9a80b6220 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4005ab0 tx=0x7ff9b4005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:34.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.612+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 msgr2=0x7ff9c419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:34.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.612+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000d950 tx=0x7ff9b000de10 comp rx=0 tx=0).stop 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 shutdown_connections 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff9a80b3d60 0x7ff9a80b6220 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff9c410ef30 0x7ff9c41a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9c4075b10 0x7ff9c419b490 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 --2- 172.21.15.138:0/2177919685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9c40751f0 0x7ff9c419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:34.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 >> 172.21.15.138:0/2177919685 conn(0x7ff9c40fe350 msgr2=0x7ff9c4106990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:34.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 shutdown_connections 2024-08-20T01:06:34.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:34.613+0000 7ff9c9600640 1 -- 172.21.15.138:0/2177919685 wait complete. 2024-08-20T01:06:35.436 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: pgmap v314: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:35 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:35.824 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: pgmap v314: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:35.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:35 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: pgmap v314: 129 pgs: 129 active+clean; 609 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 2.9 KiB/s wr, 9 op/s 2024-08-20T01:06:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='client.14907 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:06:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]} : dispatch 2024-08-20T01:06:35.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:35.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]} : dispatch 2024-08-20T01:06:35.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:35 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]} : dispatch 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.387+0000 7fa52a400640 1 -- 172.21.15.138:0/3099150055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 msgr2=0x7fa52410aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.387+0000 7fa52a400640 1 --2- 172.21.15.138:0/3099150055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52410aae0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fa50c009900 tx=0x7fa50c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 -- 172.21.15.138:0/3099150055 shutdown_connections 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 --2- 172.21.15.138:0/3099150055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa52410b020 0x7fa524111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 --2- 172.21.15.138:0/3099150055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52410aae0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 --2- 172.21.15.138:0/3099150055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa524104c90 0x7fa524105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 -- 172.21.15.138:0/3099150055 >> 172.21.15.138:0/3099150055 conn(0x7fa5241006c0 msgr2=0x7fa524102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 -- 172.21.15.138:0/3099150055 shutdown_connections 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 -- 172.21.15.138:0/3099150055 wait complete. 2024-08-20T01:06:36.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.388+0000 7fa52a400640 1 Processor -- start 2024-08-20T01:06:36.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 -- start start 2024-08-20T01:06:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa524104c90 0x7fa52419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:36.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 0x7fa5241a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa524114ca0 con 0x7fa52410a6a0 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa524114b20 con 0x7fa52410b020 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa52a400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa524114e20 con 0x7fa524104c90 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa522a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa522a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40308/0 (socket says 172.21.15.138:40308) 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa522a00640 1 -- 172.21.15.138:0/3231855765 learned_addr learned my addr 172.21.15.138:0/3231855765 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa523400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa524104c90 0x7fa52419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.389+0000 7fa523e00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 0x7fa5241a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 -- 172.21.15.138:0/3231855765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa524104c90 msgr2=0x7fa52419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa524104c90 0x7fa52419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 -- 172.21.15.138:0/3231855765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 msgr2=0x7fa5241a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 0x7fa5241a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 -- 172.21.15.138:0/3231855765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5241a1220 con 0x7fa52410a6a0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa523e00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 0x7fa5241a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa522a00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005940 tx=0x7fa50c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c03f030 con 0x7fa52410a6a0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5241a14b0 con 0x7fa52410a6a0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.390+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa50c003cb0 con 0x7fa52410a6a0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.391+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50c035070 con 0x7fa52410a6a0 2024-08-20T01:06:36.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.391+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5241a7bd0 con 0x7fa52410a6a0 2024-08-20T01:06:36.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.392+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa524105070 con 0x7fa52410a6a0 2024-08-20T01:06:36.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.394+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fa50c042030 con 0x7fa52410a6a0 2024-08-20T01:06:36.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.395+0000 7fa51be00640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 0x7fa5080b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:36.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.395+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 9365+0+0 (secure 0 0 0) 0x7fa50c104b80 con 0x7fa52410a6a0 2024-08-20T01:06:36.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.396+0000 7fa523400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 0x7fa5080b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:36.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.396+0000 7fa523400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 0x7fa5080b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa514005fd0 tx=0x7fa514009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:36.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.399+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa50c0cbba0 con 0x7fa52410a6a0 2024-08-20T01:06:36.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.626+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa52410c870 con 0x7fa52410a6a0 2024-08-20T01:06:36.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.627+0000 7fa51be00640 1 -- 172.21.15.138:0/3231855765 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fa50c0d2e20 con 0x7fa52410a6a0 2024-08-20T01:06:36.626 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:36.627 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:06:36.627 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.629+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 msgr2=0x7fa5080b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.629+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 0x7fa5080b6170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa514005fd0 tx=0x7fa514009760 comp rx=0 tx=0).stop 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.629+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 msgr2=0x7fa52419b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.629+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa50c005940 tx=0x7fa50c0042f0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 shutdown_connections 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa5080b3cb0 0x7fa5080b6170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa52410b020 0x7fa5241a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa52410a6a0 0x7fa52419b6b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 --2- 172.21.15.138:0/3231855765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa524104c90 0x7fa52419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:36.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 >> 172.21.15.138:0/3231855765 conn(0x7fa5241006c0 msgr2=0x7fa524102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:36.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 shutdown_connections 2024-08-20T01:06:36.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:36.630+0000 7fa52a400640 1 -- 172.21.15.138:0/3231855765 wait complete. 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T01:06:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:36 smithi167 ceph-mon[21965]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T01:06:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:36 smithi142 ceph-mon[21799]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.19", "id": [9, 8]}]': finished 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 10]}]': finished 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: osdmap e87: 12 total, 12 up, 12 in 2024-08-20T01:06:37.026 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:36 smithi138 ceph-mon[18076]: osdmap e88: 12 total, 12 up, 12 in 2024-08-20T01:06:37.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:06:37.782 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:37.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:37 smithi138 ceph-mon[18076]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-08-20T01:06:37.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:37 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3231855765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:06:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:37 smithi167 ceph-mon[21965]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-08-20T01:06:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:37 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3231855765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:06:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:37 smithi142 ceph-mon[21799]: pgmap v316: 129 pgs: 1 peering, 128 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.2 KiB/s wr, 9 op/s 2024-08-20T01:06:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:37 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3231855765' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:06:38.371 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 -- 172.21.15.138:0/2784061372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f810a520 msgr2=0x7ff7f810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 --2- 172.21.15.138:0/2784061372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f810a520 0x7ff7f810a960 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009900 tx=0x7ff7e402a410 comp rx=0 tx=0).stop 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 -- 172.21.15.138:0/2784061372 shutdown_connections 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 --2- 172.21.15.138:0/2784061372 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 --2- 172.21.15.138:0/2784061372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f810a520 0x7ff7f810a960 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 --2- 172.21.15.138:0/2784061372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f8104b10 0x7ff7f8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 -- 172.21.15.138:0/2784061372 >> 172.21.15.138:0/2784061372 conn(0x7ff7f8100540 msgr2=0x7ff7f8102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 -- 172.21.15.138:0/2784061372 shutdown_connections 2024-08-20T01:06:38.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.372+0000 7ff7fea00640 1 -- 172.21.15.138:0/2784061372 wait complete. 2024-08-20T01:06:38.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.373+0000 7ff7fea00640 1 Processor -- start 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 -- start start 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f810a520 0x7ff7f819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8114b70 con 0x7ff7f8104b10 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f81149f0 con 0x7ff7f810a520 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f8114cf0 con 0x7ff7f810aea0 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fd600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36702/0 (socket says 172.21.15.138:36702) 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fd600640 1 -- 172.21.15.138:0/3579267757 learned_addr learned my addr 172.21.15.138:0/3579267757 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.374+0000 7ff7fcc00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f810a520 0x7ff7f819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:38.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fe000640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 -- 172.21.15.138:0/3579267757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 msgr2=0x7ff7f81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 -- 172.21.15.138:0/3579267757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f810a520 msgr2=0x7ff7f819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:38.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f810a520 0x7ff7f819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 -- 172.21.15.138:0/3579267757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f81a0fd0 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fe000640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fd600640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00cd80 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec0108e0 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec018980 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.375+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f81a12c0 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.376+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f81a7980 con 0x7ff7f8104b10 2024-08-20T01:06:38.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.377+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f8104ef0 con 0x7ff7f8104b10 2024-08-20T01:06:38.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.379+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff7ec010430 con 0x7ff7f8104b10 2024-08-20T01:06:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.380+0000 7ff7f6000640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 0x7ff7e00b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.380+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff7ec0e4670 con 0x7ff7f8104b10 2024-08-20T01:06:38.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.380+0000 7ff7fcc00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 0x7ff7e00b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:38.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.381+0000 7ff7fcc00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 0x7ff7e00b6220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005ab0 tx=0x7ff7e4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:38.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.384+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff7ec0ab690 con 0x7ff7f8104b10 2024-08-20T01:06:38.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.570+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff7f81a7cb0 con 0x7ff7f8104b10 2024-08-20T01:06:38.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.572+0000 7ff7f6000640 1 -- 172.21.15.138:0/3579267757 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7ff7ec0aaf90 con 0x7ff7f8104b10 2024-08-20T01:06:38.571 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:38.572 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 msgr2=0x7ff7e00b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 0x7ff7e00b6220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4005ab0 tx=0x7ff7e4005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 msgr2=0x7ff7f819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00e9f0 tx=0x7ff7ec00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 shutdown_connections 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7e00b3d60 0x7ff7e00b6220 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7f810aea0 0x7ff7f81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7f810a520 0x7ff7f819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 --2- 172.21.15.138:0/3579267757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7f8104b10 0x7ff7f819af20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:38.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.574+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 >> 172.21.15.138:0/3579267757 conn(0x7ff7f8100540 msgr2=0x7ff7f8102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.575+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 shutdown_connections 2024-08-20T01:06:38.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:38.575+0000 7ff7fea00640 1 -- 172.21.15.138:0/3579267757 wait complete. 2024-08-20T01:06:38.939 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:38 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3579267757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:38.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:38 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3579267757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:38.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:38 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3579267757' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:06:39.332 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T01:06:39.332 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:06:39.700 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:39.839 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:39 smithi138 ceph-mon[18076]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-08-20T01:06:39.839 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:39 smithi138 ceph-mon[18076]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-20T01:06:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:39 smithi167 ceph-mon[21965]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-08-20T01:06:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:39 smithi167 ceph-mon[21965]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-20T01:06:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:39 smithi142 ceph-mon[21799]: pgmap v318: 129 pgs: 1 activating, 1 peering, 127 active+clean; 631 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.6 KiB/s wr, 6 op/s 2024-08-20T01:06:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:39 smithi142 ceph-mon[21799]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 -- 172.21.15.138:0/2014756634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 msgr2=0x7f47d410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 --2- 172.21.15.138:0/2014756634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d410f640 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009ca0 tx=0x7f47cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 -- 172.21.15.138:0/2014756634 shutdown_connections 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 --2- 172.21.15.138:0/2014756634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d410f640 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 --2- 172.21.15.138:0/2014756634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d41084c0 0x7f47d4108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 --2- 172.21.15.138:0/2014756634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d4102ab0 0x7f47d4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.315+0000 7f47d9200640 1 -- 172.21.15.138:0/2014756634 >> 172.21.15.138:0/2014756634 conn(0x7f47d40fe500 msgr2=0x7f47d4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:40.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.316+0000 7f47d9200640 1 -- 172.21.15.138:0/2014756634 shutdown_connections 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.316+0000 7f47d9200640 1 -- 172.21.15.138:0/2014756634 wait complete. 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 Processor -- start 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 -- start start 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 0x7f47d419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d41084c0 0x7f47d419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4112b50 con 0x7f47d4108e40 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47d41129d0 con 0x7f47d4102ab0 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d9200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47d4112cd0 con 0x7f47d41084c0 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36722/0 (socket says 172.21.15.138:36722) 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d2800640 1 -- 172.21.15.138:0/1345073287 learned_addr learned my addr 172.21.15.138:0/1345073287 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:40.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.317+0000 7f47d1e00640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 0x7f47d419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47c9400640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d41084c0 0x7f47d419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 -- 172.21.15.138:0/1345073287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d41084c0 msgr2=0x7f47d419b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d41084c0 0x7f47d419b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 -- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 msgr2=0x7f47d41a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 -- 172.21.15.138:0/1345073287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d41a11d0 con 0x7f47d4102ab0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d2800640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47d1e00640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 0x7f47d419b120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9f0 tx=0x7f47c400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.318+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400ccc0 con 0x7f47d4102ab0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.319+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d41a14c0 con 0x7f47d4102ab0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.319+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c4005020 con 0x7f47d4102ab0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.319+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4004a00 con 0x7f47d4102ab0 2024-08-20T01:06:40.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.319+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d41a7b80 con 0x7f47d4102ab0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.320+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d4102e90 con 0x7f47d4102ab0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.322+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f47c4004ba0 con 0x7f47d4102ab0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.323+0000 7f47cb400640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 0x7f47b80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.324+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f47c40e42d0 con 0x7f47d4102ab0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.324+0000 7f47c9400640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 0x7f47b80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.324+0000 7f47c9400640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 0x7f47b80b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47d4107d40 tx=0x7f47bc005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:40.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.325+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f47c40ab380 con 0x7f47d4102ab0 2024-08-20T01:06:40.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.506+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f47d410f090 con 0x7f47b80b3cb0 2024-08-20T01:06:40.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.513+0000 7f47cb400640 1 -- 172.21.15.138:0/1345073287 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f47d410f090 con 0x7f47b80b3cb0 2024-08-20T01:06:40.517 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:40.518 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 msgr2=0x7f47b80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 0x7f47b80b6170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f47d4107d40 tx=0x7f47bc005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 msgr2=0x7f47d419b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 0x7f47d419b120 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47c400e9f0 tx=0x7f47c400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 shutdown_connections 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47b80b3cb0 0x7f47b80b6170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47d4108e40 0x7f47d41a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47d41084c0 0x7f47d419b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 --2- 172.21.15.138:0/1345073287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4102ab0 0x7f47d419b120 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:40.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 >> 172.21.15.138:0/1345073287 conn(0x7f47d40fe500 msgr2=0x7f47d40758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:40.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 shutdown_connections 2024-08-20T01:06:40.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:40.518+0000 7f47d9200640 1 -- 172.21.15.138:0/1345073287 wait complete. 2024-08-20T01:06:41.320 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-08-20T01:06:40.485184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650763,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":218,"num_read_kb":200,"num_write":288,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352716,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589108,"statfs":{"total":1151940427776,"available":1151579246592,"internally_reserved":0,"allocated":12025856,"data_stored":8044556,"data_compressed":403487,"data_compressed_allocated":544768,"data_compressed_original":2588672,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27070,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":21,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621083+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621083+0000","last_peered":"2024-08-20T01:06:36.621083+0000","last_clean":"2024-08-20T01:06:36.621083+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621083+0000","last_undegraded":"2024-08-20T01:06:36.621083+0000","last_fullsized":"2024-08-20T01:06:36.621083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564621+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:06:36.564621+0000","last_peered":"2024-08-20T01:06:36.564621+0000","last_clean":"2024-08-20T01:06:36.564621+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:06:36.564621+0000","last_undegraded":"2024-08-20T01:06:36.564621+0000","last_fullsized":"2024-08-20T01:06:36.564621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"peering","last_fresh":"2024-08-20T01:05:40.063072+0000","last_change":"2024-08-20T01:06:35.563750+0000","last_active":"2024-08-20T01:06:35.563750+0000","last_peered":"2024-08-20T01:05:40.063072+0000","last_clean":"2024-08-20T01:05:40.063072+0000","last_became_active":"2024-08-20T01:05:35.968597+0000","last_became_peered":"2024-08-20T01:05:35.968597+0000","last_unstale":"2024-08-20T01:05:40.063072+0000","last_undegraded":"2024-08-20T01:05:40.063072+0000","last_fullsized":"2024-08-20T01:05:40.063072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564871+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564871+0000","last_peered":"2024-08-20T01:06:36.564871+0000","last_clean":"2024-08-20T01:06:36.564871+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564871+0000","last_undegraded":"2024-08-20T01:06:36.564871+0000","last_fullsized":"2024-08-20T01:06:36.564871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425408+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:06:37.425408+0000","last_peered":"2024-08-20T01:06:37.425408+0000","last_clean":"2024-08-20T01:06:37.425408+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:06:37.425408+0000","last_undegraded":"2024-08-20T01:06:37.425408+0000","last_fullsized":"2024-08-20T01:06:37.425408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063059+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:05:40.063059+0000","last_peered":"2024-08-20T01:05:40.063059+0000","last_clean":"2024-08-20T01:05:40.063059+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:05:40.063059+0000","last_undegraded":"2024-08-20T01:05:40.063059+0000","last_fullsized":"2024-08-20T01:05:40.063059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642316+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:06:36.642316+0000","last_peered":"2024-08-20T01:06:36.642316+0000","last_clean":"2024-08-20T01:06:36.642316+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:06:36.642316+0000","last_undegraded":"2024-08-20T01:06:36.642316+0000","last_fullsized":"2024-08-20T01:06:36.642316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064464+0000","last_change":"2024-08-20T01:05:35.969178+0000","last_active":"2024-08-20T01:05:40.064464+0000","last_peered":"2024-08-20T01:05:40.064464+0000","last_clean":"2024-08-20T01:05:40.064464+0000","last_became_active":"2024-08-20T01:05:35.968411+0000","last_became_peered":"2024-08-20T01:05:35.968411+0000","last_unstale":"2024-08-20T01:05:40.064464+0000","last_undegraded":"2024-08-20T01:05:40.064464+0000","last_fullsized":"2024-08-20T01:05:40.064464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:42:43.856816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647417+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:06:36.647417+0000","last_peered":"2024-08-20T01:06:36.647417+0000","last_clean":"2024-08-20T01:06:36.647417+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:06:36.647417+0000","last_undegraded":"2024-08-20T01:06:36.647417+0000","last_fullsized":"2024-08-20T01:06:36.647417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064093+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:05:40.064093+0000","last_peered":"2024-08-20T01:05:40.064093+0000","last_clean":"2024-08-20T01:05:40.064093+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:05:40.064093+0000","last_undegraded":"2024-08-20T01:05:40.064093+0000","last_fullsized":"2024-08-20T01:05:40.064093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900542+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:06:36.900542+0000","last_peered":"2024-08-20T01:06:36.900542+0000","last_clean":"2024-08-20T01:06:36.900542+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:06:36.900542+0000","last_undegraded":"2024-08-20T01:06:36.900542+0000","last_fullsized":"2024-08-20T01:06:36.900542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063325+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:05:40.063325+0000","last_peered":"2024-08-20T01:05:40.063325+0000","last_clean":"2024-08-20T01:05:40.063325+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:05:40.063325+0000","last_undegraded":"2024-08-20T01:05:40.063325+0000","last_fullsized":"2024-08-20T01:05:40.063325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564754+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:06:36.564754+0000","last_peered":"2024-08-20T01:06:36.564754+0000","last_clean":"2024-08-20T01:06:36.564754+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:06:36.564754+0000","last_undegraded":"2024-08-20T01:06:36.564754+0000","last_fullsized":"2024-08-20T01:06:36.564754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563108+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:35.563108+0000","last_peered":"2024-08-20T01:06:35.563108+0000","last_clean":"2024-08-20T01:06:35.563108+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:35.563108+0000","last_undegraded":"2024-08-20T01:06:35.563108+0000","last_fullsized":"2024-08-20T01:06:35.563108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064434+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:05:40.064434+0000","last_peered":"2024-08-20T01:05:40.064434+0000","last_clean":"2024-08-20T01:05:40.064434+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:05:40.064434+0000","last_undegraded":"2024-08-20T01:05:40.064434+0000","last_fullsized":"2024-08-20T01:05:40.064434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.062918+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:05:40.062918+0000","last_peered":"2024-08-20T01:05:40.062918+0000","last_clean":"2024-08-20T01:05:40.062918+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:05:40.062918+0000","last_undegraded":"2024-08-20T01:05:40.062918+0000","last_fullsized":"2024-08-20T01:05:40.062918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564536+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564536+0000","last_peered":"2024-08-20T01:06:36.564536+0000","last_clean":"2024-08-20T01:06:36.564536+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564536+0000","last_undegraded":"2024-08-20T01:06:36.564536+0000","last_fullsized":"2024-08-20T01:06:36.564536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567485+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:06:36.567485+0000","last_peered":"2024-08-20T01:06:36.567485+0000","last_clean":"2024-08-20T01:06:36.567485+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:06:36.567485+0000","last_undegraded":"2024-08-20T01:06:36.567485+0000","last_fullsized":"2024-08-20T01:06:36.567485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642212+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642212+0000","last_peered":"2024-08-20T01:06:36.642212+0000","last_clean":"2024-08-20T01:06:36.642212+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642212+0000","last_undegraded":"2024-08-20T01:06:36.642212+0000","last_fullsized":"2024-08-20T01:06:36.642212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063575+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:05:40.063575+0000","last_peered":"2024-08-20T01:05:40.063575+0000","last_clean":"2024-08-20T01:05:40.063575+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:05:40.063575+0000","last_undegraded":"2024-08-20T01:05:40.063575+0000","last_fullsized":"2024-08-20T01:05:40.063575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425236+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:06:37.425236+0000","last_peered":"2024-08-20T01:06:37.425236+0000","last_clean":"2024-08-20T01:06:37.425236+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:06:37.425236+0000","last_undegraded":"2024-08-20T01:06:37.425236+0000","last_fullsized":"2024-08-20T01:06:37.425236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:18.605774+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:18.605774+0000","last_peered":"2024-08-20T01:06:18.605774+0000","last_clean":"2024-08-20T01:06:18.605774+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:18.605774+0000","last_undegraded":"2024-08-20T01:06:18.605774+0000","last_fullsized":"2024-08-20T01:06:18.605774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563129+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:35.563129+0000","last_peered":"2024-08-20T01:06:35.563129+0000","last_clean":"2024-08-20T01:06:35.563129+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:35.563129+0000","last_undegraded":"2024-08-20T01:06:35.563129+0000","last_fullsized":"2024-08-20T01:06:35.563129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564938+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:06:36.564938+0000","last_peered":"2024-08-20T01:06:36.564938+0000","last_clean":"2024-08-20T01:06:36.564938+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:06:36.564938+0000","last_undegraded":"2024-08-20T01:06:36.564938+0000","last_fullsized":"2024-08-20T01:06:36.564938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.622441+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:06:36.622441+0000","last_peered":"2024-08-20T01:06:36.622441+0000","last_clean":"2024-08-20T01:06:36.622441+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:06:36.622441+0000","last_undegraded":"2024-08-20T01:06:36.622441+0000","last_fullsized":"2024-08-20T01:06:36.622441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564087+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564087+0000","last_peered":"2024-08-20T01:06:36.564087+0000","last_clean":"2024-08-20T01:06:36.564087+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564087+0000","last_undegraded":"2024-08-20T01:06:36.564087+0000","last_fullsized":"2024-08-20T01:06:36.564087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567052+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:06:36.567052+0000","last_peered":"2024-08-20T01:06:36.567052+0000","last_clean":"2024-08-20T01:06:36.567052+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:06:36.567052+0000","last_undegraded":"2024-08-20T01:06:36.567052+0000","last_fullsized":"2024-08-20T01:06:36.567052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566662+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566662+0000","last_peered":"2024-08-20T01:06:36.566662+0000","last_clean":"2024-08-20T01:06:36.566662+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566662+0000","last_undegraded":"2024-08-20T01:06:36.566662+0000","last_fullsized":"2024-08-20T01:06:36.566662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064483+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:05:40.064483+0000","last_peered":"2024-08-20T01:05:40.064483+0000","last_clean":"2024-08-20T01:05:40.064483+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:05:40.064483+0000","last_undegraded":"2024-08-20T01:05:40.064483+0000","last_fullsized":"2024-08-20T01:05:40.064483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644635+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:06:36.644635+0000","last_peered":"2024-08-20T01:06:36.644635+0000","last_clean":"2024-08-20T01:06:36.644635+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:06:36.644635+0000","last_undegraded":"2024-08-20T01:06:36.644635+0000","last_fullsized":"2024-08-20T01:06:36.644635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620802+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620802+0000","last_peered":"2024-08-20T01:06:36.620802+0000","last_clean":"2024-08-20T01:06:36.620802+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620802+0000","last_undegraded":"2024-08-20T01:06:36.620802+0000","last_fullsized":"2024-08-20T01:06:36.620802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644713+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:06:36.644713+0000","last_peered":"2024-08-20T01:06:36.644713+0000","last_clean":"2024-08-20T01:06:36.644713+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:06:36.644713+0000","last_undegraded":"2024-08-20T01:06:36.644713+0000","last_fullsized":"2024-08-20T01:06:36.644713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900709+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900709+0000","last_peered":"2024-08-20T01:06:36.900709+0000","last_clean":"2024-08-20T01:06:36.900709+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900709+0000","last_undegraded":"2024-08-20T01:06:36.900709+0000","last_fullsized":"2024-08-20T01:06:36.900709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566848+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566848+0000","last_peered":"2024-08-20T01:06:36.566848+0000","last_clean":"2024-08-20T01:06:36.566848+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566848+0000","last_undegraded":"2024-08-20T01:06:36.566848+0000","last_fullsized":"2024-08-20T01:06:36.566848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647086+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:06:36.647086+0000","last_peered":"2024-08-20T01:06:36.647086+0000","last_clean":"2024-08-20T01:06:36.647086+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:06:36.647086+0000","last_undegraded":"2024-08-20T01:06:36.647086+0000","last_fullsized":"2024-08-20T01:06:36.647086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.645366+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:06:38.645366+0000","last_peered":"2024-08-20T01:06:38.645366+0000","last_clean":"2024-08-20T01:06:38.645366+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:06:38.645366+0000","last_undegraded":"2024-08-20T01:06:38.645366+0000","last_fullsized":"2024-08-20T01:06:38.645366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566529+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:06:36.566529+0000","last_peered":"2024-08-20T01:06:36.566529+0000","last_clean":"2024-08-20T01:06:36.566529+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:06:36.566529+0000","last_undegraded":"2024-08-20T01:06:36.566529+0000","last_fullsized":"2024-08-20T01:06:36.566529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900529+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900529+0000","last_peered":"2024-08-20T01:06:36.900529+0000","last_clean":"2024-08-20T01:06:36.900529+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900529+0000","last_undegraded":"2024-08-20T01:06:36.900529+0000","last_fullsized":"2024-08-20T01:06:36.900529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564431+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:06:36.564431+0000","last_peered":"2024-08-20T01:06:36.564431+0000","last_clean":"2024-08-20T01:06:36.564431+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:06:36.564431+0000","last_undegraded":"2024-08-20T01:06:36.564431+0000","last_fullsized":"2024-08-20T01:06:36.564431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:20.615714+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:20.615714+0000","last_peered":"2024-08-20T01:06:20.615714+0000","last_clean":"2024-08-20T01:06:20.615714+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:20.615714+0000","last_undegraded":"2024-08-20T01:06:20.615714+0000","last_fullsized":"2024-08-20T01:06:20.615714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900668+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900668+0000","last_peered":"2024-08-20T01:06:36.900668+0000","last_clean":"2024-08-20T01:06:36.900668+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900668+0000","last_undegraded":"2024-08-20T01:06:36.900668+0000","last_fullsized":"2024-08-20T01:06:36.900668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901014+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:06:36.901014+0000","last_peered":"2024-08-20T01:06:36.901014+0000","last_clean":"2024-08-20T01:06:36.901014+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:06:36.901014+0000","last_undegraded":"2024-08-20T01:06:36.901014+0000","last_fullsized":"2024-08-20T01:06:36.901014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563387+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:35.563387+0000","last_peered":"2024-08-20T01:06:35.563387+0000","last_clean":"2024-08-20T01:06:35.563387+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:35.563387+0000","last_undegraded":"2024-08-20T01:06:35.563387+0000","last_fullsized":"2024-08-20T01:06:35.563387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900715+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:06:36.900715+0000","last_peered":"2024-08-20T01:06:36.900715+0000","last_clean":"2024-08-20T01:06:36.900715+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:06:36.900715+0000","last_undegraded":"2024-08-20T01:06:36.900715+0000","last_fullsized":"2024-08-20T01:06:36.900715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647579+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647579+0000","last_peered":"2024-08-20T01:06:36.647579+0000","last_clean":"2024-08-20T01:06:36.647579+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647579+0000","last_undegraded":"2024-08-20T01:06:36.647579+0000","last_fullsized":"2024-08-20T01:06:36.647579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900966+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:06:36.900966+0000","last_peered":"2024-08-20T01:06:36.900966+0000","last_clean":"2024-08-20T01:06:36.900966+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:06:36.900966+0000","last_undegraded":"2024-08-20T01:06:36.900966+0000","last_fullsized":"2024-08-20T01:06:36.900966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567528+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567528+0000","last_peered":"2024-08-20T01:06:36.567528+0000","last_clean":"2024-08-20T01:06:36.567528+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567528+0000","last_undegraded":"2024-08-20T01:06:36.567528+0000","last_fullsized":"2024-08-20T01:06:36.567528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064549+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:05:40.064549+0000","last_peered":"2024-08-20T01:05:40.064549+0000","last_clean":"2024-08-20T01:05:40.064549+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:05:40.064549+0000","last_undegraded":"2024-08-20T01:05:40.064549+0000","last_fullsized":"2024-08-20T01:05:40.064549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":5,"reported_epoch":88,"state":"activating","last_fresh":"2024-08-20T01:06:36.568460+0000","last_change":"2024-08-20T01:06:36.567147+0000","last_active":"2024-08-20T01:05:34.937557+0000","last_peered":"2024-08-20T01:05:34.937557+0000","last_clean":"2024-08-20T01:05:34.937557+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-20T01:06:36.568460+0000","last_undegraded":"2024-08-20T01:06:36.568460+0000","last_fullsized":"2024-08-20T01:06:36.568460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":["8(0)","0(1)","1(3)","3(2)","5(0)"],"object_location_counts":[],"blocked_by":[1,3],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647257+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:06:36.647257+0000","last_peered":"2024-08-20T01:06:36.647257+0000","last_clean":"2024-08-20T01:06:36.647257+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:06:36.647257+0000","last_undegraded":"2024-08-20T01:06:36.647257+0000","last_fullsized":"2024-08-20T01:06:36.647257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:19.613482+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:19.613482+0000","last_peered":"2024-08-20T01:06:19.613482+0000","last_clean":"2024-08-20T01:06:19.613482+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:19.613482+0000","last_undegraded":"2024-08-20T01:06:19.613482+0000","last_fullsized":"2024-08-20T01:06:19.613482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900387+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:06:36.900387+0000","last_peered":"2024-08-20T01:06:36.900387+0000","last_clean":"2024-08-20T01:06:36.900387+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:06:36.900387+0000","last_undegraded":"2024-08-20T01:06:36.900387+0000","last_fullsized":"2024-08-20T01:06:36.900387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:33.638697+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:06:33.638697+0000","last_peered":"2024-08-20T01:06:33.638697+0000","last_clean":"2024-08-20T01:06:33.638697+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:06:33.638697+0000","last_undegraded":"2024-08-20T01:06:33.638697+0000","last_fullsized":"2024-08-20T01:06:33.638697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900625+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:06:36.900625+0000","last_peered":"2024-08-20T01:06:36.900625+0000","last_clean":"2024-08-20T01:06:36.900625+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:06:36.900625+0000","last_undegraded":"2024-08-20T01:06:36.900625+0000","last_fullsized":"2024-08-20T01:06:36.900625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064236+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:05:40.064236+0000","last_peered":"2024-08-20T01:05:40.064236+0000","last_clean":"2024-08-20T01:05:40.064236+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:05:40.064236+0000","last_undegraded":"2024-08-20T01:05:40.064236+0000","last_fullsized":"2024-08-20T01:05:40.064236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567030+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:06:36.567030+0000","last_peered":"2024-08-20T01:06:36.567030+0000","last_clean":"2024-08-20T01:06:36.567030+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:06:36.567030+0000","last_undegraded":"2024-08-20T01:06:36.567030+0000","last_fullsized":"2024-08-20T01:06:36.567030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562951+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:35.562951+0000","last_peered":"2024-08-20T01:06:35.562951+0000","last_clean":"2024-08-20T01:06:35.562951+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:35.562951+0000","last_undegraded":"2024-08-20T01:06:35.562951+0000","last_fullsized":"2024-08-20T01:06:35.562951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566373+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:06:36.566373+0000","last_peered":"2024-08-20T01:06:36.566373+0000","last_clean":"2024-08-20T01:06:36.566373+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:06:36.566373+0000","last_undegraded":"2024-08-20T01:06:36.566373+0000","last_fullsized":"2024-08-20T01:06:36.566373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.650696+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:06:38.650696+0000","last_peered":"2024-08-20T01:06:38.650696+0000","last_clean":"2024-08-20T01:06:38.650696+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:06:38.650696+0000","last_undegraded":"2024-08-20T01:06:38.650696+0000","last_fullsized":"2024-08-20T01:06:38.650696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642413+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:06:36.642413+0000","last_peered":"2024-08-20T01:06:36.642413+0000","last_clean":"2024-08-20T01:06:36.642413+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:06:36.642413+0000","last_undegraded":"2024-08-20T01:06:36.642413+0000","last_fullsized":"2024-08-20T01:06:36.642413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064666+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:05:40.064666+0000","last_peered":"2024-08-20T01:05:40.064666+0000","last_clean":"2024-08-20T01:05:40.064666+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:05:40.064666+0000","last_undegraded":"2024-08-20T01:05:40.064666+0000","last_fullsized":"2024-08-20T01:05:40.064666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425415+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:06:37.425415+0000","last_peered":"2024-08-20T01:06:37.425415+0000","last_clean":"2024-08-20T01:06:37.425415+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:06:37.425415+0000","last_undegraded":"2024-08-20T01:06:37.425415+0000","last_fullsized":"2024-08-20T01:06:37.425415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642659+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:06:36.642659+0000","last_peered":"2024-08-20T01:06:36.642659+0000","last_clean":"2024-08-20T01:06:36.642659+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:06:36.642659+0000","last_undegraded":"2024-08-20T01:06:36.642659+0000","last_fullsized":"2024-08-20T01:06:36.642659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.648612+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:06:37.648612+0000","last_peered":"2024-08-20T01:06:37.648612+0000","last_clean":"2024-08-20T01:06:37.648612+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:06:37.648612+0000","last_undegraded":"2024-08-20T01:06:37.648612+0000","last_fullsized":"2024-08-20T01:06:37.648612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425093+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425093+0000","last_peered":"2024-08-20T01:06:37.425093+0000","last_clean":"2024-08-20T01:06:37.425093+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425093+0000","last_undegraded":"2024-08-20T01:06:37.425093+0000","last_fullsized":"2024-08-20T01:06:37.425093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620212+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:06:36.620212+0000","last_peered":"2024-08-20T01:06:36.620212+0000","last_clean":"2024-08-20T01:06:36.620212+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:06:36.620212+0000","last_undegraded":"2024-08-20T01:06:36.620212+0000","last_fullsized":"2024-08-20T01:06:36.620212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064522+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:05:40.064522+0000","last_peered":"2024-08-20T01:05:40.064522+0000","last_clean":"2024-08-20T01:05:40.064522+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:05:40.064522+0000","last_undegraded":"2024-08-20T01:05:40.064522+0000","last_fullsized":"2024-08-20T01:05:40.064522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564203+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:06:36.564203+0000","last_peered":"2024-08-20T01:06:36.564203+0000","last_clean":"2024-08-20T01:06:36.564203+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:06:36.564203+0000","last_undegraded":"2024-08-20T01:06:36.564203+0000","last_fullsized":"2024-08-20T01:06:36.564203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647471+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:06:36.647471+0000","last_peered":"2024-08-20T01:06:36.647471+0000","last_clean":"2024-08-20T01:06:36.647471+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:06:36.647471+0000","last_undegraded":"2024-08-20T01:06:36.647471+0000","last_fullsized":"2024-08-20T01:06:36.647471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563035+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:35.563035+0000","last_peered":"2024-08-20T01:06:35.563035+0000","last_clean":"2024-08-20T01:06:35.563035+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:35.563035+0000","last_undegraded":"2024-08-20T01:06:35.563035+0000","last_fullsized":"2024-08-20T01:06:35.563035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.649367+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:06:37.649367+0000","last_peered":"2024-08-20T01:06:37.649367+0000","last_clean":"2024-08-20T01:06:37.649367+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:06:37.649367+0000","last_undegraded":"2024-08-20T01:06:37.649367+0000","last_fullsized":"2024-08-20T01:06:37.649367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562930+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:35.562930+0000","last_peered":"2024-08-20T01:06:35.562930+0000","last_clean":"2024-08-20T01:06:35.562930+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:35.562930+0000","last_undegraded":"2024-08-20T01:06:35.562930+0000","last_fullsized":"2024-08-20T01:06:35.562930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.062891+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:05:40.062891+0000","last_peered":"2024-08-20T01:05:40.062891+0000","last_clean":"2024-08-20T01:05:40.062891+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:05:40.062891+0000","last_undegraded":"2024-08-20T01:05:40.062891+0000","last_fullsized":"2024-08-20T01:05:40.062891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563347+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:06:35.563347+0000","last_peered":"2024-08-20T01:06:35.563347+0000","last_clean":"2024-08-20T01:06:35.563347+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:06:35.563347+0000","last_undegraded":"2024-08-20T01:06:35.563347+0000","last_fullsized":"2024-08-20T01:06:35.563347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562790+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:06:35.562790+0000","last_peered":"2024-08-20T01:06:35.562790+0000","last_clean":"2024-08-20T01:06:35.562790+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:06:35.562790+0000","last_undegraded":"2024-08-20T01:06:35.562790+0000","last_fullsized":"2024-08-20T01:06:35.562790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425165+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:06:37.425165+0000","last_peered":"2024-08-20T01:06:37.425165+0000","last_clean":"2024-08-20T01:06:37.425165+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:06:37.425165+0000","last_undegraded":"2024-08-20T01:06:37.425165+0000","last_fullsized":"2024-08-20T01:06:37.425165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567032+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:06:36.567032+0000","last_peered":"2024-08-20T01:06:36.567032+0000","last_clean":"2024-08-20T01:06:36.567032+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:06:36.567032+0000","last_undegraded":"2024-08-20T01:06:36.567032+0000","last_fullsized":"2024-08-20T01:06:36.567032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064523+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:05:40.064523+0000","last_peered":"2024-08-20T01:05:40.064523+0000","last_clean":"2024-08-20T01:05:40.064523+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:05:40.064523+0000","last_undegraded":"2024-08-20T01:05:40.064523+0000","last_fullsized":"2024-08-20T01:05:40.064523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563134+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:06:35.563134+0000","last_peered":"2024-08-20T01:06:35.563134+0000","last_clean":"2024-08-20T01:06:35.563134+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:06:35.563134+0000","last_undegraded":"2024-08-20T01:06:35.563134+0000","last_fullsized":"2024-08-20T01:06:35.563134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564876+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:06:36.564876+0000","last_peered":"2024-08-20T01:06:36.564876+0000","last_clean":"2024-08-20T01:06:36.564876+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:06:36.564876+0000","last_undegraded":"2024-08-20T01:06:36.564876+0000","last_fullsized":"2024-08-20T01:06:36.564876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647031+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:06:36.647031+0000","last_peered":"2024-08-20T01:06:36.647031+0000","last_clean":"2024-08-20T01:06:36.647031+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:06:36.647031+0000","last_undegraded":"2024-08-20T01:06:36.647031+0000","last_fullsized":"2024-08-20T01:06:36.647031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564381+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:06:36.564381+0000","last_peered":"2024-08-20T01:06:36.564381+0000","last_clean":"2024-08-20T01:06:36.564381+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:06:36.564381+0000","last_undegraded":"2024-08-20T01:06:36.564381+0000","last_fullsized":"2024-08-20T01:06:36.564381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642299+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:06:36.642299+0000","last_peered":"2024-08-20T01:06:36.642299+0000","last_clean":"2024-08-20T01:06:36.642299+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:06:36.642299+0000","last_undegraded":"2024-08-20T01:06:36.642299+0000","last_fullsized":"2024-08-20T01:06:36.642299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566850+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:06:36.566850+0000","last_peered":"2024-08-20T01:06:36.566850+0000","last_clean":"2024-08-20T01:06:36.566850+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:06:36.566850+0000","last_undegraded":"2024-08-20T01:06:36.566850+0000","last_fullsized":"2024-08-20T01:06:36.566850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064174+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:05:40.064174+0000","last_peered":"2024-08-20T01:05:40.064174+0000","last_clean":"2024-08-20T01:05:40.064174+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:05:40.064174+0000","last_undegraded":"2024-08-20T01:05:40.064174+0000","last_fullsized":"2024-08-20T01:05:40.064174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563052+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:06:35.563052+0000","last_peered":"2024-08-20T01:06:35.563052+0000","last_clean":"2024-08-20T01:06:35.563052+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:06:35.563052+0000","last_undegraded":"2024-08-20T01:06:35.563052+0000","last_fullsized":"2024-08-20T01:06:35.563052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620898+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:06:36.620898+0000","last_peered":"2024-08-20T01:06:36.620898+0000","last_clean":"2024-08-20T01:06:36.620898+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:06:36.620898+0000","last_undegraded":"2024-08-20T01:06:36.620898+0000","last_fullsized":"2024-08-20T01:06:36.620898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064709+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:05:40.064709+0000","last_peered":"2024-08-20T01:05:40.064709+0000","last_clean":"2024-08-20T01:05:40.064709+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:05:40.064709+0000","last_undegraded":"2024-08-20T01:05:40.064709+0000","last_fullsized":"2024-08-20T01:05:40.064709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642084+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:06:36.642084+0000","last_peered":"2024-08-20T01:06:36.642084+0000","last_clean":"2024-08-20T01:06:36.642084+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:06:36.642084+0000","last_undegraded":"2024-08-20T01:06:36.642084+0000","last_fullsized":"2024-08-20T01:06:36.642084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425202+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:06:37.425202+0000","last_peered":"2024-08-20T01:06:37.425202+0000","last_clean":"2024-08-20T01:06:37.425202+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:06:37.425202+0000","last_undegraded":"2024-08-20T01:06:37.425202+0000","last_fullsized":"2024-08-20T01:06:37.425202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.424796+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:06:37.424796+0000","last_peered":"2024-08-20T01:06:37.424796+0000","last_clean":"2024-08-20T01:06:37.424796+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:06:37.424796+0000","last_undegraded":"2024-08-20T01:06:37.424796+0000","last_fullsized":"2024-08-20T01:06:37.424796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566819+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:06:36.566819+0000","last_peered":"2024-08-20T01:06:36.566819+0000","last_clean":"2024-08-20T01:06:36.566819+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:06:36.566819+0000","last_undegraded":"2024-08-20T01:06:36.566819+0000","last_fullsized":"2024-08-20T01:06:36.566819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064564+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:05:40.064564+0000","last_peered":"2024-08-20T01:05:40.064564+0000","last_clean":"2024-08-20T01:05:40.064564+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:05:40.064564+0000","last_undegraded":"2024-08-20T01:05:40.064564+0000","last_fullsized":"2024-08-20T01:05:40.064564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900949+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:06:36.900949+0000","last_peered":"2024-08-20T01:06:36.900949+0000","last_clean":"2024-08-20T01:06:36.900949+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:06:36.900949+0000","last_undegraded":"2024-08-20T01:06:36.900949+0000","last_fullsized":"2024-08-20T01:06:36.900949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.645012+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:06:36.645012+0000","last_peered":"2024-08-20T01:06:36.645012+0000","last_clean":"2024-08-20T01:06:36.645012+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:06:36.645012+0000","last_undegraded":"2024-08-20T01:06:36.645012+0000","last_fullsized":"2024-08-20T01:06:36.645012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642836+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:06:36.642836+0000","last_peered":"2024-08-20T01:06:36.642836+0000","last_clean":"2024-08-20T01:06:36.642836+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:06:36.642836+0000","last_undegraded":"2024-08-20T01:06:36.642836+0000","last_fullsized":"2024-08-20T01:06:36.642836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620109+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:06:36.620109+0000","last_peered":"2024-08-20T01:06:36.620109+0000","last_clean":"2024-08-20T01:06:36.620109+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:06:36.620109+0000","last_undegraded":"2024-08-20T01:06:36.620109+0000","last_fullsized":"2024-08-20T01:06:36.620109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900485+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:06:36.900485+0000","last_peered":"2024-08-20T01:06:36.900485+0000","last_clean":"2024-08-20T01:06:36.900485+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:06:36.900485+0000","last_undegraded":"2024-08-20T01:06:36.900485+0000","last_fullsized":"2024-08-20T01:06:36.900485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644298+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:06:36.644298+0000","last_peered":"2024-08-20T01:06:36.644298+0000","last_clean":"2024-08-20T01:06:36.644298+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:06:36.644298+0000","last_undegraded":"2024-08-20T01:06:36.644298+0000","last_fullsized":"2024-08-20T01:06:36.644298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567413+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:06:36.567413+0000","last_peered":"2024-08-20T01:06:36.567413+0000","last_clean":"2024-08-20T01:06:36.567413+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:06:36.567413+0000","last_undegraded":"2024-08-20T01:06:36.567413+0000","last_fullsized":"2024-08-20T01:06:36.567413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620731+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:06:36.620731+0000","last_peered":"2024-08-20T01:06:36.620731+0000","last_clean":"2024-08-20T01:06:36.620731+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:06:36.620731+0000","last_undegraded":"2024-08-20T01:06:36.620731+0000","last_fullsized":"2024-08-20T01:06:36.620731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425241+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:06:37.425241+0000","last_peered":"2024-08-20T01:06:37.425241+0000","last_clean":"2024-08-20T01:06:37.425241+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:06:37.425241+0000","last_undegraded":"2024-08-20T01:06:37.425241+0000","last_fullsized":"2024-08-20T01:06:37.425241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425195+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:06:37.425195+0000","last_peered":"2024-08-20T01:06:37.425195+0000","last_clean":"2024-08-20T01:06:37.425195+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:06:37.425195+0000","last_undegraded":"2024-08-20T01:06:37.425195+0000","last_fullsized":"2024-08-20T01:06:37.425195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064704+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:05:40.064704+0000","last_peered":"2024-08-20T01:05:40.064704+0000","last_clean":"2024-08-20T01:05:40.064704+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:05:40.064704+0000","last_undegraded":"2024-08-20T01:05:40.064704+0000","last_fullsized":"2024-08-20T01:05:40.064704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60161,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":171953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":498014,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":1156343,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":507627,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1113245,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":1108833,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":497964,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":546860,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":554800,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:06:41.326 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:06:41.697 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:41.836 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:41 smithi138 ceph-mon[18076]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-08-20T01:06:41.836 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:41 smithi138 ceph-mon[18076]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:41 smithi167 ceph-mon[21965]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-08-20T01:06:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:41 smithi167 ceph-mon[21965]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:41 smithi142 ceph-mon[21799]: pgmap v319: 129 pgs: 1 activating, 1 peering, 127 active+clean; 636 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.4 KiB/s wr, 9 op/s 2024-08-20T01:06:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:41 smithi142 ceph-mon[21799]: from='client.24959 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 -- 172.21.15.138:0/1197528108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077d20 msgr2=0x7f77f0112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 --2- 172.21.15.138:0/1197528108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077d20 0x7f77f0112fb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77dc009900 tx=0x7f77dc02a410 comp rx=0 tx=0).stop 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 -- 172.21.15.138:0/1197528108 shutdown_connections 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 --2- 172.21.15.138:0/1197528108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 --2- 172.21.15.138:0/1197528108 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077d20 0x7f77f0112fb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 --2- 172.21.15.138:0/1197528108 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077400 0x7f77f00777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 -- 172.21.15.138:0/1197528108 >> 172.21.15.138:0/1197528108 conn(0x7f77f0100550 msgr2=0x7f77f0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 -- 172.21.15.138:0/1197528108 shutdown_connections 2024-08-20T01:06:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.293+0000 7f77f4a00640 1 -- 172.21.15.138:0/1197528108 wait complete. 2024-08-20T01:06:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 Processor -- start 2024-08-20T01:06:42.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 -- start start 2024-08-20T01:06:42.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 0x7f77f019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:42.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077d20 0x7f77f019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.294+0000 7f77f4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0118f50 con 0x7f77f0077400 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77f4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77f0118dd0 con 0x7f77f01134f0 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77f4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77f01190d0 con 0x7f77f0077d20 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 0x7f77f019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77ef400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 0x7f77f019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36740/0 (socket says 172.21.15.138:36740) 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77ef400640 1 -- 172.21.15.138:0/3374319595 learned_addr learned my addr 172.21.15.138:0/3374319595 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77eea00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077d20 0x7f77f019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 -- 172.21.15.138:0/3374319595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077d20 msgr2=0x7f77f019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077d20 0x7f77f019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 -- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 msgr2=0x7f77f019f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:42.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 0x7f77f019f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.295+0000 7f77efe00640 1 -- 172.21.15.138:0/3374319595 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f01a53e0 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77efe00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01a4c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e8003b70 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77e8003d10 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e80107a0 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f01a56d0 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.296+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f01abd90 con 0x7f77f01134f0 2024-08-20T01:06:42.297 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.297+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f00777e0 con 0x7f77f01134f0 2024-08-20T01:06:42.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.299+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f77e8010940 con 0x7f77f01134f0 2024-08-20T01:06:42.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.301+0000 7f77e7e00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 0x7f77d40b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:42.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.301+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f77e80e4840 con 0x7f77f01134f0 2024-08-20T01:06:42.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.301+0000 7f77ef400640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 0x7f77d40b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:42.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.302+0000 7f77ef400640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 0x7f77d40b62a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009cd0 tx=0x7f77e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:42.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.302+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f77e80ab8f0 con 0x7f77f01134f0 2024-08-20T01:06:42.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.482+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77f0115580 con 0x7f77d40b3de0 2024-08-20T01:06:42.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.489+0000 7f77e7e00640 1 -- 172.21.15.138:0/3374319595 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340543 (secure 0 0 0) 0x7f77f0115580 con 0x7f77d40b3de0 2024-08-20T01:06:42.492 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:42.493 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 msgr2=0x7f77d40b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 0x7f77d40b62a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f77e0009cd0 tx=0x7f77e0005f90 comp rx=0 tx=0).stop 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 msgr2=0x7f77f01a4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01a4c10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f77e8005fd0 tx=0x7f77e800c6a0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 shutdown_connections 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77d40b3de0 0x7f77d40b62a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77f01134f0 0x7f77f01a4c10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77f0077d20 0x7f77f019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 --2- 172.21.15.138:0/3374319595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77f0077400 0x7f77f019f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:42.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 >> 172.21.15.138:0/3374319595 conn(0x7f77f0100550 msgr2=0x7f77f0102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:42.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 shutdown_connections 2024-08-20T01:06:42.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:42.494+0000 7f77f4a00640 1 -- 172.21.15.138:0/3374319595 wait complete. 2024-08-20T01:06:42.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:42 smithi138 ceph-mon[18076]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:42 smithi138 ceph-mon[18076]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-08-20T01:06:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:42 smithi138 ceph-mon[18076]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-08-20T01:06:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:42 smithi138 ceph-mon[18076]: Cluster is now healthy 2024-08-20T01:06:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:42 smithi167 ceph-mon[21965]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:42 smithi167 ceph-mon[21965]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-08-20T01:06:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:42 smithi167 ceph-mon[21965]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-08-20T01:06:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:42 smithi167 ceph-mon[21965]: Cluster is now healthy 2024-08-20T01:06:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:42 smithi142 ceph-mon[21799]: from='client.24965 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:42 smithi142 ceph-mon[21799]: pgmap v320: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.1 KiB/s wr, 9 op/s 2024-08-20T01:06:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:42 smithi142 ceph-mon[21799]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-08-20T01:06:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:42 smithi142 ceph-mon[21799]: Cluster is now healthy 2024-08-20T01:06:43.271 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-08-20T01:06:40.485184+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650763,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":218,"num_read_kb":200,"num_write":288,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":418,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352716,"kb_used_data":11744,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589108,"statfs":{"total":1151940427776,"available":1151579246592,"internally_reserved":0,"allocated":12025856,"data_stored":8044556,"data_compressed":403487,"data_compressed_allocated":544768,"data_compressed_original":2588672,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27070,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":58,"num_read_kb":58,"num_write":21,"num_write_kb":35,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621083+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621083+0000","last_peered":"2024-08-20T01:06:36.621083+0000","last_clean":"2024-08-20T01:06:36.621083+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621083+0000","last_undegraded":"2024-08-20T01:06:36.621083+0000","last_fullsized":"2024-08-20T01:06:36.621083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564621+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:06:36.564621+0000","last_peered":"2024-08-20T01:06:36.564621+0000","last_clean":"2024-08-20T01:06:36.564621+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:06:36.564621+0000","last_undegraded":"2024-08-20T01:06:36.564621+0000","last_fullsized":"2024-08-20T01:06:36.564621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":86,"state":"peering","last_fresh":"2024-08-20T01:05:40.063072+0000","last_change":"2024-08-20T01:06:35.563750+0000","last_active":"2024-08-20T01:06:35.563750+0000","last_peered":"2024-08-20T01:05:40.063072+0000","last_clean":"2024-08-20T01:05:40.063072+0000","last_became_active":"2024-08-20T01:05:35.968597+0000","last_became_peered":"2024-08-20T01:05:35.968597+0000","last_unstale":"2024-08-20T01:05:40.063072+0000","last_undegraded":"2024-08-20T01:05:40.063072+0000","last_fullsized":"2024-08-20T01:05:40.063072+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,3,8,9],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564871+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564871+0000","last_peered":"2024-08-20T01:06:36.564871+0000","last_clean":"2024-08-20T01:06:36.564871+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564871+0000","last_undegraded":"2024-08-20T01:06:36.564871+0000","last_fullsized":"2024-08-20T01:06:36.564871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425408+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:06:37.425408+0000","last_peered":"2024-08-20T01:06:37.425408+0000","last_clean":"2024-08-20T01:06:37.425408+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:06:37.425408+0000","last_undegraded":"2024-08-20T01:06:37.425408+0000","last_fullsized":"2024-08-20T01:06:37.425408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063059+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:05:40.063059+0000","last_peered":"2024-08-20T01:05:40.063059+0000","last_clean":"2024-08-20T01:05:40.063059+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:05:40.063059+0000","last_undegraded":"2024-08-20T01:05:40.063059+0000","last_fullsized":"2024-08-20T01:05:40.063059+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642316+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:06:36.642316+0000","last_peered":"2024-08-20T01:06:36.642316+0000","last_clean":"2024-08-20T01:06:36.642316+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:06:36.642316+0000","last_undegraded":"2024-08-20T01:06:36.642316+0000","last_fullsized":"2024-08-20T01:06:36.642316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064464+0000","last_change":"2024-08-20T01:05:35.969178+0000","last_active":"2024-08-20T01:05:40.064464+0000","last_peered":"2024-08-20T01:05:40.064464+0000","last_clean":"2024-08-20T01:05:40.064464+0000","last_became_active":"2024-08-20T01:05:35.968411+0000","last_became_peered":"2024-08-20T01:05:35.968411+0000","last_unstale":"2024-08-20T01:05:40.064464+0000","last_undegraded":"2024-08-20T01:05:40.064464+0000","last_fullsized":"2024-08-20T01:05:40.064464+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:42:43.856816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647417+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:06:36.647417+0000","last_peered":"2024-08-20T01:06:36.647417+0000","last_clean":"2024-08-20T01:06:36.647417+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:06:36.647417+0000","last_undegraded":"2024-08-20T01:06:36.647417+0000","last_fullsized":"2024-08-20T01:06:36.647417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064093+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:05:40.064093+0000","last_peered":"2024-08-20T01:05:40.064093+0000","last_clean":"2024-08-20T01:05:40.064093+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:05:40.064093+0000","last_undegraded":"2024-08-20T01:05:40.064093+0000","last_fullsized":"2024-08-20T01:05:40.064093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900542+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:06:36.900542+0000","last_peered":"2024-08-20T01:06:36.900542+0000","last_clean":"2024-08-20T01:06:36.900542+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:06:36.900542+0000","last_undegraded":"2024-08-20T01:06:36.900542+0000","last_fullsized":"2024-08-20T01:06:36.900542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063325+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:05:40.063325+0000","last_peered":"2024-08-20T01:05:40.063325+0000","last_clean":"2024-08-20T01:05:40.063325+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:05:40.063325+0000","last_undegraded":"2024-08-20T01:05:40.063325+0000","last_fullsized":"2024-08-20T01:05:40.063325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564754+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:06:36.564754+0000","last_peered":"2024-08-20T01:06:36.564754+0000","last_clean":"2024-08-20T01:06:36.564754+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:06:36.564754+0000","last_undegraded":"2024-08-20T01:06:36.564754+0000","last_fullsized":"2024-08-20T01:06:36.564754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":40,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563108+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:35.563108+0000","last_peered":"2024-08-20T01:06:35.563108+0000","last_clean":"2024-08-20T01:06:35.563108+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:35.563108+0000","last_undegraded":"2024-08-20T01:06:35.563108+0000","last_fullsized":"2024-08-20T01:06:35.563108+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064434+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:05:40.064434+0000","last_peered":"2024-08-20T01:05:40.064434+0000","last_clean":"2024-08-20T01:05:40.064434+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:05:40.064434+0000","last_undegraded":"2024-08-20T01:05:40.064434+0000","last_fullsized":"2024-08-20T01:05:40.064434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.062918+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:05:40.062918+0000","last_peered":"2024-08-20T01:05:40.062918+0000","last_clean":"2024-08-20T01:05:40.062918+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:05:40.062918+0000","last_undegraded":"2024-08-20T01:05:40.062918+0000","last_fullsized":"2024-08-20T01:05:40.062918+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564536+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564536+0000","last_peered":"2024-08-20T01:06:36.564536+0000","last_clean":"2024-08-20T01:06:36.564536+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564536+0000","last_undegraded":"2024-08-20T01:06:36.564536+0000","last_fullsized":"2024-08-20T01:06:36.564536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567485+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:06:36.567485+0000","last_peered":"2024-08-20T01:06:36.567485+0000","last_clean":"2024-08-20T01:06:36.567485+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:06:36.567485+0000","last_undegraded":"2024-08-20T01:06:36.567485+0000","last_fullsized":"2024-08-20T01:06:36.567485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642212+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642212+0000","last_peered":"2024-08-20T01:06:36.642212+0000","last_clean":"2024-08-20T01:06:36.642212+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642212+0000","last_undegraded":"2024-08-20T01:06:36.642212+0000","last_fullsized":"2024-08-20T01:06:36.642212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.063575+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:05:40.063575+0000","last_peered":"2024-08-20T01:05:40.063575+0000","last_clean":"2024-08-20T01:05:40.063575+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:05:40.063575+0000","last_undegraded":"2024-08-20T01:05:40.063575+0000","last_fullsized":"2024-08-20T01:05:40.063575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425236+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:06:37.425236+0000","last_peered":"2024-08-20T01:06:37.425236+0000","last_clean":"2024-08-20T01:06:37.425236+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:06:37.425236+0000","last_undegraded":"2024-08-20T01:06:37.425236+0000","last_fullsized":"2024-08-20T01:06:37.425236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":41,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:18.605774+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:18.605774+0000","last_peered":"2024-08-20T01:06:18.605774+0000","last_clean":"2024-08-20T01:06:18.605774+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:18.605774+0000","last_undegraded":"2024-08-20T01:06:18.605774+0000","last_fullsized":"2024-08-20T01:06:18.605774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563129+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:35.563129+0000","last_peered":"2024-08-20T01:06:35.563129+0000","last_clean":"2024-08-20T01:06:35.563129+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:35.563129+0000","last_undegraded":"2024-08-20T01:06:35.563129+0000","last_fullsized":"2024-08-20T01:06:35.563129+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564938+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:06:36.564938+0000","last_peered":"2024-08-20T01:06:36.564938+0000","last_clean":"2024-08-20T01:06:36.564938+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:06:36.564938+0000","last_undegraded":"2024-08-20T01:06:36.564938+0000","last_fullsized":"2024-08-20T01:06:36.564938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.622441+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:06:36.622441+0000","last_peered":"2024-08-20T01:06:36.622441+0000","last_clean":"2024-08-20T01:06:36.622441+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:06:36.622441+0000","last_undegraded":"2024-08-20T01:06:36.622441+0000","last_fullsized":"2024-08-20T01:06:36.622441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564087+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564087+0000","last_peered":"2024-08-20T01:06:36.564087+0000","last_clean":"2024-08-20T01:06:36.564087+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564087+0000","last_undegraded":"2024-08-20T01:06:36.564087+0000","last_fullsized":"2024-08-20T01:06:36.564087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567052+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:06:36.567052+0000","last_peered":"2024-08-20T01:06:36.567052+0000","last_clean":"2024-08-20T01:06:36.567052+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:06:36.567052+0000","last_undegraded":"2024-08-20T01:06:36.567052+0000","last_fullsized":"2024-08-20T01:06:36.567052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566662+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566662+0000","last_peered":"2024-08-20T01:06:36.566662+0000","last_clean":"2024-08-20T01:06:36.566662+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566662+0000","last_undegraded":"2024-08-20T01:06:36.566662+0000","last_fullsized":"2024-08-20T01:06:36.566662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064483+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:05:40.064483+0000","last_peered":"2024-08-20T01:05:40.064483+0000","last_clean":"2024-08-20T01:05:40.064483+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:05:40.064483+0000","last_undegraded":"2024-08-20T01:05:40.064483+0000","last_fullsized":"2024-08-20T01:05:40.064483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644635+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:06:36.644635+0000","last_peered":"2024-08-20T01:06:36.644635+0000","last_clean":"2024-08-20T01:06:36.644635+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:06:36.644635+0000","last_undegraded":"2024-08-20T01:06:36.644635+0000","last_fullsized":"2024-08-20T01:06:36.644635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620802+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620802+0000","last_peered":"2024-08-20T01:06:36.620802+0000","last_clean":"2024-08-20T01:06:36.620802+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620802+0000","last_undegraded":"2024-08-20T01:06:36.620802+0000","last_fullsized":"2024-08-20T01:06:36.620802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644713+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:06:36.644713+0000","last_peered":"2024-08-20T01:06:36.644713+0000","last_clean":"2024-08-20T01:06:36.644713+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:06:36.644713+0000","last_undegraded":"2024-08-20T01:06:36.644713+0000","last_fullsized":"2024-08-20T01:06:36.644713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900709+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900709+0000","last_peered":"2024-08-20T01:06:36.900709+0000","last_clean":"2024-08-20T01:06:36.900709+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900709+0000","last_undegraded":"2024-08-20T01:06:36.900709+0000","last_fullsized":"2024-08-20T01:06:36.900709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566848+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566848+0000","last_peered":"2024-08-20T01:06:36.566848+0000","last_clean":"2024-08-20T01:06:36.566848+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566848+0000","last_undegraded":"2024-08-20T01:06:36.566848+0000","last_fullsized":"2024-08-20T01:06:36.566848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647086+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:06:36.647086+0000","last_peered":"2024-08-20T01:06:36.647086+0000","last_clean":"2024-08-20T01:06:36.647086+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:06:36.647086+0000","last_undegraded":"2024-08-20T01:06:36.647086+0000","last_fullsized":"2024-08-20T01:06:36.647086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.645366+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:06:38.645366+0000","last_peered":"2024-08-20T01:06:38.645366+0000","last_clean":"2024-08-20T01:06:38.645366+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:06:38.645366+0000","last_undegraded":"2024-08-20T01:06:38.645366+0000","last_fullsized":"2024-08-20T01:06:38.645366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566529+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:06:36.566529+0000","last_peered":"2024-08-20T01:06:36.566529+0000","last_clean":"2024-08-20T01:06:36.566529+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:06:36.566529+0000","last_undegraded":"2024-08-20T01:06:36.566529+0000","last_fullsized":"2024-08-20T01:06:36.566529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900529+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900529+0000","last_peered":"2024-08-20T01:06:36.900529+0000","last_clean":"2024-08-20T01:06:36.900529+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900529+0000","last_undegraded":"2024-08-20T01:06:36.900529+0000","last_fullsized":"2024-08-20T01:06:36.900529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564431+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:06:36.564431+0000","last_peered":"2024-08-20T01:06:36.564431+0000","last_clean":"2024-08-20T01:06:36.564431+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:06:36.564431+0000","last_undegraded":"2024-08-20T01:06:36.564431+0000","last_fullsized":"2024-08-20T01:06:36.564431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:20.615714+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:20.615714+0000","last_peered":"2024-08-20T01:06:20.615714+0000","last_clean":"2024-08-20T01:06:20.615714+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:20.615714+0000","last_undegraded":"2024-08-20T01:06:20.615714+0000","last_fullsized":"2024-08-20T01:06:20.615714+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900668+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900668+0000","last_peered":"2024-08-20T01:06:36.900668+0000","last_clean":"2024-08-20T01:06:36.900668+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900668+0000","last_undegraded":"2024-08-20T01:06:36.900668+0000","last_fullsized":"2024-08-20T01:06:36.900668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901014+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:06:36.901014+0000","last_peered":"2024-08-20T01:06:36.901014+0000","last_clean":"2024-08-20T01:06:36.901014+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:06:36.901014+0000","last_undegraded":"2024-08-20T01:06:36.901014+0000","last_fullsized":"2024-08-20T01:06:36.901014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":43,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563387+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:35.563387+0000","last_peered":"2024-08-20T01:06:35.563387+0000","last_clean":"2024-08-20T01:06:35.563387+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:35.563387+0000","last_undegraded":"2024-08-20T01:06:35.563387+0000","last_fullsized":"2024-08-20T01:06:35.563387+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900715+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:06:36.900715+0000","last_peered":"2024-08-20T01:06:36.900715+0000","last_clean":"2024-08-20T01:06:36.900715+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:06:36.900715+0000","last_undegraded":"2024-08-20T01:06:36.900715+0000","last_fullsized":"2024-08-20T01:06:36.900715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647579+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647579+0000","last_peered":"2024-08-20T01:06:36.647579+0000","last_clean":"2024-08-20T01:06:36.647579+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647579+0000","last_undegraded":"2024-08-20T01:06:36.647579+0000","last_fullsized":"2024-08-20T01:06:36.647579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900966+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:06:36.900966+0000","last_peered":"2024-08-20T01:06:36.900966+0000","last_clean":"2024-08-20T01:06:36.900966+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:06:36.900966+0000","last_undegraded":"2024-08-20T01:06:36.900966+0000","last_fullsized":"2024-08-20T01:06:36.900966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567528+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567528+0000","last_peered":"2024-08-20T01:06:36.567528+0000","last_clean":"2024-08-20T01:06:36.567528+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567528+0000","last_undegraded":"2024-08-20T01:06:36.567528+0000","last_fullsized":"2024-08-20T01:06:36.567528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064549+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:05:40.064549+0000","last_peered":"2024-08-20T01:05:40.064549+0000","last_clean":"2024-08-20T01:05:40.064549+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:05:40.064549+0000","last_undegraded":"2024-08-20T01:05:40.064549+0000","last_fullsized":"2024-08-20T01:05:40.064549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":5,"reported_epoch":88,"state":"activating","last_fresh":"2024-08-20T01:06:36.568460+0000","last_change":"2024-08-20T01:06:36.567147+0000","last_active":"2024-08-20T01:05:34.937557+0000","last_peered":"2024-08-20T01:05:34.937557+0000","last_clean":"2024-08-20T01:05:34.937557+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-08-20T01:06:36.568460+0000","last_undegraded":"2024-08-20T01:06:36.568460+0000","last_fullsized":"2024-08-20T01:06:36.568460+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":["8(0)","0(1)","1(3)","3(2)","5(0)"],"object_location_counts":[],"blocked_by":[1,3],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647257+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:06:36.647257+0000","last_peered":"2024-08-20T01:06:36.647257+0000","last_clean":"2024-08-20T01:06:36.647257+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:06:36.647257+0000","last_undegraded":"2024-08-20T01:06:36.647257+0000","last_fullsized":"2024-08-20T01:06:36.647257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:19.613482+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:19.613482+0000","last_peered":"2024-08-20T01:06:19.613482+0000","last_clean":"2024-08-20T01:06:19.613482+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:19.613482+0000","last_undegraded":"2024-08-20T01:06:19.613482+0000","last_fullsized":"2024-08-20T01:06:19.613482+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900387+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:06:36.900387+0000","last_peered":"2024-08-20T01:06:36.900387+0000","last_clean":"2024-08-20T01:06:36.900387+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:06:36.900387+0000","last_undegraded":"2024-08-20T01:06:36.900387+0000","last_fullsized":"2024-08-20T01:06:36.900387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":57,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:06:33.638697+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:06:33.638697+0000","last_peered":"2024-08-20T01:06:33.638697+0000","last_clean":"2024-08-20T01:06:33.638697+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:06:33.638697+0000","last_undegraded":"2024-08-20T01:06:33.638697+0000","last_fullsized":"2024-08-20T01:06:33.638697+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900625+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:06:36.900625+0000","last_peered":"2024-08-20T01:06:36.900625+0000","last_clean":"2024-08-20T01:06:36.900625+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:06:36.900625+0000","last_undegraded":"2024-08-20T01:06:36.900625+0000","last_fullsized":"2024-08-20T01:06:36.900625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064236+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:05:40.064236+0000","last_peered":"2024-08-20T01:05:40.064236+0000","last_clean":"2024-08-20T01:05:40.064236+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:05:40.064236+0000","last_undegraded":"2024-08-20T01:05:40.064236+0000","last_fullsized":"2024-08-20T01:05:40.064236+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567030+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:06:36.567030+0000","last_peered":"2024-08-20T01:06:36.567030+0000","last_clean":"2024-08-20T01:06:36.567030+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:06:36.567030+0000","last_undegraded":"2024-08-20T01:06:36.567030+0000","last_fullsized":"2024-08-20T01:06:36.567030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":46,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562951+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:35.562951+0000","last_peered":"2024-08-20T01:06:35.562951+0000","last_clean":"2024-08-20T01:06:35.562951+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:35.562951+0000","last_undegraded":"2024-08-20T01:06:35.562951+0000","last_fullsized":"2024-08-20T01:06:35.562951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566373+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:06:36.566373+0000","last_peered":"2024-08-20T01:06:36.566373+0000","last_clean":"2024-08-20T01:06:36.566373+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:06:36.566373+0000","last_undegraded":"2024-08-20T01:06:36.566373+0000","last_fullsized":"2024-08-20T01:06:36.566373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.650696+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:06:38.650696+0000","last_peered":"2024-08-20T01:06:38.650696+0000","last_clean":"2024-08-20T01:06:38.650696+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:06:38.650696+0000","last_undegraded":"2024-08-20T01:06:38.650696+0000","last_fullsized":"2024-08-20T01:06:38.650696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642413+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:06:36.642413+0000","last_peered":"2024-08-20T01:06:36.642413+0000","last_clean":"2024-08-20T01:06:36.642413+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:06:36.642413+0000","last_undegraded":"2024-08-20T01:06:36.642413+0000","last_fullsized":"2024-08-20T01:06:36.642413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064666+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:05:40.064666+0000","last_peered":"2024-08-20T01:05:40.064666+0000","last_clean":"2024-08-20T01:05:40.064666+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:05:40.064666+0000","last_undegraded":"2024-08-20T01:05:40.064666+0000","last_fullsized":"2024-08-20T01:05:40.064666+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425415+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:06:37.425415+0000","last_peered":"2024-08-20T01:06:37.425415+0000","last_clean":"2024-08-20T01:06:37.425415+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:06:37.425415+0000","last_undegraded":"2024-08-20T01:06:37.425415+0000","last_fullsized":"2024-08-20T01:06:37.425415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642659+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:06:36.642659+0000","last_peered":"2024-08-20T01:06:36.642659+0000","last_clean":"2024-08-20T01:06:36.642659+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:06:36.642659+0000","last_undegraded":"2024-08-20T01:06:36.642659+0000","last_fullsized":"2024-08-20T01:06:36.642659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.648612+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:06:37.648612+0000","last_peered":"2024-08-20T01:06:37.648612+0000","last_clean":"2024-08-20T01:06:37.648612+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:06:37.648612+0000","last_undegraded":"2024-08-20T01:06:37.648612+0000","last_fullsized":"2024-08-20T01:06:37.648612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425093+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425093+0000","last_peered":"2024-08-20T01:06:37.425093+0000","last_clean":"2024-08-20T01:06:37.425093+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425093+0000","last_undegraded":"2024-08-20T01:06:37.425093+0000","last_fullsized":"2024-08-20T01:06:37.425093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620212+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:06:36.620212+0000","last_peered":"2024-08-20T01:06:36.620212+0000","last_clean":"2024-08-20T01:06:36.620212+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:06:36.620212+0000","last_undegraded":"2024-08-20T01:06:36.620212+0000","last_fullsized":"2024-08-20T01:06:36.620212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064522+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:05:40.064522+0000","last_peered":"2024-08-20T01:05:40.064522+0000","last_clean":"2024-08-20T01:05:40.064522+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:05:40.064522+0000","last_undegraded":"2024-08-20T01:05:40.064522+0000","last_fullsized":"2024-08-20T01:05:40.064522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564203+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:06:36.564203+0000","last_peered":"2024-08-20T01:06:36.564203+0000","last_clean":"2024-08-20T01:06:36.564203+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:06:36.564203+0000","last_undegraded":"2024-08-20T01:06:36.564203+0000","last_fullsized":"2024-08-20T01:06:36.564203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647471+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:06:36.647471+0000","last_peered":"2024-08-20T01:06:36.647471+0000","last_clean":"2024-08-20T01:06:36.647471+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:06:36.647471+0000","last_undegraded":"2024-08-20T01:06:36.647471+0000","last_fullsized":"2024-08-20T01:06:36.647471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563035+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:35.563035+0000","last_peered":"2024-08-20T01:06:35.563035+0000","last_clean":"2024-08-20T01:06:35.563035+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:35.563035+0000","last_undegraded":"2024-08-20T01:06:35.563035+0000","last_fullsized":"2024-08-20T01:06:35.563035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.649367+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:06:37.649367+0000","last_peered":"2024-08-20T01:06:37.649367+0000","last_clean":"2024-08-20T01:06:37.649367+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:06:37.649367+0000","last_undegraded":"2024-08-20T01:06:37.649367+0000","last_fullsized":"2024-08-20T01:06:37.649367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562930+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:35.562930+0000","last_peered":"2024-08-20T01:06:35.562930+0000","last_clean":"2024-08-20T01:06:35.562930+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:35.562930+0000","last_undegraded":"2024-08-20T01:06:35.562930+0000","last_fullsized":"2024-08-20T01:06:35.562930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.062891+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:05:40.062891+0000","last_peered":"2024-08-20T01:05:40.062891+0000","last_clean":"2024-08-20T01:05:40.062891+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:05:40.062891+0000","last_undegraded":"2024-08-20T01:05:40.062891+0000","last_fullsized":"2024-08-20T01:05:40.062891+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563347+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:06:35.563347+0000","last_peered":"2024-08-20T01:06:35.563347+0000","last_clean":"2024-08-20T01:06:35.563347+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:06:35.563347+0000","last_undegraded":"2024-08-20T01:06:35.563347+0000","last_fullsized":"2024-08-20T01:06:35.563347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.562790+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:06:35.562790+0000","last_peered":"2024-08-20T01:06:35.562790+0000","last_clean":"2024-08-20T01:06:35.562790+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:06:35.562790+0000","last_undegraded":"2024-08-20T01:06:35.562790+0000","last_fullsized":"2024-08-20T01:06:35.562790+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425165+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:06:37.425165+0000","last_peered":"2024-08-20T01:06:37.425165+0000","last_clean":"2024-08-20T01:06:37.425165+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:06:37.425165+0000","last_undegraded":"2024-08-20T01:06:37.425165+0000","last_fullsized":"2024-08-20T01:06:37.425165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567032+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:06:36.567032+0000","last_peered":"2024-08-20T01:06:36.567032+0000","last_clean":"2024-08-20T01:06:36.567032+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:06:36.567032+0000","last_undegraded":"2024-08-20T01:06:36.567032+0000","last_fullsized":"2024-08-20T01:06:36.567032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064523+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:05:40.064523+0000","last_peered":"2024-08-20T01:05:40.064523+0000","last_clean":"2024-08-20T01:05:40.064523+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:05:40.064523+0000","last_undegraded":"2024-08-20T01:05:40.064523+0000","last_fullsized":"2024-08-20T01:05:40.064523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563134+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:06:35.563134+0000","last_peered":"2024-08-20T01:06:35.563134+0000","last_clean":"2024-08-20T01:06:35.563134+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:06:35.563134+0000","last_undegraded":"2024-08-20T01:06:35.563134+0000","last_fullsized":"2024-08-20T01:06:35.563134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564876+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:06:36.564876+0000","last_peered":"2024-08-20T01:06:36.564876+0000","last_clean":"2024-08-20T01:06:36.564876+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:06:36.564876+0000","last_undegraded":"2024-08-20T01:06:36.564876+0000","last_fullsized":"2024-08-20T01:06:36.564876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647031+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:06:36.647031+0000","last_peered":"2024-08-20T01:06:36.647031+0000","last_clean":"2024-08-20T01:06:36.647031+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:06:36.647031+0000","last_undegraded":"2024-08-20T01:06:36.647031+0000","last_fullsized":"2024-08-20T01:06:36.647031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564381+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:06:36.564381+0000","last_peered":"2024-08-20T01:06:36.564381+0000","last_clean":"2024-08-20T01:06:36.564381+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:06:36.564381+0000","last_undegraded":"2024-08-20T01:06:36.564381+0000","last_fullsized":"2024-08-20T01:06:36.564381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642299+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:06:36.642299+0000","last_peered":"2024-08-20T01:06:36.642299+0000","last_clean":"2024-08-20T01:06:36.642299+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:06:36.642299+0000","last_undegraded":"2024-08-20T01:06:36.642299+0000","last_fullsized":"2024-08-20T01:06:36.642299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566850+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:06:36.566850+0000","last_peered":"2024-08-20T01:06:36.566850+0000","last_clean":"2024-08-20T01:06:36.566850+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:06:36.566850+0000","last_undegraded":"2024-08-20T01:06:36.566850+0000","last_fullsized":"2024-08-20T01:06:36.566850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064174+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:05:40.064174+0000","last_peered":"2024-08-20T01:05:40.064174+0000","last_clean":"2024-08-20T01:05:40.064174+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:05:40.064174+0000","last_undegraded":"2024-08-20T01:05:40.064174+0000","last_fullsized":"2024-08-20T01:05:40.064174+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-08-20T01:06:35.563052+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:06:35.563052+0000","last_peered":"2024-08-20T01:06:35.563052+0000","last_clean":"2024-08-20T01:06:35.563052+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:06:35.563052+0000","last_undegraded":"2024-08-20T01:06:35.563052+0000","last_fullsized":"2024-08-20T01:06:35.563052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620898+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:06:36.620898+0000","last_peered":"2024-08-20T01:06:36.620898+0000","last_clean":"2024-08-20T01:06:36.620898+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:06:36.620898+0000","last_undegraded":"2024-08-20T01:06:36.620898+0000","last_fullsized":"2024-08-20T01:06:36.620898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064709+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:05:40.064709+0000","last_peered":"2024-08-20T01:05:40.064709+0000","last_clean":"2024-08-20T01:05:40.064709+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:05:40.064709+0000","last_undegraded":"2024-08-20T01:05:40.064709+0000","last_fullsized":"2024-08-20T01:05:40.064709+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642084+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:06:36.642084+0000","last_peered":"2024-08-20T01:06:36.642084+0000","last_clean":"2024-08-20T01:06:36.642084+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:06:36.642084+0000","last_undegraded":"2024-08-20T01:06:36.642084+0000","last_fullsized":"2024-08-20T01:06:36.642084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425202+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:06:37.425202+0000","last_peered":"2024-08-20T01:06:37.425202+0000","last_clean":"2024-08-20T01:06:37.425202+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:06:37.425202+0000","last_undegraded":"2024-08-20T01:06:37.425202+0000","last_fullsized":"2024-08-20T01:06:37.425202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.424796+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:06:37.424796+0000","last_peered":"2024-08-20T01:06:37.424796+0000","last_clean":"2024-08-20T01:06:37.424796+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:06:37.424796+0000","last_undegraded":"2024-08-20T01:06:37.424796+0000","last_fullsized":"2024-08-20T01:06:37.424796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566819+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:06:36.566819+0000","last_peered":"2024-08-20T01:06:36.566819+0000","last_clean":"2024-08-20T01:06:36.566819+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:06:36.566819+0000","last_undegraded":"2024-08-20T01:06:36.566819+0000","last_fullsized":"2024-08-20T01:06:36.566819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064564+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:05:40.064564+0000","last_peered":"2024-08-20T01:05:40.064564+0000","last_clean":"2024-08-20T01:05:40.064564+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:05:40.064564+0000","last_undegraded":"2024-08-20T01:05:40.064564+0000","last_fullsized":"2024-08-20T01:05:40.064564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900949+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:06:36.900949+0000","last_peered":"2024-08-20T01:06:36.900949+0000","last_clean":"2024-08-20T01:06:36.900949+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:06:36.900949+0000","last_undegraded":"2024-08-20T01:06:36.900949+0000","last_fullsized":"2024-08-20T01:06:36.900949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.645012+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:06:36.645012+0000","last_peered":"2024-08-20T01:06:36.645012+0000","last_clean":"2024-08-20T01:06:36.645012+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:06:36.645012+0000","last_undegraded":"2024-08-20T01:06:36.645012+0000","last_fullsized":"2024-08-20T01:06:36.645012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642836+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:06:36.642836+0000","last_peered":"2024-08-20T01:06:36.642836+0000","last_clean":"2024-08-20T01:06:36.642836+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:06:36.642836+0000","last_undegraded":"2024-08-20T01:06:36.642836+0000","last_fullsized":"2024-08-20T01:06:36.642836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620109+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:06:36.620109+0000","last_peered":"2024-08-20T01:06:36.620109+0000","last_clean":"2024-08-20T01:06:36.620109+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:06:36.620109+0000","last_undegraded":"2024-08-20T01:06:36.620109+0000","last_fullsized":"2024-08-20T01:06:36.620109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900485+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:06:36.900485+0000","last_peered":"2024-08-20T01:06:36.900485+0000","last_clean":"2024-08-20T01:06:36.900485+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:06:36.900485+0000","last_undegraded":"2024-08-20T01:06:36.900485+0000","last_fullsized":"2024-08-20T01:06:36.900485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644298+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:06:36.644298+0000","last_peered":"2024-08-20T01:06:36.644298+0000","last_clean":"2024-08-20T01:06:36.644298+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:06:36.644298+0000","last_undegraded":"2024-08-20T01:06:36.644298+0000","last_fullsized":"2024-08-20T01:06:36.644298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567413+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:06:36.567413+0000","last_peered":"2024-08-20T01:06:36.567413+0000","last_clean":"2024-08-20T01:06:36.567413+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:06:36.567413+0000","last_undegraded":"2024-08-20T01:06:36.567413+0000","last_fullsized":"2024-08-20T01:06:36.567413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620731+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:06:36.620731+0000","last_peered":"2024-08-20T01:06:36.620731+0000","last_clean":"2024-08-20T01:06:36.620731+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:06:36.620731+0000","last_undegraded":"2024-08-20T01:06:36.620731+0000","last_fullsized":"2024-08-20T01:06:36.620731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425241+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:06:37.425241+0000","last_peered":"2024-08-20T01:06:37.425241+0000","last_clean":"2024-08-20T01:06:37.425241+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:06:37.425241+0000","last_undegraded":"2024-08-20T01:06:37.425241+0000","last_fullsized":"2024-08-20T01:06:37.425241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425195+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:06:37.425195+0000","last_peered":"2024-08-20T01:06:37.425195+0000","last_clean":"2024-08-20T01:06:37.425195+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:06:37.425195+0000","last_undegraded":"2024-08-20T01:06:37.425195+0000","last_fullsized":"2024-08-20T01:06:37.425195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-08-20T01:05:40.064704+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:05:40.064704+0000","last_peered":"2024-08-20T01:05:40.064704+0000","last_clean":"2024-08-20T01:05:40.064704+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:05:40.064704+0000","last_undegraded":"2024-08-20T01:05:40.064704+0000","last_fullsized":"2024-08-20T01:05:40.064704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60161,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":137,"num_read_kb":131,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":171953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678043,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":954368,"data_stored":498014,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":1156343,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234027,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":507627,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1113245,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":1108833,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953536,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":497964,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378699,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":546860,"data_compressed":20720,"data_compressed_allocated":28672,"data_compressed_original":57344,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":554800,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":53636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:06:43.276 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:06:43.647 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 -- 172.21.15.138:0/490309640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 msgr2=0x7f81f4112fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 --2- 172.21.15.138:0/490309640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f4112fb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81e0009900 tx=0x7f81e002a400 comp rx=0 tx=0).stop 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 -- 172.21.15.138:0/490309640 shutdown_connections 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 --2- 172.21.15.138:0/490309640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f41134f0 0x7f81f41158e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 --2- 172.21.15.138:0/490309640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f4112fb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 --2- 172.21.15.138:0/490309640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f4077400 0x7f81f40777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 -- 172.21.15.138:0/490309640 >> 172.21.15.138:0/490309640 conn(0x7f81f4100550 msgr2=0x7f81f4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 -- 172.21.15.138:0/490309640 shutdown_connections 2024-08-20T01:06:44.224 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.224+0000 7f81faa00640 1 -- 172.21.15.138:0/490309640 wait complete. 2024-08-20T01:06:44.225 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.225+0000 7f81faa00640 1 Processor -- start 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.225+0000 7f81faa00640 1 -- start start 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.225+0000 7f81faa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f419f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 0x7f81f41a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81faa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4118f50 con 0x7f81f4077d20 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81faa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81f4118dd0 con 0x7f81f41134f0 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81faa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81f41190d0 con 0x7f81f4077400 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:60712/0 (socket says 172.21.15.138:60712) 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 -- 172.21.15.138:0/372483016 learned_addr learned my addr 172.21.15.138:0/372483016 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f8c00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f419f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:44.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81fa000640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 0x7f81f41a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 -- 172.21.15.138:0/372483016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 msgr2=0x7f81f41a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 0x7f81f41a4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 -- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 msgr2=0x7f81f419f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f419f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81f9600640 1 -- 172.21.15.138:0/372483016 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f41a5410 con 0x7f81f4077400 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.226+0000 7f81fa000640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 0x7f81f41a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:44.227 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81f9600640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9f0 tx=0x7f81e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800cd80 con 0x7f81f4077400 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f41a5700 con 0x7f81f4077400 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81e80108e0 con 0x7f81f4077400 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8018980 con 0x7f81f4077400 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.227+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f41abdc0 con 0x7f81f4077400 2024-08-20T01:06:44.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.228+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f40777e0 con 0x7f81f4077400 2024-08-20T01:06:44.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.231+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f81e8010430 con 0x7f81f4077400 2024-08-20T01:06:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.232+0000 7f81f2000640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 0x7f81dc0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.232+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f81e80e4a90 con 0x7f81f4077400 2024-08-20T01:06:44.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.232+0000 7f81f8c00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 0x7f81dc0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:44.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.233+0000 7f81f8c00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 0x7f81dc0b6220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81e0005ab0 tx=0x7f81e0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:44.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.234+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f81e80ab460 con 0x7f81f4077400 2024-08-20T01:06:44.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.413+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f81f4115500 con 0x7f81dc0b3d60 2024-08-20T01:06:44.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.419+0000 7f81f2000640 1 -- 172.21.15.138:0/372483016 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340629 (secure 0 0 0) 0x7f81f4115500 con 0x7f81dc0b3d60 2024-08-20T01:06:44.422 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:44.423 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 msgr2=0x7f81dc0b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 0x7f81dc0b6220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f81e0005ab0 tx=0x7f81e0005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 msgr2=0x7f81f419f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f81e800e9f0 tx=0x7f81e800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 shutdown_connections 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81dc0b3d60 0x7f81dc0b6220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81f41134f0 0x7f81f41a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81f4077d20 0x7f81f419f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 --2- 172.21.15.138:0/372483016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81f4077400 0x7f81f419f360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:44.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 >> 172.21.15.138:0/372483016 conn(0x7f81f4100550 msgr2=0x7f81f4102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:44.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 shutdown_connections 2024-08-20T01:06:44.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:44.424+0000 7f81faa00640 1 -- 172.21.15.138:0/372483016 wait complete. 2024-08-20T01:06:45.192 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-08-20T01:06:42.490520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650763,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":223,"num_read_kb":200,"num_write":288,"num_write_kb":2261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352816,"kb_used_data":11832,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589008,"statfs":{"total":1151940427776,"available":1151579144192,"internally_reserved":0,"allocated":12115968,"data_stored":8116252,"data_compressed":424198,"data_compressed_allocated":573440,"data_compressed_original":2646016,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27070,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":63,"num_read_kb":58,"num_write":17,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.006983"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621083+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621083+0000","last_peered":"2024-08-20T01:06:36.621083+0000","last_clean":"2024-08-20T01:06:36.621083+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621083+0000","last_undegraded":"2024-08-20T01:06:36.621083+0000","last_fullsized":"2024-08-20T01:06:36.621083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564621+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:06:36.564621+0000","last_peered":"2024-08-20T01:06:36.564621+0000","last_clean":"2024-08-20T01:06:36.564621+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:06:36.564621+0000","last_undegraded":"2024-08-20T01:06:36.564621+0000","last_fullsized":"2024-08-20T01:06:36.564621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426580+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.426580+0000","last_peered":"2024-08-20T01:06:37.426580+0000","last_clean":"2024-08-20T01:06:37.426580+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.426580+0000","last_undegraded":"2024-08-20T01:06:37.426580+0000","last_fullsized":"2024-08-20T01:06:37.426580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564871+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564871+0000","last_peered":"2024-08-20T01:06:36.564871+0000","last_clean":"2024-08-20T01:06:36.564871+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564871+0000","last_undegraded":"2024-08-20T01:06:36.564871+0000","last_fullsized":"2024-08-20T01:06:36.564871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425408+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:06:37.425408+0000","last_peered":"2024-08-20T01:06:37.425408+0000","last_clean":"2024-08-20T01:06:37.425408+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:06:37.425408+0000","last_undegraded":"2024-08-20T01:06:37.425408+0000","last_fullsized":"2024-08-20T01:06:37.425408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564424+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564424+0000","last_peered":"2024-08-20T01:06:36.564424+0000","last_clean":"2024-08-20T01:06:36.564424+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564424+0000","last_undegraded":"2024-08-20T01:06:36.564424+0000","last_fullsized":"2024-08-20T01:06:36.564424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642316+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:06:36.642316+0000","last_peered":"2024-08-20T01:06:36.642316+0000","last_clean":"2024-08-20T01:06:36.642316+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:06:36.642316+0000","last_undegraded":"2024-08-20T01:06:36.642316+0000","last_fullsized":"2024-08-20T01:06:36.642316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649109+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649109+0000","last_peered":"2024-08-20T01:06:36.649109+0000","last_clean":"2024-08-20T01:06:36.649109+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649109+0000","last_undegraded":"2024-08-20T01:06:36.649109+0000","last_fullsized":"2024-08-20T01:06:36.649109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647417+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:06:36.647417+0000","last_peered":"2024-08-20T01:06:36.647417+0000","last_clean":"2024-08-20T01:06:36.647417+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:06:36.647417+0000","last_undegraded":"2024-08-20T01:06:36.647417+0000","last_fullsized":"2024-08-20T01:06:36.647417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564564+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564564+0000","last_peered":"2024-08-20T01:06:36.564564+0000","last_clean":"2024-08-20T01:06:36.564564+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564564+0000","last_undegraded":"2024-08-20T01:06:36.564564+0000","last_fullsized":"2024-08-20T01:06:36.564564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900542+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:06:36.900542+0000","last_peered":"2024-08-20T01:06:36.900542+0000","last_clean":"2024-08-20T01:06:36.900542+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:06:36.900542+0000","last_undegraded":"2024-08-20T01:06:36.900542+0000","last_fullsized":"2024-08-20T01:06:36.900542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564813+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564813+0000","last_peered":"2024-08-20T01:06:36.564813+0000","last_clean":"2024-08-20T01:06:36.564813+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564813+0000","last_undegraded":"2024-08-20T01:06:36.564813+0000","last_fullsized":"2024-08-20T01:06:36.564813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564754+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:06:36.564754+0000","last_peered":"2024-08-20T01:06:36.564754+0000","last_clean":"2024-08-20T01:06:36.564754+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:06:36.564754+0000","last_undegraded":"2024-08-20T01:06:36.564754+0000","last_fullsized":"2024-08-20T01:06:36.564754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564610+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564610+0000","last_peered":"2024-08-20T01:06:36.564610+0000","last_clean":"2024-08-20T01:06:36.564610+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564610+0000","last_undegraded":"2024-08-20T01:06:36.564610+0000","last_fullsized":"2024-08-20T01:06:36.564610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338323+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:06:37.338323+0000","last_peered":"2024-08-20T01:06:37.338323+0000","last_clean":"2024-08-20T01:06:37.338323+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:06:37.338323+0000","last_undegraded":"2024-08-20T01:06:37.338323+0000","last_fullsized":"2024-08-20T01:06:37.338323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564536+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564536+0000","last_peered":"2024-08-20T01:06:36.564536+0000","last_clean":"2024-08-20T01:06:36.564536+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564536+0000","last_undegraded":"2024-08-20T01:06:36.564536+0000","last_fullsized":"2024-08-20T01:06:36.564536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567485+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:06:36.567485+0000","last_peered":"2024-08-20T01:06:36.567485+0000","last_clean":"2024-08-20T01:06:36.567485+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:06:36.567485+0000","last_undegraded":"2024-08-20T01:06:36.567485+0000","last_fullsized":"2024-08-20T01:06:36.567485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642212+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642212+0000","last_peered":"2024-08-20T01:06:36.642212+0000","last_clean":"2024-08-20T01:06:36.642212+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642212+0000","last_undegraded":"2024-08-20T01:06:36.642212+0000","last_fullsized":"2024-08-20T01:06:36.642212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564951+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.564951+0000","last_peered":"2024-08-20T01:06:36.564951+0000","last_clean":"2024-08-20T01:06:36.564951+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.564951+0000","last_undegraded":"2024-08-20T01:06:36.564951+0000","last_fullsized":"2024-08-20T01:06:36.564951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425236+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:06:37.425236+0000","last_peered":"2024-08-20T01:06:37.425236+0000","last_clean":"2024-08-20T01:06:37.425236+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:06:37.425236+0000","last_undegraded":"2024-08-20T01:06:37.425236+0000","last_fullsized":"2024-08-20T01:06:37.425236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565469+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565469+0000","last_peered":"2024-08-20T01:06:36.565469+0000","last_clean":"2024-08-20T01:06:36.565469+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565469+0000","last_undegraded":"2024-08-20T01:06:36.565469+0000","last_fullsized":"2024-08-20T01:06:36.565469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566954+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.566954+0000","last_peered":"2024-08-20T01:06:36.566954+0000","last_clean":"2024-08-20T01:06:36.566954+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.566954+0000","last_undegraded":"2024-08-20T01:06:36.566954+0000","last_fullsized":"2024-08-20T01:06:36.566954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564938+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:06:36.564938+0000","last_peered":"2024-08-20T01:06:36.564938+0000","last_clean":"2024-08-20T01:06:36.564938+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:06:36.564938+0000","last_undegraded":"2024-08-20T01:06:36.564938+0000","last_fullsized":"2024-08-20T01:06:36.564938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'10","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.622441+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:06:36.622441+0000","last_peered":"2024-08-20T01:06:36.622441+0000","last_clean":"2024-08-20T01:06:36.622441+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:06:36.622441+0000","last_undegraded":"2024-08-20T01:06:36.622441+0000","last_fullsized":"2024-08-20T01:06:36.622441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564087+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564087+0000","last_peered":"2024-08-20T01:06:36.564087+0000","last_clean":"2024-08-20T01:06:36.564087+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564087+0000","last_undegraded":"2024-08-20T01:06:36.564087+0000","last_fullsized":"2024-08-20T01:06:36.564087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567052+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:06:36.567052+0000","last_peered":"2024-08-20T01:06:36.567052+0000","last_clean":"2024-08-20T01:06:36.567052+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:06:36.567052+0000","last_undegraded":"2024-08-20T01:06:36.567052+0000","last_fullsized":"2024-08-20T01:06:36.567052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566662+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566662+0000","last_peered":"2024-08-20T01:06:36.566662+0000","last_clean":"2024-08-20T01:06:36.566662+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566662+0000","last_undegraded":"2024-08-20T01:06:36.566662+0000","last_fullsized":"2024-08-20T01:06:36.566662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338256+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338256+0000","last_peered":"2024-08-20T01:06:37.338256+0000","last_clean":"2024-08-20T01:06:37.338256+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338256+0000","last_undegraded":"2024-08-20T01:06:37.338256+0000","last_fullsized":"2024-08-20T01:06:37.338256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644635+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:06:36.644635+0000","last_peered":"2024-08-20T01:06:36.644635+0000","last_clean":"2024-08-20T01:06:36.644635+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:06:36.644635+0000","last_undegraded":"2024-08-20T01:06:36.644635+0000","last_fullsized":"2024-08-20T01:06:36.644635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620802+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620802+0000","last_peered":"2024-08-20T01:06:36.620802+0000","last_clean":"2024-08-20T01:06:36.620802+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620802+0000","last_undegraded":"2024-08-20T01:06:36.620802+0000","last_fullsized":"2024-08-20T01:06:36.620802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644713+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:06:36.644713+0000","last_peered":"2024-08-20T01:06:36.644713+0000","last_clean":"2024-08-20T01:06:36.644713+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:06:36.644713+0000","last_undegraded":"2024-08-20T01:06:36.644713+0000","last_fullsized":"2024-08-20T01:06:36.644713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900709+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900709+0000","last_peered":"2024-08-20T01:06:36.900709+0000","last_clean":"2024-08-20T01:06:36.900709+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900709+0000","last_undegraded":"2024-08-20T01:06:36.900709+0000","last_fullsized":"2024-08-20T01:06:36.900709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566848+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566848+0000","last_peered":"2024-08-20T01:06:36.566848+0000","last_clean":"2024-08-20T01:06:36.566848+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566848+0000","last_undegraded":"2024-08-20T01:06:36.566848+0000","last_fullsized":"2024-08-20T01:06:36.566848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647086+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:06:36.647086+0000","last_peered":"2024-08-20T01:06:36.647086+0000","last_clean":"2024-08-20T01:06:36.647086+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:06:36.647086+0000","last_undegraded":"2024-08-20T01:06:36.647086+0000","last_fullsized":"2024-08-20T01:06:36.647086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":65,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.645366+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:06:38.645366+0000","last_peered":"2024-08-20T01:06:38.645366+0000","last_clean":"2024-08-20T01:06:38.645366+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:06:38.645366+0000","last_undegraded":"2024-08-20T01:06:38.645366+0000","last_fullsized":"2024-08-20T01:06:38.645366+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566529+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:06:36.566529+0000","last_peered":"2024-08-20T01:06:36.566529+0000","last_clean":"2024-08-20T01:06:36.566529+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:06:36.566529+0000","last_undegraded":"2024-08-20T01:06:36.566529+0000","last_fullsized":"2024-08-20T01:06:36.566529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900529+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900529+0000","last_peered":"2024-08-20T01:06:36.900529+0000","last_clean":"2024-08-20T01:06:36.900529+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900529+0000","last_undegraded":"2024-08-20T01:06:36.900529+0000","last_fullsized":"2024-08-20T01:06:36.900529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564431+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:06:36.564431+0000","last_peered":"2024-08-20T01:06:36.564431+0000","last_clean":"2024-08-20T01:06:36.564431+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:06:36.564431+0000","last_undegraded":"2024-08-20T01:06:36.564431+0000","last_fullsized":"2024-08-20T01:06:36.564431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565285+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565285+0000","last_peered":"2024-08-20T01:06:36.565285+0000","last_clean":"2024-08-20T01:06:36.565285+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565285+0000","last_undegraded":"2024-08-20T01:06:36.565285+0000","last_fullsized":"2024-08-20T01:06:36.565285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900668+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900668+0000","last_peered":"2024-08-20T01:06:36.900668+0000","last_clean":"2024-08-20T01:06:36.900668+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900668+0000","last_undegraded":"2024-08-20T01:06:36.900668+0000","last_fullsized":"2024-08-20T01:06:36.900668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901014+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:06:36.901014+0000","last_peered":"2024-08-20T01:06:36.901014+0000","last_clean":"2024-08-20T01:06:36.901014+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:06:36.901014+0000","last_undegraded":"2024-08-20T01:06:36.901014+0000","last_fullsized":"2024-08-20T01:06:36.901014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567767+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567767+0000","last_peered":"2024-08-20T01:06:36.567767+0000","last_clean":"2024-08-20T01:06:36.567767+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567767+0000","last_undegraded":"2024-08-20T01:06:36.567767+0000","last_fullsized":"2024-08-20T01:06:36.567767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900715+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:06:36.900715+0000","last_peered":"2024-08-20T01:06:36.900715+0000","last_clean":"2024-08-20T01:06:36.900715+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:06:36.900715+0000","last_undegraded":"2024-08-20T01:06:36.900715+0000","last_fullsized":"2024-08-20T01:06:36.900715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647579+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647579+0000","last_peered":"2024-08-20T01:06:36.647579+0000","last_clean":"2024-08-20T01:06:36.647579+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647579+0000","last_undegraded":"2024-08-20T01:06:36.647579+0000","last_fullsized":"2024-08-20T01:06:36.647579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900966+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:06:36.900966+0000","last_peered":"2024-08-20T01:06:36.900966+0000","last_clean":"2024-08-20T01:06:36.900966+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:06:36.900966+0000","last_undegraded":"2024-08-20T01:06:36.900966+0000","last_fullsized":"2024-08-20T01:06:36.900966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567528+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567528+0000","last_peered":"2024-08-20T01:06:36.567528+0000","last_clean":"2024-08-20T01:06:36.567528+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567528+0000","last_undegraded":"2024-08-20T01:06:36.567528+0000","last_fullsized":"2024-08-20T01:06:36.567528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338655+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:06:37.338655+0000","last_peered":"2024-08-20T01:06:37.338655+0000","last_clean":"2024-08-20T01:06:37.338655+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:06:37.338655+0000","last_undegraded":"2024-08-20T01:06:37.338655+0000","last_fullsized":"2024-08-20T01:06:37.338655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647257+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:06:36.647257+0000","last_peered":"2024-08-20T01:06:36.647257+0000","last_clean":"2024-08-20T01:06:36.647257+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:06:36.647257+0000","last_undegraded":"2024-08-20T01:06:36.647257+0000","last_fullsized":"2024-08-20T01:06:36.647257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338120+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338120+0000","last_peered":"2024-08-20T01:06:37.338120+0000","last_clean":"2024-08-20T01:06:37.338120+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338120+0000","last_undegraded":"2024-08-20T01:06:37.338120+0000","last_fullsized":"2024-08-20T01:06:37.338120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900387+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:06:36.900387+0000","last_peered":"2024-08-20T01:06:36.900387+0000","last_clean":"2024-08-20T01:06:36.900387+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:06:36.900387+0000","last_undegraded":"2024-08-20T01:06:36.900387+0000","last_fullsized":"2024-08-20T01:06:36.900387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":66,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.646054+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:06:38.646054+0000","last_peered":"2024-08-20T01:06:38.646054+0000","last_clean":"2024-08-20T01:06:38.646054+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:06:38.646054+0000","last_undegraded":"2024-08-20T01:06:38.646054+0000","last_fullsized":"2024-08-20T01:06:38.646054+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":23,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900625+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:06:36.900625+0000","last_peered":"2024-08-20T01:06:36.900625+0000","last_clean":"2024-08-20T01:06:36.900625+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:06:36.900625+0000","last_undegraded":"2024-08-20T01:06:36.900625+0000","last_fullsized":"2024-08-20T01:06:36.900625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338281+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338281+0000","last_peered":"2024-08-20T01:06:37.338281+0000","last_clean":"2024-08-20T01:06:37.338281+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338281+0000","last_undegraded":"2024-08-20T01:06:37.338281+0000","last_fullsized":"2024-08-20T01:06:37.338281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567030+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:06:36.567030+0000","last_peered":"2024-08-20T01:06:36.567030+0000","last_clean":"2024-08-20T01:06:36.567030+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:06:36.567030+0000","last_undegraded":"2024-08-20T01:06:36.567030+0000","last_fullsized":"2024-08-20T01:06:36.567030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566684+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566684+0000","last_peered":"2024-08-20T01:06:36.566684+0000","last_clean":"2024-08-20T01:06:36.566684+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566684+0000","last_undegraded":"2024-08-20T01:06:36.566684+0000","last_fullsized":"2024-08-20T01:06:36.566684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566373+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:06:36.566373+0000","last_peered":"2024-08-20T01:06:36.566373+0000","last_clean":"2024-08-20T01:06:36.566373+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:06:36.566373+0000","last_undegraded":"2024-08-20T01:06:36.566373+0000","last_fullsized":"2024-08-20T01:06:36.566373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'30","reported_seq":116,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.650696+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:06:38.650696+0000","last_peered":"2024-08-20T01:06:38.650696+0000","last_clean":"2024-08-20T01:06:38.650696+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:06:38.650696+0000","last_undegraded":"2024-08-20T01:06:38.650696+0000","last_fullsized":"2024-08-20T01:06:38.650696+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56160,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":73,"num_write":31,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642413+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:06:36.642413+0000","last_peered":"2024-08-20T01:06:36.642413+0000","last_clean":"2024-08-20T01:06:36.642413+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:06:36.642413+0000","last_undegraded":"2024-08-20T01:06:36.642413+0000","last_fullsized":"2024-08-20T01:06:36.642413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564971+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.564971+0000","last_peered":"2024-08-20T01:06:36.564971+0000","last_clean":"2024-08-20T01:06:36.564971+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.564971+0000","last_undegraded":"2024-08-20T01:06:36.564971+0000","last_fullsized":"2024-08-20T01:06:36.564971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425415+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:06:37.425415+0000","last_peered":"2024-08-20T01:06:37.425415+0000","last_clean":"2024-08-20T01:06:37.425415+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:06:37.425415+0000","last_undegraded":"2024-08-20T01:06:37.425415+0000","last_fullsized":"2024-08-20T01:06:37.425415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642659+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:06:36.642659+0000","last_peered":"2024-08-20T01:06:36.642659+0000","last_clean":"2024-08-20T01:06:36.642659+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:06:36.642659+0000","last_undegraded":"2024-08-20T01:06:36.642659+0000","last_fullsized":"2024-08-20T01:06:36.642659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":63,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.648612+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:06:37.648612+0000","last_peered":"2024-08-20T01:06:37.648612+0000","last_clean":"2024-08-20T01:06:37.648612+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:06:37.648612+0000","last_undegraded":"2024-08-20T01:06:37.648612+0000","last_fullsized":"2024-08-20T01:06:37.648612+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":19,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425093+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425093+0000","last_peered":"2024-08-20T01:06:37.425093+0000","last_clean":"2024-08-20T01:06:37.425093+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425093+0000","last_undegraded":"2024-08-20T01:06:37.425093+0000","last_fullsized":"2024-08-20T01:06:37.425093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620212+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:06:36.620212+0000","last_peered":"2024-08-20T01:06:36.620212+0000","last_clean":"2024-08-20T01:06:36.620212+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:06:36.620212+0000","last_undegraded":"2024-08-20T01:06:36.620212+0000","last_fullsized":"2024-08-20T01:06:36.620212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338439+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338439+0000","last_peered":"2024-08-20T01:06:37.338439+0000","last_clean":"2024-08-20T01:06:37.338439+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338439+0000","last_undegraded":"2024-08-20T01:06:37.338439+0000","last_fullsized":"2024-08-20T01:06:37.338439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564203+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:06:36.564203+0000","last_peered":"2024-08-20T01:06:36.564203+0000","last_clean":"2024-08-20T01:06:36.564203+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:06:36.564203+0000","last_undegraded":"2024-08-20T01:06:36.564203+0000","last_fullsized":"2024-08-20T01:06:36.564203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647471+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:06:36.647471+0000","last_peered":"2024-08-20T01:06:36.647471+0000","last_clean":"2024-08-20T01:06:36.647471+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:06:36.647471+0000","last_undegraded":"2024-08-20T01:06:36.647471+0000","last_fullsized":"2024-08-20T01:06:36.647471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566490+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566490+0000","last_peered":"2024-08-20T01:06:36.566490+0000","last_clean":"2024-08-20T01:06:36.566490+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566490+0000","last_undegraded":"2024-08-20T01:06:36.566490+0000","last_fullsized":"2024-08-20T01:06:36.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":69,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.649367+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:06:37.649367+0000","last_peered":"2024-08-20T01:06:37.649367+0000","last_clean":"2024-08-20T01:06:37.649367+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:06:37.649367+0000","last_undegraded":"2024-08-20T01:06:37.649367+0000","last_fullsized":"2024-08-20T01:06:37.649367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567358+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567358+0000","last_peered":"2024-08-20T01:06:36.567358+0000","last_clean":"2024-08-20T01:06:36.567358+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567358+0000","last_undegraded":"2024-08-20T01:06:36.567358+0000","last_fullsized":"2024-08-20T01:06:36.567358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338434+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:06:37.338434+0000","last_peered":"2024-08-20T01:06:37.338434+0000","last_clean":"2024-08-20T01:06:37.338434+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:06:37.338434+0000","last_undegraded":"2024-08-20T01:06:37.338434+0000","last_fullsized":"2024-08-20T01:06:37.338434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567133+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:06:36.567133+0000","last_peered":"2024-08-20T01:06:36.567133+0000","last_clean":"2024-08-20T01:06:36.567133+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:06:36.567133+0000","last_undegraded":"2024-08-20T01:06:36.567133+0000","last_fullsized":"2024-08-20T01:06:36.567133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566829+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:06:36.566829+0000","last_peered":"2024-08-20T01:06:36.566829+0000","last_clean":"2024-08-20T01:06:36.566829+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:06:36.566829+0000","last_undegraded":"2024-08-20T01:06:36.566829+0000","last_fullsized":"2024-08-20T01:06:36.566829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425165+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:06:37.425165+0000","last_peered":"2024-08-20T01:06:37.425165+0000","last_clean":"2024-08-20T01:06:37.425165+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:06:37.425165+0000","last_undegraded":"2024-08-20T01:06:37.425165+0000","last_fullsized":"2024-08-20T01:06:37.425165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567032+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:06:36.567032+0000","last_peered":"2024-08-20T01:06:36.567032+0000","last_clean":"2024-08-20T01:06:36.567032+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:06:36.567032+0000","last_undegraded":"2024-08-20T01:06:36.567032+0000","last_fullsized":"2024-08-20T01:06:36.567032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338685+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:06:37.338685+0000","last_peered":"2024-08-20T01:06:37.338685+0000","last_clean":"2024-08-20T01:06:37.338685+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:06:37.338685+0000","last_undegraded":"2024-08-20T01:06:37.338685+0000","last_fullsized":"2024-08-20T01:06:37.338685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567354+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:06:36.567354+0000","last_peered":"2024-08-20T01:06:36.567354+0000","last_clean":"2024-08-20T01:06:36.567354+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:06:36.567354+0000","last_undegraded":"2024-08-20T01:06:36.567354+0000","last_fullsized":"2024-08-20T01:06:36.567354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564876+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:06:36.564876+0000","last_peered":"2024-08-20T01:06:36.564876+0000","last_clean":"2024-08-20T01:06:36.564876+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:06:36.564876+0000","last_undegraded":"2024-08-20T01:06:36.564876+0000","last_fullsized":"2024-08-20T01:06:36.564876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647031+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:06:36.647031+0000","last_peered":"2024-08-20T01:06:36.647031+0000","last_clean":"2024-08-20T01:06:36.647031+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:06:36.647031+0000","last_undegraded":"2024-08-20T01:06:36.647031+0000","last_fullsized":"2024-08-20T01:06:36.647031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564381+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:06:36.564381+0000","last_peered":"2024-08-20T01:06:36.564381+0000","last_clean":"2024-08-20T01:06:36.564381+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:06:36.564381+0000","last_undegraded":"2024-08-20T01:06:36.564381+0000","last_fullsized":"2024-08-20T01:06:36.564381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642299+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:06:36.642299+0000","last_peered":"2024-08-20T01:06:36.642299+0000","last_clean":"2024-08-20T01:06:36.642299+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:06:36.642299+0000","last_undegraded":"2024-08-20T01:06:36.642299+0000","last_fullsized":"2024-08-20T01:06:36.642299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566850+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:06:36.566850+0000","last_peered":"2024-08-20T01:06:36.566850+0000","last_clean":"2024-08-20T01:06:36.566850+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:06:36.566850+0000","last_undegraded":"2024-08-20T01:06:36.566850+0000","last_fullsized":"2024-08-20T01:06:36.566850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338353+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:06:37.338353+0000","last_peered":"2024-08-20T01:06:37.338353+0000","last_clean":"2024-08-20T01:06:37.338353+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:06:37.338353+0000","last_undegraded":"2024-08-20T01:06:37.338353+0000","last_fullsized":"2024-08-20T01:06:37.338353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567521+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:06:36.567521+0000","last_peered":"2024-08-20T01:06:36.567521+0000","last_clean":"2024-08-20T01:06:36.567521+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:06:36.567521+0000","last_undegraded":"2024-08-20T01:06:36.567521+0000","last_fullsized":"2024-08-20T01:06:36.567521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620898+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:06:36.620898+0000","last_peered":"2024-08-20T01:06:36.620898+0000","last_clean":"2024-08-20T01:06:36.620898+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:06:36.620898+0000","last_undegraded":"2024-08-20T01:06:36.620898+0000","last_fullsized":"2024-08-20T01:06:36.620898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338717+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:06:37.338717+0000","last_peered":"2024-08-20T01:06:37.338717+0000","last_clean":"2024-08-20T01:06:37.338717+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:06:37.338717+0000","last_undegraded":"2024-08-20T01:06:37.338717+0000","last_fullsized":"2024-08-20T01:06:37.338717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642084+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:06:36.642084+0000","last_peered":"2024-08-20T01:06:36.642084+0000","last_clean":"2024-08-20T01:06:36.642084+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:06:36.642084+0000","last_undegraded":"2024-08-20T01:06:36.642084+0000","last_fullsized":"2024-08-20T01:06:36.642084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425202+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:06:37.425202+0000","last_peered":"2024-08-20T01:06:37.425202+0000","last_clean":"2024-08-20T01:06:37.425202+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:06:37.425202+0000","last_undegraded":"2024-08-20T01:06:37.425202+0000","last_fullsized":"2024-08-20T01:06:37.425202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.424796+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:06:37.424796+0000","last_peered":"2024-08-20T01:06:37.424796+0000","last_clean":"2024-08-20T01:06:37.424796+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:06:37.424796+0000","last_undegraded":"2024-08-20T01:06:37.424796+0000","last_fullsized":"2024-08-20T01:06:37.424796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566819+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:06:36.566819+0000","last_peered":"2024-08-20T01:06:36.566819+0000","last_clean":"2024-08-20T01:06:36.566819+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:06:36.566819+0000","last_undegraded":"2024-08-20T01:06:36.566819+0000","last_fullsized":"2024-08-20T01:06:36.566819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565000+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:06:36.565000+0000","last_peered":"2024-08-20T01:06:36.565000+0000","last_clean":"2024-08-20T01:06:36.565000+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:06:36.565000+0000","last_undegraded":"2024-08-20T01:06:36.565000+0000","last_fullsized":"2024-08-20T01:06:36.565000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900949+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:06:36.900949+0000","last_peered":"2024-08-20T01:06:36.900949+0000","last_clean":"2024-08-20T01:06:36.900949+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:06:36.900949+0000","last_undegraded":"2024-08-20T01:06:36.900949+0000","last_fullsized":"2024-08-20T01:06:36.900949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.645012+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:06:36.645012+0000","last_peered":"2024-08-20T01:06:36.645012+0000","last_clean":"2024-08-20T01:06:36.645012+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:06:36.645012+0000","last_undegraded":"2024-08-20T01:06:36.645012+0000","last_fullsized":"2024-08-20T01:06:36.645012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642836+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:06:36.642836+0000","last_peered":"2024-08-20T01:06:36.642836+0000","last_clean":"2024-08-20T01:06:36.642836+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:06:36.642836+0000","last_undegraded":"2024-08-20T01:06:36.642836+0000","last_fullsized":"2024-08-20T01:06:36.642836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620109+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:06:36.620109+0000","last_peered":"2024-08-20T01:06:36.620109+0000","last_clean":"2024-08-20T01:06:36.620109+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:06:36.620109+0000","last_undegraded":"2024-08-20T01:06:36.620109+0000","last_fullsized":"2024-08-20T01:06:36.620109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900485+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:06:36.900485+0000","last_peered":"2024-08-20T01:06:36.900485+0000","last_clean":"2024-08-20T01:06:36.900485+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:06:36.900485+0000","last_undegraded":"2024-08-20T01:06:36.900485+0000","last_fullsized":"2024-08-20T01:06:36.900485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644298+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:06:36.644298+0000","last_peered":"2024-08-20T01:06:36.644298+0000","last_clean":"2024-08-20T01:06:36.644298+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:06:36.644298+0000","last_undegraded":"2024-08-20T01:06:36.644298+0000","last_fullsized":"2024-08-20T01:06:36.644298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567413+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:06:36.567413+0000","last_peered":"2024-08-20T01:06:36.567413+0000","last_clean":"2024-08-20T01:06:36.567413+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:06:36.567413+0000","last_undegraded":"2024-08-20T01:06:36.567413+0000","last_fullsized":"2024-08-20T01:06:36.567413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620731+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:06:36.620731+0000","last_peered":"2024-08-20T01:06:36.620731+0000","last_clean":"2024-08-20T01:06:36.620731+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:06:36.620731+0000","last_undegraded":"2024-08-20T01:06:36.620731+0000","last_fullsized":"2024-08-20T01:06:36.620731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425241+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:06:37.425241+0000","last_peered":"2024-08-20T01:06:37.425241+0000","last_clean":"2024-08-20T01:06:37.425241+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:06:37.425241+0000","last_undegraded":"2024-08-20T01:06:37.425241+0000","last_fullsized":"2024-08-20T01:06:37.425241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425195+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:06:37.425195+0000","last_peered":"2024-08-20T01:06:37.425195+0000","last_clean":"2024-08-20T01:06:37.425195+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:06:37.425195+0000","last_undegraded":"2024-08-20T01:06:37.425195+0000","last_fullsized":"2024-08-20T01:06:37.425195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338711+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:06:37.338711+0000","last_peered":"2024-08-20T01:06:37.338711+0000","last_clean":"2024-08-20T01:06:37.338711+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:06:37.338711+0000","last_undegraded":"2024-08-20T01:06:37.338711+0000","last_fullsized":"2024-08-20T01:06:37.338711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60161,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":142,"num_read_kb":131,"num_write":163,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":176218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874273,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1105920,"data_stored":1156343,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037798,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234028,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1113245,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593783,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790012,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":1108833,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149765,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378700,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1052672,"data_stored":570403,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542223,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1011712,"data_stored":554800,"data_compressed":23677,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705749,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":51895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:06:45.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph status --format=json 2024-08-20T01:06:45.566 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:45.849 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:45 smithi138 ceph-mon[18076]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:45.850 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:45 smithi138 ceph-mon[18076]: pgmap v321: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 688 B/s wr, 4 op/s 2024-08-20T01:06:45.933 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:45 smithi167 ceph-mon[21965]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:45.934 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:45 smithi167 ceph-mon[21965]: pgmap v321: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 688 B/s wr, 4 op/s 2024-08-20T01:06:45.966 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:45 smithi142 ceph-mon[21799]: from='client.24952 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:45.967 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:45 smithi142 ceph-mon[21799]: pgmap v321: 129 pgs: 129 active+clean; 636 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 688 B/s wr, 4 op/s 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 -- 172.21.15.138:0/2052507078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a44104ca0 msgr2=0x7f1a44105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 --2- 172.21.15.138:0/2052507078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a44104ca0 0x7f1a44105080 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009900 tx=0x7f1a3402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 -- 172.21.15.138:0/2052507078 shutdown_connections 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 --2- 172.21.15.138:0/2052507078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a44111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 --2- 172.21.15.138:0/2052507078 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a4410a6b0 0x7f1a4410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 --2- 172.21.15.138:0/2052507078 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a44104ca0 0x7f1a44105080 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 -- 172.21.15.138:0/2052507078 >> 172.21.15.138:0/2052507078 conn(0x7f1a44100730 msgr2=0x7f1a44102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 -- 172.21.15.138:0/2052507078 shutdown_connections 2024-08-20T01:06:46.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.149+0000 7f1a49c00640 1 -- 172.21.15.138:0/2052507078 wait complete. 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.150+0000 7f1a49c00640 1 Processor -- start 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 -- start start 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a44104ca0 0x7f1a4419b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4410a6b0 0x7f1a4419b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44114cf0 con 0x7f1a4410a6b0 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44114b70 con 0x7f1a4410b030 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a49c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a44114e70 con 0x7f1a44104ca0 2024-08-20T01:06:46.151 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a42a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a44104ca0 0x7f1a4419b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a43400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38846/0 (socket says 172.21.15.138:38846) 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a43400640 1 -- 172.21.15.138:0/1434771731 learned_addr learned my addr 172.21.15.138:0/1434771731 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.151+0000 7f1a3be00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4410a6b0 0x7f1a4419b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 -- 172.21.15.138:0/1434771731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a44104ca0 msgr2=0x7f1a4419b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a44104ca0 0x7f1a4419b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 -- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4410a6b0 msgr2=0x7f1a4419b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4410a6b0 0x7f1a4419b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 -- 172.21.15.138:0/1434771731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441a12d0 con 0x7f1a4410b030 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a43400640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00eb10 tx=0x7f1a3c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:46.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c004b20 con 0x7f1a4410b030 2024-08-20T01:06:46.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a3c004cc0 con 0x7f1a4410b030 2024-08-20T01:06:46.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a441a15c0 con 0x7f1a4410b030 2024-08-20T01:06:46.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c010670 con 0x7f1a4410b030 2024-08-20T01:06:46.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.152+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a7c80 con 0x7f1a4410b030 2024-08-20T01:06:46.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.153+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a44105080 con 0x7f1a4410b030 2024-08-20T01:06:46.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.155+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f1a3c005020 con 0x7f1a4410b030 2024-08-20T01:06:46.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.157+0000 7f1a3b400640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 0x7f1a280b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:46.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.157+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1a3c0e48f0 con 0x7f1a4410b030 2024-08-20T01:06:46.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.157+0000 7f1a42a00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 0x7f1a280b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:46.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.158+0000 7f1a42a00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 0x7f1a280b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a34031440 tx=0x7f1a34005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:46.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.158+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f1a3c0ab910 con 0x7f1a4410b030 2024-08-20T01:06:46.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.417+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1a4419c350 con 0x7f1a4410b030 2024-08-20T01:06:46.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.417+0000 7f1a3b400640 1 -- 172.21.15.138:0/1434771731 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2231 (secure 0 0 0) 0x7f1a4419c350 con 0x7f1a4410b030 2024-08-20T01:06:46.417 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:46.417 INFO:teuthology.orchestra.run.smithi138.stdout:{"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":503,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":88,"num_osds":12,"num_up_osds":12,"osd_up_since":1724115869,"num_in_osds":12,"osd_in_since":1724115846,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":129}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":650763,"bytes_used":361295872,"bytes_avail":1151579131904,"bytes_total":1151940427776,"read_bytes_sec":2752,"write_bytes_sec":688,"read_op_per_sec":3,"write_op_per_sec":1},"fsmap":{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"e","status":"up:standby-replay","gid":24674},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24679},{"filesystem_id":1,"rank":0,"name":"i","status":"up:standby-replay","gid":24694},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24704},{"filesystem_id":1,"rank":1,"name":"g","status":"up:active","gid":24706},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24743}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"services":{"dashboard":"https://172.21.15.138:8443/"}},"servicemap":{"epoch":28,"modified":"2024-08-20T01:06:36.484292+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-08-20T01:06:46.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 msgr2=0x7f1a280b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:46.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 0x7f1a280b6170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1a34031440 tx=0x7f1a34005b50 comp rx=0 tx=0).stop 2024-08-20T01:06:46.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 msgr2=0x7f1a441a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:46.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c00eb10 tx=0x7f1a3c00c2d0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 shutdown_connections 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a280b3cb0 0x7f1a280b6170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a4410b030 0x7f1a441a0b00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a4410a6b0 0x7f1a4419b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 --2- 172.21.15.138:0/1434771731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a44104ca0 0x7f1a4419b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 >> 172.21.15.138:0/1434771731 conn(0x7f1a44100730 msgr2=0x7f1a44102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 shutdown_connections 2024-08-20T01:06:46.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:46.420+0000 7f1a49c00640 1 -- 172.21.15.138:0/1434771731 wait complete. 2024-08-20T01:06:46.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:46 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1434771731' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T01:06:46.861 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:46 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1434771731' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T01:06:46.884 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:46 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1434771731' entity='client.admin' cmd={"prefix": "status", "format": "json"} : dispatch 2024-08-20T01:06:47.337 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-08-20T01:06:47.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:06:47.704 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:47 smithi138 ceph-mon[18076]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T01:06:47.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:47 smithi167 ceph-mon[21965]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T01:06:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:47 smithi142 ceph-mon[21799]: pgmap v322: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.6 KiB/s wr, 7 op/s 2024-08-20T01:06:48.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.317+0000 7f6865600640 1 -- 172.21.15.138:0/2980698261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860102ab0 msgr2=0x7f6860102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:48.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.317+0000 7f6865600640 1 --2- 172.21.15.138:0/2980698261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860102ab0 0x7f6860102e90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6850009900 tx=0x7f685002a410 comp rx=0 tx=0).stop 2024-08-20T01:06:48.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 -- 172.21.15.138:0/2980698261 shutdown_connections 2024-08-20T01:06:48.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 --2- 172.21.15.138:0/2980698261 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860108e40 0x7f686010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 --2- 172.21.15.138:0/2980698261 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f6860108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 --2- 172.21.15.138:0/2980698261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860102ab0 0x7f6860102e90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 -- 172.21.15.138:0/2980698261 >> 172.21.15.138:0/2980698261 conn(0x7f68600fe500 msgr2=0x7f6860100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 -- 172.21.15.138:0/2980698261 shutdown_connections 2024-08-20T01:06:48.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.318+0000 7f6865600640 1 -- 172.21.15.138:0/2980698261 wait complete. 2024-08-20T01:06:48.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 Processor -- start 2024-08-20T01:06:48.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 -- start start 2024-08-20T01:06:48.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 0x7f686019b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f686019b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112b10 con 0x7f6860108e40 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112990 con 0x7f68601084c0 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f6865600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6860112c90 con 0x7f6860102ab0 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.319+0000 7f685f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58992/0 (socket says 172.21.15.138:58992) 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685f400640 1 -- 172.21.15.138:0/1197053984 learned_addr learned my addr 172.21.15.138:0/1197053984 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f686019b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685ea00640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 0x7f686019b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:48.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 -- 172.21.15.138:0/1197053984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 msgr2=0x7f686019b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 0x7f686019b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 -- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 msgr2=0x7f68601a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 -- 172.21.15.138:0/1197053984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68601a1290 con 0x7f68601084c0 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685ea00640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 0x7f686019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685f400640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.320+0000 7f685e000640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f686019b720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:48.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.321+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c00ccc0 con 0x7f68601084c0 2024-08-20T01:06:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.321+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68601a1580 con 0x7f68601084c0 2024-08-20T01:06:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.321+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684c0108e0 con 0x7f68601084c0 2024-08-20T01:06:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.321+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684c018880 con 0x7f68601084c0 2024-08-20T01:06:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.321+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68601a7e10 con 0x7f68601084c0 2024-08-20T01:06:48.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.322+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6860103d60 con 0x7f68601084c0 2024-08-20T01:06:48.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.325+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f684c010430 con 0x7f68601084c0 2024-08-20T01:06:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.326+0000 7f6857400640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 0x7f68440b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.327+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f684c0e4a90 con 0x7f68601084c0 2024-08-20T01:06:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.327+0000 7f685ea00640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 0x7f68440b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.327+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f684c0abab0 con 0x7f68601084c0 2024-08-20T01:06:48.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.327+0000 7f685ea00640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 0x7f68440b61f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f6850005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:48.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.507+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68600757f0 con 0x7f68440b3d30 2024-08-20T01:06:48.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.516+0000 7f6857400640 1 -- 172.21.15.138:0/1197053984 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340596 (secure 0 0 0) 0x7f68600757f0 con 0x7f68440b3d30 2024-08-20T01:06:48.518 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:48.520 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:06:48.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 msgr2=0x7f68440b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:48.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 0x7f68440b61f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f6850005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:48.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 msgr2=0x7f686019b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:48.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f686019b720 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f684c00e9f0 tx=0x7f684c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 shutdown_connections 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f68440b3d30 0x7f68440b61f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6860108e40 0x7f68601a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f68601084c0 0x7f686019b720 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 --2- 172.21.15.138:0/1197053984 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6860102ab0 0x7f686019b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 >> 172.21.15.138:0/1197053984 conn(0x7f68600fe500 msgr2=0x7f68600750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 shutdown_connections 2024-08-20T01:06:48.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:48.522+0000 7f6865600640 1 -- 172.21.15.138:0/1197053984 wait complete. 2024-08-20T01:06:49.497 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-08-20T01:06:48.492441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657569,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":254,"num_read_kb":220,"num_write":294,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352840,"kb_used_data":11868,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588984,"statfs":{"total":1151940427776,"available":1151579119616,"internally_reserved":0,"allocated":12152832,"data_stored":8147991,"data_compressed":427158,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11071,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":62,"num_read_kb":44,"num_write":14,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.929338"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621083+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621083+0000","last_peered":"2024-08-20T01:06:36.621083+0000","last_clean":"2024-08-20T01:06:36.621083+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621083+0000","last_undegraded":"2024-08-20T01:06:36.621083+0000","last_fullsized":"2024-08-20T01:06:36.621083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564621+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:06:36.564621+0000","last_peered":"2024-08-20T01:06:36.564621+0000","last_clean":"2024-08-20T01:06:36.564621+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:06:36.564621+0000","last_undegraded":"2024-08-20T01:06:36.564621+0000","last_fullsized":"2024-08-20T01:06:36.564621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426580+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.426580+0000","last_peered":"2024-08-20T01:06:37.426580+0000","last_clean":"2024-08-20T01:06:37.426580+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.426580+0000","last_undegraded":"2024-08-20T01:06:37.426580+0000","last_fullsized":"2024-08-20T01:06:37.426580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564871+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564871+0000","last_peered":"2024-08-20T01:06:36.564871+0000","last_clean":"2024-08-20T01:06:36.564871+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564871+0000","last_undegraded":"2024-08-20T01:06:36.564871+0000","last_fullsized":"2024-08-20T01:06:36.564871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425408+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:06:37.425408+0000","last_peered":"2024-08-20T01:06:37.425408+0000","last_clean":"2024-08-20T01:06:37.425408+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:06:37.425408+0000","last_undegraded":"2024-08-20T01:06:37.425408+0000","last_fullsized":"2024-08-20T01:06:37.425408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564424+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564424+0000","last_peered":"2024-08-20T01:06:36.564424+0000","last_clean":"2024-08-20T01:06:36.564424+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564424+0000","last_undegraded":"2024-08-20T01:06:36.564424+0000","last_fullsized":"2024-08-20T01:06:36.564424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642316+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:06:36.642316+0000","last_peered":"2024-08-20T01:06:36.642316+0000","last_clean":"2024-08-20T01:06:36.642316+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:06:36.642316+0000","last_undegraded":"2024-08-20T01:06:36.642316+0000","last_fullsized":"2024-08-20T01:06:36.642316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649109+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649109+0000","last_peered":"2024-08-20T01:06:36.649109+0000","last_clean":"2024-08-20T01:06:36.649109+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649109+0000","last_undegraded":"2024-08-20T01:06:36.649109+0000","last_fullsized":"2024-08-20T01:06:36.649109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647417+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:06:36.647417+0000","last_peered":"2024-08-20T01:06:36.647417+0000","last_clean":"2024-08-20T01:06:36.647417+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:06:36.647417+0000","last_undegraded":"2024-08-20T01:06:36.647417+0000","last_fullsized":"2024-08-20T01:06:36.647417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564564+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564564+0000","last_peered":"2024-08-20T01:06:36.564564+0000","last_clean":"2024-08-20T01:06:36.564564+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564564+0000","last_undegraded":"2024-08-20T01:06:36.564564+0000","last_fullsized":"2024-08-20T01:06:36.564564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900542+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:06:36.900542+0000","last_peered":"2024-08-20T01:06:36.900542+0000","last_clean":"2024-08-20T01:06:36.900542+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:06:36.900542+0000","last_undegraded":"2024-08-20T01:06:36.900542+0000","last_fullsized":"2024-08-20T01:06:36.900542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564813+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564813+0000","last_peered":"2024-08-20T01:06:36.564813+0000","last_clean":"2024-08-20T01:06:36.564813+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564813+0000","last_undegraded":"2024-08-20T01:06:36.564813+0000","last_fullsized":"2024-08-20T01:06:36.564813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564754+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:06:36.564754+0000","last_peered":"2024-08-20T01:06:36.564754+0000","last_clean":"2024-08-20T01:06:36.564754+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:06:36.564754+0000","last_undegraded":"2024-08-20T01:06:36.564754+0000","last_fullsized":"2024-08-20T01:06:36.564754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564610+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564610+0000","last_peered":"2024-08-20T01:06:36.564610+0000","last_clean":"2024-08-20T01:06:36.564610+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564610+0000","last_undegraded":"2024-08-20T01:06:36.564610+0000","last_fullsized":"2024-08-20T01:06:36.564610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338323+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:06:37.338323+0000","last_peered":"2024-08-20T01:06:37.338323+0000","last_clean":"2024-08-20T01:06:37.338323+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:06:37.338323+0000","last_undegraded":"2024-08-20T01:06:37.338323+0000","last_fullsized":"2024-08-20T01:06:37.338323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564536+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564536+0000","last_peered":"2024-08-20T01:06:36.564536+0000","last_clean":"2024-08-20T01:06:36.564536+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564536+0000","last_undegraded":"2024-08-20T01:06:36.564536+0000","last_fullsized":"2024-08-20T01:06:36.564536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567485+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:06:36.567485+0000","last_peered":"2024-08-20T01:06:36.567485+0000","last_clean":"2024-08-20T01:06:36.567485+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:06:36.567485+0000","last_undegraded":"2024-08-20T01:06:36.567485+0000","last_fullsized":"2024-08-20T01:06:36.567485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642212+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642212+0000","last_peered":"2024-08-20T01:06:36.642212+0000","last_clean":"2024-08-20T01:06:36.642212+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642212+0000","last_undegraded":"2024-08-20T01:06:36.642212+0000","last_fullsized":"2024-08-20T01:06:36.642212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564951+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.564951+0000","last_peered":"2024-08-20T01:06:36.564951+0000","last_clean":"2024-08-20T01:06:36.564951+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.564951+0000","last_undegraded":"2024-08-20T01:06:36.564951+0000","last_fullsized":"2024-08-20T01:06:36.564951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425236+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:06:37.425236+0000","last_peered":"2024-08-20T01:06:37.425236+0000","last_clean":"2024-08-20T01:06:37.425236+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:06:37.425236+0000","last_undegraded":"2024-08-20T01:06:37.425236+0000","last_fullsized":"2024-08-20T01:06:37.425236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565469+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565469+0000","last_peered":"2024-08-20T01:06:36.565469+0000","last_clean":"2024-08-20T01:06:36.565469+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565469+0000","last_undegraded":"2024-08-20T01:06:36.565469+0000","last_fullsized":"2024-08-20T01:06:36.565469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566954+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.566954+0000","last_peered":"2024-08-20T01:06:36.566954+0000","last_clean":"2024-08-20T01:06:36.566954+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.566954+0000","last_undegraded":"2024-08-20T01:06:36.566954+0000","last_fullsized":"2024-08-20T01:06:36.566954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564938+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:06:36.564938+0000","last_peered":"2024-08-20T01:06:36.564938+0000","last_clean":"2024-08-20T01:06:36.564938+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:06:36.564938+0000","last_undegraded":"2024-08-20T01:06:36.564938+0000","last_fullsized":"2024-08-20T01:06:36.564938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.619700+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:06:38.619700+0000","last_peered":"2024-08-20T01:06:38.619700+0000","last_clean":"2024-08-20T01:06:38.619700+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:06:38.619700+0000","last_undegraded":"2024-08-20T01:06:38.619700+0000","last_fullsized":"2024-08-20T01:06:38.619700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564087+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564087+0000","last_peered":"2024-08-20T01:06:36.564087+0000","last_clean":"2024-08-20T01:06:36.564087+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564087+0000","last_undegraded":"2024-08-20T01:06:36.564087+0000","last_fullsized":"2024-08-20T01:06:36.564087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567052+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:06:36.567052+0000","last_peered":"2024-08-20T01:06:36.567052+0000","last_clean":"2024-08-20T01:06:36.567052+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:06:36.567052+0000","last_undegraded":"2024-08-20T01:06:36.567052+0000","last_fullsized":"2024-08-20T01:06:36.567052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566662+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566662+0000","last_peered":"2024-08-20T01:06:36.566662+0000","last_clean":"2024-08-20T01:06:36.566662+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566662+0000","last_undegraded":"2024-08-20T01:06:36.566662+0000","last_fullsized":"2024-08-20T01:06:36.566662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338256+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338256+0000","last_peered":"2024-08-20T01:06:37.338256+0000","last_clean":"2024-08-20T01:06:37.338256+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338256+0000","last_undegraded":"2024-08-20T01:06:37.338256+0000","last_fullsized":"2024-08-20T01:06:37.338256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644635+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:06:36.644635+0000","last_peered":"2024-08-20T01:06:36.644635+0000","last_clean":"2024-08-20T01:06:36.644635+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:06:36.644635+0000","last_undegraded":"2024-08-20T01:06:36.644635+0000","last_fullsized":"2024-08-20T01:06:36.644635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620802+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620802+0000","last_peered":"2024-08-20T01:06:36.620802+0000","last_clean":"2024-08-20T01:06:36.620802+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620802+0000","last_undegraded":"2024-08-20T01:06:36.620802+0000","last_fullsized":"2024-08-20T01:06:36.620802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644713+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:06:36.644713+0000","last_peered":"2024-08-20T01:06:36.644713+0000","last_clean":"2024-08-20T01:06:36.644713+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:06:36.644713+0000","last_undegraded":"2024-08-20T01:06:36.644713+0000","last_fullsized":"2024-08-20T01:06:36.644713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900709+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900709+0000","last_peered":"2024-08-20T01:06:36.900709+0000","last_clean":"2024-08-20T01:06:36.900709+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900709+0000","last_undegraded":"2024-08-20T01:06:36.900709+0000","last_fullsized":"2024-08-20T01:06:36.900709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566848+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566848+0000","last_peered":"2024-08-20T01:06:36.566848+0000","last_clean":"2024-08-20T01:06:36.566848+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566848+0000","last_undegraded":"2024-08-20T01:06:36.566848+0000","last_fullsized":"2024-08-20T01:06:36.566848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647086+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:06:36.647086+0000","last_peered":"2024-08-20T01:06:36.647086+0000","last_clean":"2024-08-20T01:06:36.647086+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:06:36.647086+0000","last_undegraded":"2024-08-20T01:06:36.647086+0000","last_fullsized":"2024-08-20T01:06:36.647086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:43.653134+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:06:43.653134+0000","last_peered":"2024-08-20T01:06:43.653134+0000","last_clean":"2024-08-20T01:06:43.653134+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:06:43.653134+0000","last_undegraded":"2024-08-20T01:06:43.653134+0000","last_fullsized":"2024-08-20T01:06:43.653134+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566529+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:06:36.566529+0000","last_peered":"2024-08-20T01:06:36.566529+0000","last_clean":"2024-08-20T01:06:36.566529+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:06:36.566529+0000","last_undegraded":"2024-08-20T01:06:36.566529+0000","last_fullsized":"2024-08-20T01:06:36.566529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900529+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900529+0000","last_peered":"2024-08-20T01:06:36.900529+0000","last_clean":"2024-08-20T01:06:36.900529+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900529+0000","last_undegraded":"2024-08-20T01:06:36.900529+0000","last_fullsized":"2024-08-20T01:06:36.900529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564431+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:06:36.564431+0000","last_peered":"2024-08-20T01:06:36.564431+0000","last_clean":"2024-08-20T01:06:36.564431+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:06:36.564431+0000","last_undegraded":"2024-08-20T01:06:36.564431+0000","last_fullsized":"2024-08-20T01:06:36.564431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565285+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565285+0000","last_peered":"2024-08-20T01:06:36.565285+0000","last_clean":"2024-08-20T01:06:36.565285+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565285+0000","last_undegraded":"2024-08-20T01:06:36.565285+0000","last_fullsized":"2024-08-20T01:06:36.565285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":117,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900668+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900668+0000","last_peered":"2024-08-20T01:06:36.900668+0000","last_clean":"2024-08-20T01:06:36.900668+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900668+0000","last_undegraded":"2024-08-20T01:06:36.900668+0000","last_fullsized":"2024-08-20T01:06:36.900668+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901014+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:06:36.901014+0000","last_peered":"2024-08-20T01:06:36.901014+0000","last_clean":"2024-08-20T01:06:36.901014+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:06:36.901014+0000","last_undegraded":"2024-08-20T01:06:36.901014+0000","last_fullsized":"2024-08-20T01:06:36.901014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567767+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567767+0000","last_peered":"2024-08-20T01:06:36.567767+0000","last_clean":"2024-08-20T01:06:36.567767+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567767+0000","last_undegraded":"2024-08-20T01:06:36.567767+0000","last_fullsized":"2024-08-20T01:06:36.567767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900715+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:06:36.900715+0000","last_peered":"2024-08-20T01:06:36.900715+0000","last_clean":"2024-08-20T01:06:36.900715+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:06:36.900715+0000","last_undegraded":"2024-08-20T01:06:36.900715+0000","last_fullsized":"2024-08-20T01:06:36.900715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647579+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647579+0000","last_peered":"2024-08-20T01:06:36.647579+0000","last_clean":"2024-08-20T01:06:36.647579+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647579+0000","last_undegraded":"2024-08-20T01:06:36.647579+0000","last_fullsized":"2024-08-20T01:06:36.647579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900966+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:06:36.900966+0000","last_peered":"2024-08-20T01:06:36.900966+0000","last_clean":"2024-08-20T01:06:36.900966+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:06:36.900966+0000","last_undegraded":"2024-08-20T01:06:36.900966+0000","last_fullsized":"2024-08-20T01:06:36.900966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567528+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567528+0000","last_peered":"2024-08-20T01:06:36.567528+0000","last_clean":"2024-08-20T01:06:36.567528+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567528+0000","last_undegraded":"2024-08-20T01:06:36.567528+0000","last_fullsized":"2024-08-20T01:06:36.567528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338655+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:06:37.338655+0000","last_peered":"2024-08-20T01:06:37.338655+0000","last_clean":"2024-08-20T01:06:37.338655+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:06:37.338655+0000","last_undegraded":"2024-08-20T01:06:37.338655+0000","last_fullsized":"2024-08-20T01:06:37.338655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647257+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:06:36.647257+0000","last_peered":"2024-08-20T01:06:36.647257+0000","last_clean":"2024-08-20T01:06:36.647257+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:06:36.647257+0000","last_undegraded":"2024-08-20T01:06:36.647257+0000","last_fullsized":"2024-08-20T01:06:36.647257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338120+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338120+0000","last_peered":"2024-08-20T01:06:37.338120+0000","last_clean":"2024-08-20T01:06:37.338120+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338120+0000","last_undegraded":"2024-08-20T01:06:37.338120+0000","last_fullsized":"2024-08-20T01:06:37.338120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900387+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:06:36.900387+0000","last_peered":"2024-08-20T01:06:36.900387+0000","last_clean":"2024-08-20T01:06:36.900387+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:06:36.900387+0000","last_undegraded":"2024-08-20T01:06:36.900387+0000","last_fullsized":"2024-08-20T01:06:36.900387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:43.653709+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:06:43.653709+0000","last_peered":"2024-08-20T01:06:43.653709+0000","last_clean":"2024-08-20T01:06:43.653709+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:06:43.653709+0000","last_undegraded":"2024-08-20T01:06:43.653709+0000","last_fullsized":"2024-08-20T01:06:43.653709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900625+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:06:36.900625+0000","last_peered":"2024-08-20T01:06:36.900625+0000","last_clean":"2024-08-20T01:06:36.900625+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:06:36.900625+0000","last_undegraded":"2024-08-20T01:06:36.900625+0000","last_fullsized":"2024-08-20T01:06:36.900625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338281+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338281+0000","last_peered":"2024-08-20T01:06:37.338281+0000","last_clean":"2024-08-20T01:06:37.338281+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338281+0000","last_undegraded":"2024-08-20T01:06:37.338281+0000","last_fullsized":"2024-08-20T01:06:37.338281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567030+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:06:36.567030+0000","last_peered":"2024-08-20T01:06:36.567030+0000","last_clean":"2024-08-20T01:06:36.567030+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:06:36.567030+0000","last_undegraded":"2024-08-20T01:06:36.567030+0000","last_fullsized":"2024-08-20T01:06:36.567030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566684+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566684+0000","last_peered":"2024-08-20T01:06:36.566684+0000","last_clean":"2024-08-20T01:06:36.566684+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566684+0000","last_undegraded":"2024-08-20T01:06:36.566684+0000","last_fullsized":"2024-08-20T01:06:36.566684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566373+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:06:36.566373+0000","last_peered":"2024-08-20T01:06:36.566373+0000","last_clean":"2024-08-20T01:06:36.566373+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:06:36.566373+0000","last_undegraded":"2024-08-20T01:06:36.566373+0000","last_fullsized":"2024-08-20T01:06:36.566373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:44.217043+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:06:44.217043+0000","last_peered":"2024-08-20T01:06:44.217043+0000","last_clean":"2024-08-20T01:06:44.217043+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:06:44.217043+0000","last_undegraded":"2024-08-20T01:06:44.217043+0000","last_fullsized":"2024-08-20T01:06:44.217043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62966,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":56,"num_read_kb":83,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642413+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:06:36.642413+0000","last_peered":"2024-08-20T01:06:36.642413+0000","last_clean":"2024-08-20T01:06:36.642413+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:06:36.642413+0000","last_undegraded":"2024-08-20T01:06:36.642413+0000","last_fullsized":"2024-08-20T01:06:36.642413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564971+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.564971+0000","last_peered":"2024-08-20T01:06:36.564971+0000","last_clean":"2024-08-20T01:06:36.564971+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.564971+0000","last_undegraded":"2024-08-20T01:06:36.564971+0000","last_fullsized":"2024-08-20T01:06:36.564971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425415+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:06:37.425415+0000","last_peered":"2024-08-20T01:06:37.425415+0000","last_clean":"2024-08-20T01:06:37.425415+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:06:37.425415+0000","last_undegraded":"2024-08-20T01:06:37.425415+0000","last_fullsized":"2024-08-20T01:06:37.425415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642659+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:06:36.642659+0000","last_peered":"2024-08-20T01:06:36.642659+0000","last_clean":"2024-08-20T01:06:36.642659+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:06:36.642659+0000","last_undegraded":"2024-08-20T01:06:36.642659+0000","last_fullsized":"2024-08-20T01:06:36.642659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":68,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:42.656067+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:06:42.656067+0000","last_peered":"2024-08-20T01:06:42.656067+0000","last_clean":"2024-08-20T01:06:42.656067+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:06:42.656067+0000","last_undegraded":"2024-08-20T01:06:42.656067+0000","last_fullsized":"2024-08-20T01:06:42.656067+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425093+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425093+0000","last_peered":"2024-08-20T01:06:37.425093+0000","last_clean":"2024-08-20T01:06:37.425093+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425093+0000","last_undegraded":"2024-08-20T01:06:37.425093+0000","last_fullsized":"2024-08-20T01:06:37.425093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620212+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:06:36.620212+0000","last_peered":"2024-08-20T01:06:36.620212+0000","last_clean":"2024-08-20T01:06:36.620212+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:06:36.620212+0000","last_undegraded":"2024-08-20T01:06:36.620212+0000","last_fullsized":"2024-08-20T01:06:36.620212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338439+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338439+0000","last_peered":"2024-08-20T01:06:37.338439+0000","last_clean":"2024-08-20T01:06:37.338439+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338439+0000","last_undegraded":"2024-08-20T01:06:37.338439+0000","last_fullsized":"2024-08-20T01:06:37.338439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564203+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:06:36.564203+0000","last_peered":"2024-08-20T01:06:36.564203+0000","last_clean":"2024-08-20T01:06:36.564203+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:06:36.564203+0000","last_undegraded":"2024-08-20T01:06:36.564203+0000","last_fullsized":"2024-08-20T01:06:36.564203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647471+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:06:36.647471+0000","last_peered":"2024-08-20T01:06:36.647471+0000","last_clean":"2024-08-20T01:06:36.647471+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:06:36.647471+0000","last_undegraded":"2024-08-20T01:06:36.647471+0000","last_fullsized":"2024-08-20T01:06:36.647471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566490+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566490+0000","last_peered":"2024-08-20T01:06:36.566490+0000","last_clean":"2024-08-20T01:06:36.566490+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566490+0000","last_undegraded":"2024-08-20T01:06:36.566490+0000","last_fullsized":"2024-08-20T01:06:36.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:42.656640+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:06:42.656640+0000","last_peered":"2024-08-20T01:06:42.656640+0000","last_clean":"2024-08-20T01:06:42.656640+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:06:42.656640+0000","last_undegraded":"2024-08-20T01:06:42.656640+0000","last_fullsized":"2024-08-20T01:06:42.656640+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567358+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567358+0000","last_peered":"2024-08-20T01:06:36.567358+0000","last_clean":"2024-08-20T01:06:36.567358+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567358+0000","last_undegraded":"2024-08-20T01:06:36.567358+0000","last_fullsized":"2024-08-20T01:06:36.567358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338434+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:06:37.338434+0000","last_peered":"2024-08-20T01:06:37.338434+0000","last_clean":"2024-08-20T01:06:37.338434+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:06:37.338434+0000","last_undegraded":"2024-08-20T01:06:37.338434+0000","last_fullsized":"2024-08-20T01:06:37.338434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567133+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:06:36.567133+0000","last_peered":"2024-08-20T01:06:36.567133+0000","last_clean":"2024-08-20T01:06:36.567133+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:06:36.567133+0000","last_undegraded":"2024-08-20T01:06:36.567133+0000","last_fullsized":"2024-08-20T01:06:36.567133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566829+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:06:36.566829+0000","last_peered":"2024-08-20T01:06:36.566829+0000","last_clean":"2024-08-20T01:06:36.566829+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:06:36.566829+0000","last_undegraded":"2024-08-20T01:06:36.566829+0000","last_fullsized":"2024-08-20T01:06:36.566829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425165+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:06:37.425165+0000","last_peered":"2024-08-20T01:06:37.425165+0000","last_clean":"2024-08-20T01:06:37.425165+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:06:37.425165+0000","last_undegraded":"2024-08-20T01:06:37.425165+0000","last_fullsized":"2024-08-20T01:06:37.425165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567032+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:06:36.567032+0000","last_peered":"2024-08-20T01:06:36.567032+0000","last_clean":"2024-08-20T01:06:36.567032+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:06:36.567032+0000","last_undegraded":"2024-08-20T01:06:36.567032+0000","last_fullsized":"2024-08-20T01:06:36.567032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338685+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:06:37.338685+0000","last_peered":"2024-08-20T01:06:37.338685+0000","last_clean":"2024-08-20T01:06:37.338685+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:06:37.338685+0000","last_undegraded":"2024-08-20T01:06:37.338685+0000","last_fullsized":"2024-08-20T01:06:37.338685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567354+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:06:36.567354+0000","last_peered":"2024-08-20T01:06:36.567354+0000","last_clean":"2024-08-20T01:06:36.567354+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:06:36.567354+0000","last_undegraded":"2024-08-20T01:06:36.567354+0000","last_fullsized":"2024-08-20T01:06:36.567354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564876+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:06:36.564876+0000","last_peered":"2024-08-20T01:06:36.564876+0000","last_clean":"2024-08-20T01:06:36.564876+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:06:36.564876+0000","last_undegraded":"2024-08-20T01:06:36.564876+0000","last_fullsized":"2024-08-20T01:06:36.564876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647031+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:06:36.647031+0000","last_peered":"2024-08-20T01:06:36.647031+0000","last_clean":"2024-08-20T01:06:36.647031+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:06:36.647031+0000","last_undegraded":"2024-08-20T01:06:36.647031+0000","last_fullsized":"2024-08-20T01:06:36.647031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564381+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:06:36.564381+0000","last_peered":"2024-08-20T01:06:36.564381+0000","last_clean":"2024-08-20T01:06:36.564381+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:06:36.564381+0000","last_undegraded":"2024-08-20T01:06:36.564381+0000","last_fullsized":"2024-08-20T01:06:36.564381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642299+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:06:36.642299+0000","last_peered":"2024-08-20T01:06:36.642299+0000","last_clean":"2024-08-20T01:06:36.642299+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:06:36.642299+0000","last_undegraded":"2024-08-20T01:06:36.642299+0000","last_fullsized":"2024-08-20T01:06:36.642299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566850+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:06:36.566850+0000","last_peered":"2024-08-20T01:06:36.566850+0000","last_clean":"2024-08-20T01:06:36.566850+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:06:36.566850+0000","last_undegraded":"2024-08-20T01:06:36.566850+0000","last_fullsized":"2024-08-20T01:06:36.566850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338353+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:06:37.338353+0000","last_peered":"2024-08-20T01:06:37.338353+0000","last_clean":"2024-08-20T01:06:37.338353+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:06:37.338353+0000","last_undegraded":"2024-08-20T01:06:37.338353+0000","last_fullsized":"2024-08-20T01:06:37.338353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567521+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:06:36.567521+0000","last_peered":"2024-08-20T01:06:36.567521+0000","last_clean":"2024-08-20T01:06:36.567521+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:06:36.567521+0000","last_undegraded":"2024-08-20T01:06:36.567521+0000","last_fullsized":"2024-08-20T01:06:36.567521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620898+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:06:36.620898+0000","last_peered":"2024-08-20T01:06:36.620898+0000","last_clean":"2024-08-20T01:06:36.620898+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:06:36.620898+0000","last_undegraded":"2024-08-20T01:06:36.620898+0000","last_fullsized":"2024-08-20T01:06:36.620898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338717+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:06:37.338717+0000","last_peered":"2024-08-20T01:06:37.338717+0000","last_clean":"2024-08-20T01:06:37.338717+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:06:37.338717+0000","last_undegraded":"2024-08-20T01:06:37.338717+0000","last_fullsized":"2024-08-20T01:06:37.338717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642084+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:06:36.642084+0000","last_peered":"2024-08-20T01:06:36.642084+0000","last_clean":"2024-08-20T01:06:36.642084+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:06:36.642084+0000","last_undegraded":"2024-08-20T01:06:36.642084+0000","last_fullsized":"2024-08-20T01:06:36.642084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425202+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:06:37.425202+0000","last_peered":"2024-08-20T01:06:37.425202+0000","last_clean":"2024-08-20T01:06:37.425202+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:06:37.425202+0000","last_undegraded":"2024-08-20T01:06:37.425202+0000","last_fullsized":"2024-08-20T01:06:37.425202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.424796+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:06:37.424796+0000","last_peered":"2024-08-20T01:06:37.424796+0000","last_clean":"2024-08-20T01:06:37.424796+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:06:37.424796+0000","last_undegraded":"2024-08-20T01:06:37.424796+0000","last_fullsized":"2024-08-20T01:06:37.424796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566819+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:06:36.566819+0000","last_peered":"2024-08-20T01:06:36.566819+0000","last_clean":"2024-08-20T01:06:36.566819+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:06:36.566819+0000","last_undegraded":"2024-08-20T01:06:36.566819+0000","last_fullsized":"2024-08-20T01:06:36.566819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565000+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:06:36.565000+0000","last_peered":"2024-08-20T01:06:36.565000+0000","last_clean":"2024-08-20T01:06:36.565000+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:06:36.565000+0000","last_undegraded":"2024-08-20T01:06:36.565000+0000","last_fullsized":"2024-08-20T01:06:36.565000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900949+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:06:36.900949+0000","last_peered":"2024-08-20T01:06:36.900949+0000","last_clean":"2024-08-20T01:06:36.900949+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:06:36.900949+0000","last_undegraded":"2024-08-20T01:06:36.900949+0000","last_fullsized":"2024-08-20T01:06:36.900949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.645012+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:06:36.645012+0000","last_peered":"2024-08-20T01:06:36.645012+0000","last_clean":"2024-08-20T01:06:36.645012+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:06:36.645012+0000","last_undegraded":"2024-08-20T01:06:36.645012+0000","last_fullsized":"2024-08-20T01:06:36.645012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642836+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:06:36.642836+0000","last_peered":"2024-08-20T01:06:36.642836+0000","last_clean":"2024-08-20T01:06:36.642836+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:06:36.642836+0000","last_undegraded":"2024-08-20T01:06:36.642836+0000","last_fullsized":"2024-08-20T01:06:36.642836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620109+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:06:36.620109+0000","last_peered":"2024-08-20T01:06:36.620109+0000","last_clean":"2024-08-20T01:06:36.620109+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:06:36.620109+0000","last_undegraded":"2024-08-20T01:06:36.620109+0000","last_fullsized":"2024-08-20T01:06:36.620109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900485+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:06:36.900485+0000","last_peered":"2024-08-20T01:06:36.900485+0000","last_clean":"2024-08-20T01:06:36.900485+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:06:36.900485+0000","last_undegraded":"2024-08-20T01:06:36.900485+0000","last_fullsized":"2024-08-20T01:06:36.900485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644298+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:06:36.644298+0000","last_peered":"2024-08-20T01:06:36.644298+0000","last_clean":"2024-08-20T01:06:36.644298+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:06:36.644298+0000","last_undegraded":"2024-08-20T01:06:36.644298+0000","last_fullsized":"2024-08-20T01:06:36.644298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567413+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:06:36.567413+0000","last_peered":"2024-08-20T01:06:36.567413+0000","last_clean":"2024-08-20T01:06:36.567413+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:06:36.567413+0000","last_undegraded":"2024-08-20T01:06:36.567413+0000","last_fullsized":"2024-08-20T01:06:36.567413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620731+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:06:36.620731+0000","last_peered":"2024-08-20T01:06:36.620731+0000","last_clean":"2024-08-20T01:06:36.620731+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:06:36.620731+0000","last_undegraded":"2024-08-20T01:06:36.620731+0000","last_fullsized":"2024-08-20T01:06:36.620731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425241+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:06:37.425241+0000","last_peered":"2024-08-20T01:06:37.425241+0000","last_clean":"2024-08-20T01:06:37.425241+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:06:37.425241+0000","last_undegraded":"2024-08-20T01:06:37.425241+0000","last_fullsized":"2024-08-20T01:06:37.425241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425195+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:06:37.425195+0000","last_peered":"2024-08-20T01:06:37.425195+0000","last_clean":"2024-08-20T01:06:37.425195+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:06:37.425195+0000","last_undegraded":"2024-08-20T01:06:37.425195+0000","last_fullsized":"2024-08-20T01:06:37.425195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338711+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:06:37.338711+0000","last_peered":"2024-08-20T01:06:37.338711+0000","last_clean":"2024-08-20T01:06:37.338711+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:06:37.338711+0000","last_undegraded":"2024-08-20T01:06:37.338711+0000","last_fullsized":"2024-08-20T01:06:37.338711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66967,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":173,"num_read_kb":151,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678045,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874274,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1110016,"data_stored":1160608,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037799,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234029,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430257,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1113245,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593784,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790013,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":1108833,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149766,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378701,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":577209,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":575468,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705750,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:06:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:49 smithi138 ceph-mon[18076]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T01:06:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:49 smithi138 ceph-mon[18076]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:49.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:49 smithi167 ceph-mon[21965]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T01:06:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:49 smithi167 ceph-mon[21965]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:49 smithi142 ceph-mon[21799]: pgmap v323: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-08-20T01:06:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:49 smithi142 ceph-mon[21799]: from='client.24983 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:51 smithi167 ceph-mon[21965]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-20T01:06:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:51 smithi142 ceph-mon[21799]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-20T01:06:52.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:51 smithi138 ceph-mon[18076]: pgmap v324: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-08-20T01:06:52.504 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:06:52.885 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 -- 172.21.15.138:0/1545859253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 msgr2=0x7fe48010a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 --2- 172.21.15.138:0/1545859253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48010a950 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009900 tx=0x7fe46c02a410 comp rx=0 tx=0).stop 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 -- 172.21.15.138:0/1545859253 shutdown_connections 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 --2- 172.21.15.138:0/1545859253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 0x7fe480111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 --2- 172.21.15.138:0/1545859253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48010a950 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 --2- 172.21.15.138:0/1545859253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe480104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 -- 172.21.15.138:0/1545859253 >> 172.21.15.138:0/1545859253 conn(0x7fe480100550 msgr2=0x7fe480102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 -- 172.21.15.138:0/1545859253 shutdown_connections 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.425+0000 7fe47fe00640 1 -- 172.21.15.138:0/1545859253 wait complete. 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.426+0000 7fe47fe00640 1 Processor -- start 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.426+0000 7fe47fe00640 1 -- start start 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe48019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:53.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 0x7fe4801a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe480114ba0 con 0x7fe48010a510 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe480114a20 con 0x7fe480104b00 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe480114d20 con 0x7fe48010ae90 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe48019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59004/0 (socket says 172.21.15.138:59004) 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47e000640 1 -- 172.21.15.138:0/757554653 learned_addr learned my addr 172.21.15.138:0/757554653 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47f400640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 0x7fe4801a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.427+0000 7fe47ea00640 1 -- 172.21.15.138:0/757554653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 msgr2=0x7fe4801a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47ea00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 0x7fe4801a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47ea00640 1 -- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 msgr2=0x7fe48019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47ea00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47ea00640 1 -- 172.21.15.138:0/757554653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4801a1020 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47ea00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe48019af70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe47000e9f0 tx=0x7fe47000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47000ccc0 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4700108e0 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4801a1310 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe470018880 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.428+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4801a79d0 con 0x7fe480104b00 2024-08-20T01:06:53.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.429+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe480104ee0 con 0x7fe480104b00 2024-08-20T01:06:53.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.432+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fe470010430 con 0x7fe480104b00 2024-08-20T01:06:53.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.433+0000 7fe477400640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 0x7fe4640b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:53.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.433+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe4700e4a30 con 0x7fe480104b00 2024-08-20T01:06:53.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.434+0000 7fe47e000640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 0x7fe4640b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:53.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.434+0000 7fe47e000640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 0x7fe4640b6220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe480102cf0 tx=0x7fe46c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:53.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.435+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe4700ab400 con 0x7fe480104b00 2024-08-20T01:06:53.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.619+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe480107ad0 con 0x7fe4640b3d60 2024-08-20T01:06:53.618 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:53 smithi138 ceph-mon[18076]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-20T01:06:53.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.625+0000 7fe477400640 1 -- 172.21.15.138:0/757554653 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340594 (secure 0 0 0) 0x7fe480107ad0 con 0x7fe4640b3d60 2024-08-20T01:06:53.628 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:53.630 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 msgr2=0x7fe4640b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 0x7fe4640b6220 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe480102cf0 tx=0x7fe46c005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 msgr2=0x7fe48019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe48019af70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe47000e9f0 tx=0x7fe47000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 shutdown_connections 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe4640b3d60 0x7fe4640b6220 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48010ae90 0x7fe4801a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48010a510 0x7fe48019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 --2- 172.21.15.138:0/757554653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe480104b00 0x7fe48019af70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.629+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 >> 172.21.15.138:0/757554653 conn(0x7fe480100550 msgr2=0x7fe4801039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.630+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 shutdown_connections 2024-08-20T01:06:53.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:53.630+0000 7fe47fe00640 1 -- 172.21.15.138:0/757554653 wait complete. 2024-08-20T01:06:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:53 smithi167 ceph-mon[21965]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-20T01:06:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:53 smithi142 ceph-mon[21799]: pgmap v325: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 852 B/s wr, 6 op/s 2024-08-20T01:06:54.519 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-08-20T01:06:52.493660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657569,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":285,"num_read_kb":242,"num_write":294,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352848,"kb_used_data":11876,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588976,"statfs":{"total":1151940427776,"available":1151579111424,"internally_reserved":0,"allocated":12161024,"data_stored":8154797,"data_compressed":427158,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6806,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":42,"num_write":6,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008475"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621083+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621083+0000","last_peered":"2024-08-20T01:06:36.621083+0000","last_clean":"2024-08-20T01:06:36.621083+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621083+0000","last_undegraded":"2024-08-20T01:06:36.621083+0000","last_fullsized":"2024-08-20T01:06:36.621083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564621+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:06:36.564621+0000","last_peered":"2024-08-20T01:06:36.564621+0000","last_clean":"2024-08-20T01:06:36.564621+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:06:36.564621+0000","last_undegraded":"2024-08-20T01:06:36.564621+0000","last_fullsized":"2024-08-20T01:06:36.564621+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426580+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.426580+0000","last_peered":"2024-08-20T01:06:37.426580+0000","last_clean":"2024-08-20T01:06:37.426580+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.426580+0000","last_undegraded":"2024-08-20T01:06:37.426580+0000","last_fullsized":"2024-08-20T01:06:37.426580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564871+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564871+0000","last_peered":"2024-08-20T01:06:36.564871+0000","last_clean":"2024-08-20T01:06:36.564871+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564871+0000","last_undegraded":"2024-08-20T01:06:36.564871+0000","last_fullsized":"2024-08-20T01:06:36.564871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425408+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:06:37.425408+0000","last_peered":"2024-08-20T01:06:37.425408+0000","last_clean":"2024-08-20T01:06:37.425408+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:06:37.425408+0000","last_undegraded":"2024-08-20T01:06:37.425408+0000","last_fullsized":"2024-08-20T01:06:37.425408+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564424+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564424+0000","last_peered":"2024-08-20T01:06:36.564424+0000","last_clean":"2024-08-20T01:06:36.564424+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564424+0000","last_undegraded":"2024-08-20T01:06:36.564424+0000","last_fullsized":"2024-08-20T01:06:36.564424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642316+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:06:36.642316+0000","last_peered":"2024-08-20T01:06:36.642316+0000","last_clean":"2024-08-20T01:06:36.642316+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:06:36.642316+0000","last_undegraded":"2024-08-20T01:06:36.642316+0000","last_fullsized":"2024-08-20T01:06:36.642316+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649109+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649109+0000","last_peered":"2024-08-20T01:06:36.649109+0000","last_clean":"2024-08-20T01:06:36.649109+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649109+0000","last_undegraded":"2024-08-20T01:06:36.649109+0000","last_fullsized":"2024-08-20T01:06:36.649109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647417+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:06:36.647417+0000","last_peered":"2024-08-20T01:06:36.647417+0000","last_clean":"2024-08-20T01:06:36.647417+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:06:36.647417+0000","last_undegraded":"2024-08-20T01:06:36.647417+0000","last_fullsized":"2024-08-20T01:06:36.647417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564564+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564564+0000","last_peered":"2024-08-20T01:06:36.564564+0000","last_clean":"2024-08-20T01:06:36.564564+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564564+0000","last_undegraded":"2024-08-20T01:06:36.564564+0000","last_fullsized":"2024-08-20T01:06:36.564564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900542+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:06:36.900542+0000","last_peered":"2024-08-20T01:06:36.900542+0000","last_clean":"2024-08-20T01:06:36.900542+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:06:36.900542+0000","last_undegraded":"2024-08-20T01:06:36.900542+0000","last_fullsized":"2024-08-20T01:06:36.900542+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564813+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564813+0000","last_peered":"2024-08-20T01:06:36.564813+0000","last_clean":"2024-08-20T01:06:36.564813+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564813+0000","last_undegraded":"2024-08-20T01:06:36.564813+0000","last_fullsized":"2024-08-20T01:06:36.564813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564754+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:06:36.564754+0000","last_peered":"2024-08-20T01:06:36.564754+0000","last_clean":"2024-08-20T01:06:36.564754+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:06:36.564754+0000","last_undegraded":"2024-08-20T01:06:36.564754+0000","last_fullsized":"2024-08-20T01:06:36.564754+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564610+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564610+0000","last_peered":"2024-08-20T01:06:36.564610+0000","last_clean":"2024-08-20T01:06:36.564610+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564610+0000","last_undegraded":"2024-08-20T01:06:36.564610+0000","last_fullsized":"2024-08-20T01:06:36.564610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338323+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:06:37.338323+0000","last_peered":"2024-08-20T01:06:37.338323+0000","last_clean":"2024-08-20T01:06:37.338323+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:06:37.338323+0000","last_undegraded":"2024-08-20T01:06:37.338323+0000","last_fullsized":"2024-08-20T01:06:37.338323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564536+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564536+0000","last_peered":"2024-08-20T01:06:36.564536+0000","last_clean":"2024-08-20T01:06:36.564536+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564536+0000","last_undegraded":"2024-08-20T01:06:36.564536+0000","last_fullsized":"2024-08-20T01:06:36.564536+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567485+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:06:36.567485+0000","last_peered":"2024-08-20T01:06:36.567485+0000","last_clean":"2024-08-20T01:06:36.567485+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:06:36.567485+0000","last_undegraded":"2024-08-20T01:06:36.567485+0000","last_fullsized":"2024-08-20T01:06:36.567485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642212+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642212+0000","last_peered":"2024-08-20T01:06:36.642212+0000","last_clean":"2024-08-20T01:06:36.642212+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642212+0000","last_undegraded":"2024-08-20T01:06:36.642212+0000","last_fullsized":"2024-08-20T01:06:36.642212+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564951+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.564951+0000","last_peered":"2024-08-20T01:06:36.564951+0000","last_clean":"2024-08-20T01:06:36.564951+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.564951+0000","last_undegraded":"2024-08-20T01:06:36.564951+0000","last_fullsized":"2024-08-20T01:06:36.564951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425236+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:06:37.425236+0000","last_peered":"2024-08-20T01:06:37.425236+0000","last_clean":"2024-08-20T01:06:37.425236+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:06:37.425236+0000","last_undegraded":"2024-08-20T01:06:37.425236+0000","last_fullsized":"2024-08-20T01:06:37.425236+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565469+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565469+0000","last_peered":"2024-08-20T01:06:36.565469+0000","last_clean":"2024-08-20T01:06:36.565469+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565469+0000","last_undegraded":"2024-08-20T01:06:36.565469+0000","last_fullsized":"2024-08-20T01:06:36.565469+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566954+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.566954+0000","last_peered":"2024-08-20T01:06:36.566954+0000","last_clean":"2024-08-20T01:06:36.566954+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.566954+0000","last_undegraded":"2024-08-20T01:06:36.566954+0000","last_fullsized":"2024-08-20T01:06:36.566954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564938+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:06:36.564938+0000","last_peered":"2024-08-20T01:06:36.564938+0000","last_clean":"2024-08-20T01:06:36.564938+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:06:36.564938+0000","last_undegraded":"2024-08-20T01:06:36.564938+0000","last_fullsized":"2024-08-20T01:06:36.564938+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:38.619700+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:06:38.619700+0000","last_peered":"2024-08-20T01:06:38.619700+0000","last_clean":"2024-08-20T01:06:38.619700+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:06:38.619700+0000","last_undegraded":"2024-08-20T01:06:38.619700+0000","last_fullsized":"2024-08-20T01:06:38.619700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564087+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564087+0000","last_peered":"2024-08-20T01:06:36.564087+0000","last_clean":"2024-08-20T01:06:36.564087+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564087+0000","last_undegraded":"2024-08-20T01:06:36.564087+0000","last_fullsized":"2024-08-20T01:06:36.564087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567052+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:06:36.567052+0000","last_peered":"2024-08-20T01:06:36.567052+0000","last_clean":"2024-08-20T01:06:36.567052+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:06:36.567052+0000","last_undegraded":"2024-08-20T01:06:36.567052+0000","last_fullsized":"2024-08-20T01:06:36.567052+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566662+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566662+0000","last_peered":"2024-08-20T01:06:36.566662+0000","last_clean":"2024-08-20T01:06:36.566662+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566662+0000","last_undegraded":"2024-08-20T01:06:36.566662+0000","last_fullsized":"2024-08-20T01:06:36.566662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338256+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338256+0000","last_peered":"2024-08-20T01:06:37.338256+0000","last_clean":"2024-08-20T01:06:37.338256+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338256+0000","last_undegraded":"2024-08-20T01:06:37.338256+0000","last_fullsized":"2024-08-20T01:06:37.338256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644635+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:06:36.644635+0000","last_peered":"2024-08-20T01:06:36.644635+0000","last_clean":"2024-08-20T01:06:36.644635+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:06:36.644635+0000","last_undegraded":"2024-08-20T01:06:36.644635+0000","last_fullsized":"2024-08-20T01:06:36.644635+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620802+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620802+0000","last_peered":"2024-08-20T01:06:36.620802+0000","last_clean":"2024-08-20T01:06:36.620802+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620802+0000","last_undegraded":"2024-08-20T01:06:36.620802+0000","last_fullsized":"2024-08-20T01:06:36.620802+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644713+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:06:36.644713+0000","last_peered":"2024-08-20T01:06:36.644713+0000","last_clean":"2024-08-20T01:06:36.644713+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:06:36.644713+0000","last_undegraded":"2024-08-20T01:06:36.644713+0000","last_fullsized":"2024-08-20T01:06:36.644713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900709+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900709+0000","last_peered":"2024-08-20T01:06:36.900709+0000","last_clean":"2024-08-20T01:06:36.900709+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900709+0000","last_undegraded":"2024-08-20T01:06:36.900709+0000","last_fullsized":"2024-08-20T01:06:36.900709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566848+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566848+0000","last_peered":"2024-08-20T01:06:36.566848+0000","last_clean":"2024-08-20T01:06:36.566848+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566848+0000","last_undegraded":"2024-08-20T01:06:36.566848+0000","last_fullsized":"2024-08-20T01:06:36.566848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647086+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:06:36.647086+0000","last_peered":"2024-08-20T01:06:36.647086+0000","last_clean":"2024-08-20T01:06:36.647086+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:06:36.647086+0000","last_undegraded":"2024-08-20T01:06:36.647086+0000","last_fullsized":"2024-08-20T01:06:36.647086+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:48.660223+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:06:48.660223+0000","last_peered":"2024-08-20T01:06:48.660223+0000","last_clean":"2024-08-20T01:06:48.660223+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:06:48.660223+0000","last_undegraded":"2024-08-20T01:06:48.660223+0000","last_fullsized":"2024-08-20T01:06:48.660223+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566529+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:06:36.566529+0000","last_peered":"2024-08-20T01:06:36.566529+0000","last_clean":"2024-08-20T01:06:36.566529+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:06:36.566529+0000","last_undegraded":"2024-08-20T01:06:36.566529+0000","last_fullsized":"2024-08-20T01:06:36.566529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900529+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900529+0000","last_peered":"2024-08-20T01:06:36.900529+0000","last_clean":"2024-08-20T01:06:36.900529+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900529+0000","last_undegraded":"2024-08-20T01:06:36.900529+0000","last_fullsized":"2024-08-20T01:06:36.900529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564431+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:06:36.564431+0000","last_peered":"2024-08-20T01:06:36.564431+0000","last_clean":"2024-08-20T01:06:36.564431+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:06:36.564431+0000","last_undegraded":"2024-08-20T01:06:36.564431+0000","last_fullsized":"2024-08-20T01:06:36.564431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565285+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565285+0000","last_peered":"2024-08-20T01:06:36.565285+0000","last_clean":"2024-08-20T01:06:36.565285+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565285+0000","last_undegraded":"2024-08-20T01:06:36.565285+0000","last_fullsized":"2024-08-20T01:06:36.565285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901014+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:06:36.901014+0000","last_peered":"2024-08-20T01:06:36.901014+0000","last_clean":"2024-08-20T01:06:36.901014+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:06:36.901014+0000","last_undegraded":"2024-08-20T01:06:36.901014+0000","last_fullsized":"2024-08-20T01:06:36.901014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567767+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567767+0000","last_peered":"2024-08-20T01:06:36.567767+0000","last_clean":"2024-08-20T01:06:36.567767+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567767+0000","last_undegraded":"2024-08-20T01:06:36.567767+0000","last_fullsized":"2024-08-20T01:06:36.567767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900715+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:06:36.900715+0000","last_peered":"2024-08-20T01:06:36.900715+0000","last_clean":"2024-08-20T01:06:36.900715+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:06:36.900715+0000","last_undegraded":"2024-08-20T01:06:36.900715+0000","last_fullsized":"2024-08-20T01:06:36.900715+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647579+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647579+0000","last_peered":"2024-08-20T01:06:36.647579+0000","last_clean":"2024-08-20T01:06:36.647579+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647579+0000","last_undegraded":"2024-08-20T01:06:36.647579+0000","last_fullsized":"2024-08-20T01:06:36.647579+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900966+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:06:36.900966+0000","last_peered":"2024-08-20T01:06:36.900966+0000","last_clean":"2024-08-20T01:06:36.900966+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:06:36.900966+0000","last_undegraded":"2024-08-20T01:06:36.900966+0000","last_fullsized":"2024-08-20T01:06:36.900966+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567528+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567528+0000","last_peered":"2024-08-20T01:06:36.567528+0000","last_clean":"2024-08-20T01:06:36.567528+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567528+0000","last_undegraded":"2024-08-20T01:06:36.567528+0000","last_fullsized":"2024-08-20T01:06:36.567528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338655+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:06:37.338655+0000","last_peered":"2024-08-20T01:06:37.338655+0000","last_clean":"2024-08-20T01:06:37.338655+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:06:37.338655+0000","last_undegraded":"2024-08-20T01:06:37.338655+0000","last_fullsized":"2024-08-20T01:06:37.338655+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647257+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:06:36.647257+0000","last_peered":"2024-08-20T01:06:36.647257+0000","last_clean":"2024-08-20T01:06:36.647257+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:06:36.647257+0000","last_undegraded":"2024-08-20T01:06:36.647257+0000","last_fullsized":"2024-08-20T01:06:36.647257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338120+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338120+0000","last_peered":"2024-08-20T01:06:37.338120+0000","last_clean":"2024-08-20T01:06:37.338120+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338120+0000","last_undegraded":"2024-08-20T01:06:37.338120+0000","last_fullsized":"2024-08-20T01:06:37.338120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900387+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:06:36.900387+0000","last_peered":"2024-08-20T01:06:36.900387+0000","last_clean":"2024-08-20T01:06:36.900387+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:06:36.900387+0000","last_undegraded":"2024-08-20T01:06:36.900387+0000","last_fullsized":"2024-08-20T01:06:36.900387+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:48.660865+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:06:48.660865+0000","last_peered":"2024-08-20T01:06:48.660865+0000","last_clean":"2024-08-20T01:06:48.660865+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:06:48.660865+0000","last_undegraded":"2024-08-20T01:06:48.660865+0000","last_fullsized":"2024-08-20T01:06:48.660865+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900625+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:06:36.900625+0000","last_peered":"2024-08-20T01:06:36.900625+0000","last_clean":"2024-08-20T01:06:36.900625+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:06:36.900625+0000","last_undegraded":"2024-08-20T01:06:36.900625+0000","last_fullsized":"2024-08-20T01:06:36.900625+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338281+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338281+0000","last_peered":"2024-08-20T01:06:37.338281+0000","last_clean":"2024-08-20T01:06:37.338281+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338281+0000","last_undegraded":"2024-08-20T01:06:37.338281+0000","last_fullsized":"2024-08-20T01:06:37.338281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567030+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:06:36.567030+0000","last_peered":"2024-08-20T01:06:36.567030+0000","last_clean":"2024-08-20T01:06:36.567030+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:06:36.567030+0000","last_undegraded":"2024-08-20T01:06:36.567030+0000","last_fullsized":"2024-08-20T01:06:36.567030+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566684+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566684+0000","last_peered":"2024-08-20T01:06:36.566684+0000","last_clean":"2024-08-20T01:06:36.566684+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566684+0000","last_undegraded":"2024-08-20T01:06:36.566684+0000","last_fullsized":"2024-08-20T01:06:36.566684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566373+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:06:36.566373+0000","last_peered":"2024-08-20T01:06:36.566373+0000","last_clean":"2024-08-20T01:06:36.566373+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:06:36.566373+0000","last_undegraded":"2024-08-20T01:06:36.566373+0000","last_fullsized":"2024-08-20T01:06:36.566373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":142,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:48.664667+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:06:48.664667+0000","last_peered":"2024-08-20T01:06:48.664667+0000","last_clean":"2024-08-20T01:06:48.664667+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:06:48.664667+0000","last_undegraded":"2024-08-20T01:06:48.664667+0000","last_fullsized":"2024-08-20T01:06:48.664667+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62966,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":67,"num_read_kb":95,"num_write":35,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642413+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:06:36.642413+0000","last_peered":"2024-08-20T01:06:36.642413+0000","last_clean":"2024-08-20T01:06:36.642413+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:06:36.642413+0000","last_undegraded":"2024-08-20T01:06:36.642413+0000","last_fullsized":"2024-08-20T01:06:36.642413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564971+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.564971+0000","last_peered":"2024-08-20T01:06:36.564971+0000","last_clean":"2024-08-20T01:06:36.564971+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.564971+0000","last_undegraded":"2024-08-20T01:06:36.564971+0000","last_fullsized":"2024-08-20T01:06:36.564971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425415+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:06:37.425415+0000","last_peered":"2024-08-20T01:06:37.425415+0000","last_clean":"2024-08-20T01:06:37.425415+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:06:37.425415+0000","last_undegraded":"2024-08-20T01:06:37.425415+0000","last_fullsized":"2024-08-20T01:06:37.425415+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642659+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:06:36.642659+0000","last_peered":"2024-08-20T01:06:36.642659+0000","last_clean":"2024-08-20T01:06:36.642659+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:06:36.642659+0000","last_undegraded":"2024-08-20T01:06:36.642659+0000","last_fullsized":"2024-08-20T01:06:36.642659+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":73,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:47.663322+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:06:47.663322+0000","last_peered":"2024-08-20T01:06:47.663322+0000","last_clean":"2024-08-20T01:06:47.663322+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:06:47.663322+0000","last_undegraded":"2024-08-20T01:06:47.663322+0000","last_fullsized":"2024-08-20T01:06:47.663322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425093+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425093+0000","last_peered":"2024-08-20T01:06:37.425093+0000","last_clean":"2024-08-20T01:06:37.425093+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425093+0000","last_undegraded":"2024-08-20T01:06:37.425093+0000","last_fullsized":"2024-08-20T01:06:37.425093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620212+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:06:36.620212+0000","last_peered":"2024-08-20T01:06:36.620212+0000","last_clean":"2024-08-20T01:06:36.620212+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:06:36.620212+0000","last_undegraded":"2024-08-20T01:06:36.620212+0000","last_fullsized":"2024-08-20T01:06:36.620212+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338439+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338439+0000","last_peered":"2024-08-20T01:06:37.338439+0000","last_clean":"2024-08-20T01:06:37.338439+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338439+0000","last_undegraded":"2024-08-20T01:06:37.338439+0000","last_fullsized":"2024-08-20T01:06:37.338439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564203+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:06:36.564203+0000","last_peered":"2024-08-20T01:06:36.564203+0000","last_clean":"2024-08-20T01:06:36.564203+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:06:36.564203+0000","last_undegraded":"2024-08-20T01:06:36.564203+0000","last_fullsized":"2024-08-20T01:06:36.564203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647471+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:06:36.647471+0000","last_peered":"2024-08-20T01:06:36.647471+0000","last_clean":"2024-08-20T01:06:36.647471+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:06:36.647471+0000","last_undegraded":"2024-08-20T01:06:36.647471+0000","last_fullsized":"2024-08-20T01:06:36.647471+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566490+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566490+0000","last_peered":"2024-08-20T01:06:36.566490+0000","last_clean":"2024-08-20T01:06:36.566490+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566490+0000","last_undegraded":"2024-08-20T01:06:36.566490+0000","last_fullsized":"2024-08-20T01:06:36.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"86'6","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:47.663784+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:06:47.663784+0000","last_peered":"2024-08-20T01:06:47.663784+0000","last_clean":"2024-08-20T01:06:47.663784+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:06:47.663784+0000","last_undegraded":"2024-08-20T01:06:47.663784+0000","last_fullsized":"2024-08-20T01:06:47.663784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566946+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:06:36.566946+0000","last_peered":"2024-08-20T01:06:36.566946+0000","last_clean":"2024-08-20T01:06:36.566946+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:06:36.566946+0000","last_undegraded":"2024-08-20T01:06:36.566946+0000","last_fullsized":"2024-08-20T01:06:36.566946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567358+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567358+0000","last_peered":"2024-08-20T01:06:36.567358+0000","last_clean":"2024-08-20T01:06:36.567358+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567358+0000","last_undegraded":"2024-08-20T01:06:36.567358+0000","last_fullsized":"2024-08-20T01:06:36.567358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338434+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:06:37.338434+0000","last_peered":"2024-08-20T01:06:37.338434+0000","last_clean":"2024-08-20T01:06:37.338434+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:06:37.338434+0000","last_undegraded":"2024-08-20T01:06:37.338434+0000","last_fullsized":"2024-08-20T01:06:37.338434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567133+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:06:36.567133+0000","last_peered":"2024-08-20T01:06:36.567133+0000","last_clean":"2024-08-20T01:06:36.567133+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:06:36.567133+0000","last_undegraded":"2024-08-20T01:06:36.567133+0000","last_fullsized":"2024-08-20T01:06:36.567133+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566829+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:06:36.566829+0000","last_peered":"2024-08-20T01:06:36.566829+0000","last_clean":"2024-08-20T01:06:36.566829+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:06:36.566829+0000","last_undegraded":"2024-08-20T01:06:36.566829+0000","last_fullsized":"2024-08-20T01:06:36.566829+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425165+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:06:37.425165+0000","last_peered":"2024-08-20T01:06:37.425165+0000","last_clean":"2024-08-20T01:06:37.425165+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:06:37.425165+0000","last_undegraded":"2024-08-20T01:06:37.425165+0000","last_fullsized":"2024-08-20T01:06:37.425165+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567032+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:06:36.567032+0000","last_peered":"2024-08-20T01:06:36.567032+0000","last_clean":"2024-08-20T01:06:36.567032+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:06:36.567032+0000","last_undegraded":"2024-08-20T01:06:36.567032+0000","last_fullsized":"2024-08-20T01:06:36.567032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338685+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:06:37.338685+0000","last_peered":"2024-08-20T01:06:37.338685+0000","last_clean":"2024-08-20T01:06:37.338685+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:06:37.338685+0000","last_undegraded":"2024-08-20T01:06:37.338685+0000","last_fullsized":"2024-08-20T01:06:37.338685+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567354+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:06:36.567354+0000","last_peered":"2024-08-20T01:06:36.567354+0000","last_clean":"2024-08-20T01:06:36.567354+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:06:36.567354+0000","last_undegraded":"2024-08-20T01:06:36.567354+0000","last_fullsized":"2024-08-20T01:06:36.567354+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564876+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:06:36.564876+0000","last_peered":"2024-08-20T01:06:36.564876+0000","last_clean":"2024-08-20T01:06:36.564876+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:06:36.564876+0000","last_undegraded":"2024-08-20T01:06:36.564876+0000","last_fullsized":"2024-08-20T01:06:36.564876+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647031+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:06:36.647031+0000","last_peered":"2024-08-20T01:06:36.647031+0000","last_clean":"2024-08-20T01:06:36.647031+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:06:36.647031+0000","last_undegraded":"2024-08-20T01:06:36.647031+0000","last_fullsized":"2024-08-20T01:06:36.647031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564381+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:06:36.564381+0000","last_peered":"2024-08-20T01:06:36.564381+0000","last_clean":"2024-08-20T01:06:36.564381+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:06:36.564381+0000","last_undegraded":"2024-08-20T01:06:36.564381+0000","last_fullsized":"2024-08-20T01:06:36.564381+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642299+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:06:36.642299+0000","last_peered":"2024-08-20T01:06:36.642299+0000","last_clean":"2024-08-20T01:06:36.642299+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:06:36.642299+0000","last_undegraded":"2024-08-20T01:06:36.642299+0000","last_fullsized":"2024-08-20T01:06:36.642299+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566850+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:06:36.566850+0000","last_peered":"2024-08-20T01:06:36.566850+0000","last_clean":"2024-08-20T01:06:36.566850+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:06:36.566850+0000","last_undegraded":"2024-08-20T01:06:36.566850+0000","last_fullsized":"2024-08-20T01:06:36.566850+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338353+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:06:37.338353+0000","last_peered":"2024-08-20T01:06:37.338353+0000","last_clean":"2024-08-20T01:06:37.338353+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:06:37.338353+0000","last_undegraded":"2024-08-20T01:06:37.338353+0000","last_fullsized":"2024-08-20T01:06:37.338353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567521+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:06:36.567521+0000","last_peered":"2024-08-20T01:06:36.567521+0000","last_clean":"2024-08-20T01:06:36.567521+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:06:36.567521+0000","last_undegraded":"2024-08-20T01:06:36.567521+0000","last_fullsized":"2024-08-20T01:06:36.567521+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620898+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:06:36.620898+0000","last_peered":"2024-08-20T01:06:36.620898+0000","last_clean":"2024-08-20T01:06:36.620898+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:06:36.620898+0000","last_undegraded":"2024-08-20T01:06:36.620898+0000","last_fullsized":"2024-08-20T01:06:36.620898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338717+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:06:37.338717+0000","last_peered":"2024-08-20T01:06:37.338717+0000","last_clean":"2024-08-20T01:06:37.338717+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:06:37.338717+0000","last_undegraded":"2024-08-20T01:06:37.338717+0000","last_fullsized":"2024-08-20T01:06:37.338717+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642084+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:06:36.642084+0000","last_peered":"2024-08-20T01:06:36.642084+0000","last_clean":"2024-08-20T01:06:36.642084+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:06:36.642084+0000","last_undegraded":"2024-08-20T01:06:36.642084+0000","last_fullsized":"2024-08-20T01:06:36.642084+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425202+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:06:37.425202+0000","last_peered":"2024-08-20T01:06:37.425202+0000","last_clean":"2024-08-20T01:06:37.425202+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:06:37.425202+0000","last_undegraded":"2024-08-20T01:06:37.425202+0000","last_fullsized":"2024-08-20T01:06:37.425202+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.424796+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:06:37.424796+0000","last_peered":"2024-08-20T01:06:37.424796+0000","last_clean":"2024-08-20T01:06:37.424796+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:06:37.424796+0000","last_undegraded":"2024-08-20T01:06:37.424796+0000","last_fullsized":"2024-08-20T01:06:37.424796+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566819+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:06:36.566819+0000","last_peered":"2024-08-20T01:06:36.566819+0000","last_clean":"2024-08-20T01:06:36.566819+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:06:36.566819+0000","last_undegraded":"2024-08-20T01:06:36.566819+0000","last_fullsized":"2024-08-20T01:06:36.566819+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565000+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:06:36.565000+0000","last_peered":"2024-08-20T01:06:36.565000+0000","last_clean":"2024-08-20T01:06:36.565000+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:06:36.565000+0000","last_undegraded":"2024-08-20T01:06:36.565000+0000","last_fullsized":"2024-08-20T01:06:36.565000+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900949+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:06:36.900949+0000","last_peered":"2024-08-20T01:06:36.900949+0000","last_clean":"2024-08-20T01:06:36.900949+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:06:36.900949+0000","last_undegraded":"2024-08-20T01:06:36.900949+0000","last_fullsized":"2024-08-20T01:06:36.900949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.645012+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:06:36.645012+0000","last_peered":"2024-08-20T01:06:36.645012+0000","last_clean":"2024-08-20T01:06:36.645012+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:06:36.645012+0000","last_undegraded":"2024-08-20T01:06:36.645012+0000","last_fullsized":"2024-08-20T01:06:36.645012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642836+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:06:36.642836+0000","last_peered":"2024-08-20T01:06:36.642836+0000","last_clean":"2024-08-20T01:06:36.642836+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:06:36.642836+0000","last_undegraded":"2024-08-20T01:06:36.642836+0000","last_fullsized":"2024-08-20T01:06:36.642836+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620109+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:06:36.620109+0000","last_peered":"2024-08-20T01:06:36.620109+0000","last_clean":"2024-08-20T01:06:36.620109+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:06:36.620109+0000","last_undegraded":"2024-08-20T01:06:36.620109+0000","last_fullsized":"2024-08-20T01:06:36.620109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900485+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:06:36.900485+0000","last_peered":"2024-08-20T01:06:36.900485+0000","last_clean":"2024-08-20T01:06:36.900485+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:06:36.900485+0000","last_undegraded":"2024-08-20T01:06:36.900485+0000","last_fullsized":"2024-08-20T01:06:36.900485+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644298+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:06:36.644298+0000","last_peered":"2024-08-20T01:06:36.644298+0000","last_clean":"2024-08-20T01:06:36.644298+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:06:36.644298+0000","last_undegraded":"2024-08-20T01:06:36.644298+0000","last_fullsized":"2024-08-20T01:06:36.644298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567413+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:06:36.567413+0000","last_peered":"2024-08-20T01:06:36.567413+0000","last_clean":"2024-08-20T01:06:36.567413+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:06:36.567413+0000","last_undegraded":"2024-08-20T01:06:36.567413+0000","last_fullsized":"2024-08-20T01:06:36.567413+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620731+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:06:36.620731+0000","last_peered":"2024-08-20T01:06:36.620731+0000","last_clean":"2024-08-20T01:06:36.620731+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:06:36.620731+0000","last_undegraded":"2024-08-20T01:06:36.620731+0000","last_fullsized":"2024-08-20T01:06:36.620731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425241+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:06:37.425241+0000","last_peered":"2024-08-20T01:06:37.425241+0000","last_clean":"2024-08-20T01:06:37.425241+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:06:37.425241+0000","last_undegraded":"2024-08-20T01:06:37.425241+0000","last_fullsized":"2024-08-20T01:06:37.425241+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425195+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:06:37.425195+0000","last_peered":"2024-08-20T01:06:37.425195+0000","last_clean":"2024-08-20T01:06:37.425195+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:06:37.425195+0000","last_undegraded":"2024-08-20T01:06:37.425195+0000","last_fullsized":"2024-08-20T01:06:37.425195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338711+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:06:37.338711+0000","last_peered":"2024-08-20T01:06:37.338711+0000","last_clean":"2024-08-20T01:06:37.338711+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:06:37.338711+0000","last_undegraded":"2024-08-20T01:06:37.338711+0000","last_fullsized":"2024-08-20T01:06:37.338711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66967,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":204,"num_read_kb":173,"num_write":169,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":200901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144,"ondisk_log_size":144,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678046,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874275,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1167414,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037800,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234030,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430258,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1036288,"data_stored":1113245,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593785,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1044480,"data_stored":1108833,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149767,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378702,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":577209,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542225,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":575468,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":64544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:06:54.525 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T01:06:54.525 INFO:tasks.cephadm:Setting up client nodes... 2024-08-20T01:06:54.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-08-20T01:06:54.902 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 -- 172.21.15.138:0/3732418632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac108320 msgr2=0x7f74ac108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 --2- 172.21.15.138:0/3732418632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac108320 0x7f74ac108760 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7494009900 tx=0x7f749402a410 comp rx=0 tx=0).stop 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 -- 172.21.15.138:0/3732418632 shutdown_connections 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 --2- 172.21.15.138:0/3732418632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 0x7f74ac10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 --2- 172.21.15.138:0/3732418632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac108320 0x7f74ac108760 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 --2- 172.21.15.138:0/3732418632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac102910 0x7f74ac102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 -- 172.21.15.138:0/3732418632 >> 172.21.15.138:0/3732418632 conn(0x7f74ac0fe3c0 msgr2=0x7f74ac1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 -- 172.21.15.138:0/3732418632 shutdown_connections 2024-08-20T01:06:55.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.521+0000 7f74b0c00640 1 -- 172.21.15.138:0/3732418632 wait complete. 2024-08-20T01:06:55.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 Processor -- start 2024-08-20T01:06:55.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 -- start start 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac108320 0x7f74ac19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 0x7f74ac1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac1129b0 con 0x7f74ac102910 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac112830 con 0x7f74ac108320 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.522+0000 7f74b0c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f74ac112b30 con 0x7f74ac108ca0 2024-08-20T01:06:55.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59018/0 (socket says 172.21.15.138:59018) 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74a2a00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac108320 0x7f74ac19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 -- 172.21.15.138:0/1572579059 learned_addr learned my addr 172.21.15.138:0/1572579059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74abe00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 0x7f74ac1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 -- 172.21.15.138:0/1572579059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 msgr2=0x7f74ac1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 0x7f74ac1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 -- 172.21.15.138:0/1572579059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac108320 msgr2=0x7f74ac19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac108320 0x7f74ac19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 -- 172.21.15.138:0/1572579059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74ac1a0fd0 con 0x7f74ac102910 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.523+0000 7f74ab400640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e9f0 tx=0x7f749c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.524+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c00cd80 con 0x7f74ac102910 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.524+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f749c0108e0 con 0x7f74ac102910 2024-08-20T01:06:55.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.524+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f749c018980 con 0x7f74ac102910 2024-08-20T01:06:55.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.524+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74ac1a12c0 con 0x7f74ac102910 2024-08-20T01:06:55.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.524+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74ac1a7b50 con 0x7f74ac102910 2024-08-20T01:06:55.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.525+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74ac103bc0 con 0x7f74ac102910 2024-08-20T01:06:55.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.527+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f749c010430 con 0x7f74ac102910 2024-08-20T01:06:55.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.528+0000 7f74a8c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 0x7f74900b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:55.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.529+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f749c0e44f0 con 0x7f74ac102910 2024-08-20T01:06:55.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.529+0000 7f74a2a00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 0x7f74900b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:55.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.529+0000 7f74a2a00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 0x7f74900b62a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7494005ab0 tx=0x7f7494005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:55.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.532+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f749c0aaf20 con 0x7f74ac102910 2024-08-20T01:06:55.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.786+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f74ac10ee90 con 0x7f74ac102910 2024-08-20T01:06:55.785 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:55 smithi138 ceph-mon[18076]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:55.785 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:55 smithi138 ceph-mon[18076]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-20T01:06:55.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.790+0000 7f74a8c00640 1 -- 172.21.15.138:0/1572579059 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f749c0af470 con 0x7f74ac102910 2024-08-20T01:06:55.788 INFO:teuthology.orchestra.run.smithi138.stdout:[client.0] 2024-08-20T01:06:55.788 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQAv7MNmL9/5LhAA9fjvwYLqPgJb3m6boUhTCg== 2024-08-20T01:06:55.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 msgr2=0x7f74900b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:55.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 0x7f74900b62a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7494005ab0 tx=0x7f7494005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:55.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 msgr2=0x7f74ac19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:55.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f749c00e9f0 tx=0x7f749c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 shutdown_connections 2024-08-20T01:06:55.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f74900b3de0 0x7f74900b62a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f74ac108ca0 0x7f74ac1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f74ac108320 0x7f74ac19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 --2- 172.21.15.138:0/1572579059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac102910 0x7f74ac19af20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.792+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 >> 172.21.15.138:0/1572579059 conn(0x7f74ac0fe3c0 msgr2=0x7f74ac0758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.793+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 shutdown_connections 2024-08-20T01:06:55.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:55.793+0000 7f74b0c00640 1 -- 172.21.15.138:0/1572579059 wait complete. 2024-08-20T01:06:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:55 smithi167 ceph-mon[21965]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:55 smithi167 ceph-mon[21965]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-20T01:06:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:55 smithi142 ceph-mon[21799]: from='client.24989 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:06:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:55 smithi142 ceph-mon[21799]: pgmap v326: 129 pgs: 129 active+clean; 642 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 5 op/s 2024-08-20T01:06:56.551 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:06:56.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-08-20T01:06:56.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-08-20T01:06:56.631 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph config log 1 --format=json 2024-08-20T01:06:56.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:56 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T01:06:56.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:56 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T01:06:56.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:56 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T01:06:56.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:56 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T01:06:56.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:56 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-08-20T01:06:56.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:56 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1572579059' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-08-20T01:06:57.040 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:57.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 -- 172.21.15.138:0/21395334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 msgr2=0x7ff704107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:57.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 --2- 172.21.15.138:0/21395334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff704107840 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009900 tx=0x7ff6f002a400 comp rx=0 tx=0).stop 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 -- 172.21.15.138:0/21395334 shutdown_connections 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 --2- 172.21.15.138:0/21395334 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 --2- 172.21.15.138:0/21395334 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff704107840 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 --2- 172.21.15.138:0/21395334 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 -- 172.21.15.138:0/21395334 >> 172.21.15.138:0/21395334 conn(0x7ff7040fd420 msgr2=0x7ff7040ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 -- 172.21.15.138:0/21395334 shutdown_connections 2024-08-20T01:06:57.661 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.662+0000 7ff70b200640 1 -- 172.21.15.138:0/21395334 wait complete. 2024-08-20T01:06:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 Processor -- start 2024-08-20T01:06:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 -- start start 2024-08-20T01:06:57.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff704112ba0 con 0x7ff704107400 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff704112a20 con 0x7ff7041019f0 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.663+0000 7ff70b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff704112d20 con 0x7ff704107d80 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59026/0 (socket says 172.21.15.138:59026) 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703400640 1 -- 172.21.15.138:0/993240703 learned_addr learned my addr 172.21.15.138:0/993240703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff709200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 -- 172.21.15.138:0/993240703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 msgr2=0x7ff70419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 -- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 msgr2=0x7ff7041994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 -- 172.21.15.138:0/993240703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff70419f040 con 0x7ff7041019f0 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703400640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff709200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.664+0000 7ff703e00640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704198f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9f0 tx=0x7ff6f400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:57.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.665+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400ccc0 con 0x7ff7041019f0 2024-08-20T01:06:57.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.665+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70419f330 con 0x7ff7041019f0 2024-08-20T01:06:57.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.665+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6f40108e0 con 0x7ff7041019f0 2024-08-20T01:06:57.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.665+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7041a5bc0 con 0x7ff7041019f0 2024-08-20T01:06:57.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.665+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4018750 con 0x7ff7041019f0 2024-08-20T01:06:57.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.666+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff704101dd0 con 0x7ff7041019f0 2024-08-20T01:06:57.666 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.668+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff6f4010430 con 0x7ff7041019f0 2024-08-20T01:06:57.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.670+0000 7ff700c00640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 0x7ff6e80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:57.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.670+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff6f40e4800 con 0x7ff7041019f0 2024-08-20T01:06:57.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.670+0000 7ff703400640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 0x7ff6e80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:57.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.671+0000 7ff703400640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 0x7ff6e80b61f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff704106ac0 tx=0x7ff6f0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:57.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.671+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff6f40ab820 con 0x7ff7041019f0 2024-08-20T01:06:57.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.859+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7ff7041a5d90 con 0x7ff7041019f0 2024-08-20T01:06:57.858 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:57 smithi138 ceph-mon[18076]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-20T01:06:57.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.860+0000 7ff700c00640 1 -- 172.21.15.138:0/993240703 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v52) ==== 86+0+133 (secure 0 0 0) 0x7ff6f40ab120 con 0x7ff7041019f0 2024-08-20T01:06:57.858 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:57.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 msgr2=0x7ff6e80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:57.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 0x7ff6e80b61f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff704106ac0 tx=0x7ff6f0005a20 comp rx=0 tx=0).stop 2024-08-20T01:06:57.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 msgr2=0x7ff704198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:57.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704198f90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6f400e9f0 tx=0x7ff6f400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 shutdown_connections 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff6e80b3d30 0x7ff6e80b61f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff704107d80 0x7ff70419e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff704107400 0x7ff7041994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 --2- 172.21.15.138:0/993240703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7041019f0 0x7ff704198f90 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 >> 172.21.15.138:0/993240703 conn(0x7ff7040fd420 msgr2=0x7ff7040ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 shutdown_connections 2024-08-20T01:06:57.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:57.862+0000 7ff70b200640 1 -- 172.21.15.138:0/993240703 wait complete. 2024-08-20T01:06:57.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:57 smithi167 ceph-mon[21965]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-20T01:06:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:57 smithi142 ceph-mon[21799]: pgmap v327: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1023 B/s wr, 8 op/s 2024-08-20T01:06:58.644 INFO:teuthology.orchestra.run.smithi138.stdout:[{"version":52,"timestamp":"2024-08-20T01:05:14.325763+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-08-20T01:06:58.644 INFO:tasks.ceph_manager:config epoch is 52 2024-08-20T01:06:58.645 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-08-20T01:06:58.645 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-08-20T01:06:58.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph mgr dump --format=json 2024-08-20T01:06:58.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:58 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/993240703' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T01:06:58.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:58 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/993240703' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T01:06:59.012 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:06:59.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:58 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/993240703' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-08-20T01:06:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 -- 172.21.15.138:0/2631907656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c1090e0 msgr2=0x7fec1c1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 --2- 172.21.15.138:0/2631907656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c1090e0 0x7fec1c1094c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009900 tx=0x7fec0c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 -- 172.21.15.138:0/2631907656 shutdown_connections 2024-08-20T01:06:59.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 --2- 172.21.15.138:0/2631907656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c10f470 0x7fec1c115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 --2- 172.21.15.138:0/2631907656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 0x7fec1c10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 --2- 172.21.15.138:0/2631907656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c1090e0 0x7fec1c1094c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 -- 172.21.15.138:0/2631907656 >> 172.21.15.138:0/2631907656 conn(0x7fec1c078760 msgr2=0x7fec1c07ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 -- 172.21.15.138:0/2631907656 shutdown_connections 2024-08-20T01:06:59.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.603+0000 7fec20c00640 1 -- 172.21.15.138:0/2631907656 wait complete. 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 Processor -- start 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 -- start start 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 0x7fec1c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 0x7fec1c19fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:59.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c1190d0 con 0x7fec1c10f470 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c118f50 con 0x7fec1c1090e0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.604+0000 7fec20c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec1c119250 con 0x7fec1c10eaf0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec1a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48524/0 (socket says 172.21.15.138:48524) 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 0x7fec1c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec1a800640 1 -- 172.21.15.138:0/1907489535 learned_addr learned my addr 172.21.15.138:0/1907489535 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19400640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 0x7fec1c19fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 -- 172.21.15.138:0/1907489535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 msgr2=0x7fec1c19fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 0x7fec1c19fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 -- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 msgr2=0x7fec1c1a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 -- 172.21.15.138:0/1907489535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1c1a5650 con 0x7fec1c1090e0 2024-08-20T01:06:59.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec1a800640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:06:59.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.605+0000 7fec19e00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 0x7fec1c19f5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec0c02a9e0 tx=0x7fec0c003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:59.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.606+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c03f070 con 0x7fec1c1090e0 2024-08-20T01:06:59.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.606+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0c031d60 con 0x7fec1c1090e0 2024-08-20T01:06:59.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.606+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c035030 con 0x7fec1c1090e0 2024-08-20T01:06:59.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.606+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1c1a58e0 con 0x7fec1c1090e0 2024-08-20T01:06:59.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.606+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec1c1ac000 con 0x7fec1c1090e0 2024-08-20T01:06:59.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.607+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec1c1094c0 con 0x7fec1c1090e0 2024-08-20T01:06:59.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.609+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fec0c042070 con 0x7fec1c1090e0 2024-08-20T01:06:59.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.611+0000 7fec12a00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 0x7fec000b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:06:59.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.611+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fec0c105180 con 0x7fec1c1090e0 2024-08-20T01:06:59.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.611+0000 7fec19400640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 0x7fec000b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:06:59.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.611+0000 7fec19400640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 0x7fec000b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec1c10e6c0 tx=0x7fec0800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:06:59.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.613+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fec0c049030 con 0x7fec1c1090e0 2024-08-20T01:06:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:06:59 smithi138 ceph-mon[18076]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:06:59.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.842+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fec1c110cc0 con 0x7fec1c1090e0 2024-08-20T01:06:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.847+0000 7fec12a00640 1 -- 172.21.15.138:0/1907489535 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+291043 (secure 0 0 0) 0x7fec0c0cbb50 con 0x7fec1c1090e0 2024-08-20T01:06:59.849 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 msgr2=0x7fec000b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 0x7fec000b6170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fec1c10e6c0 tx=0x7fec0800b390 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 msgr2=0x7fec1c19f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 0x7fec1c19f5a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fec0c02a9e0 tx=0x7fec0c003cd0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 shutdown_connections 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec000b3cb0 0x7fec000b6170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec1c10f470 0x7fec1c1a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec1c10eaf0 0x7fec1c19fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 --2- 172.21.15.138:0/1907489535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec1c1090e0 0x7fec1c19f5a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.851+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 >> 172.21.15.138:0/1907489535 conn(0x7fec1c078760 msgr2=0x7fec1c078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.852+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 shutdown_connections 2024-08-20T01:06:59.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:06:59.852+0000 7fec20c00640 1 -- 172.21.15.138:0/1907489535 wait complete. 2024-08-20T01:06:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:06:59 smithi167 ceph-mon[21965]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:06:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:06:59 smithi142 ceph-mon[21799]: pgmap v328: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:00.737 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":21,"flags":0,"active_gid":14220,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":2106854212},{"type":"v1","addr":"172.21.15.138:6801","nonce":2106854212}]},"active_addr":"172.21.15.138:6801/2106854212","active_change":"2024-08-20T00:58:34.377927+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24169,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24182,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.138:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":237245147}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":2008609515}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1795942723}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":1934124547}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.138:0","nonce":541869348}]}]} 2024-08-20T01:07:00.743 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-08-20T01:07:00.744 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-08-20T01:07:00.744 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:07:00.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:00 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1907489535' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T01:07:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:00 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1907489535' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T01:07:01.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:00 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1907489535' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-08-20T01:07:01.112 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:01.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 -- 172.21.15.138:0/2151245953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f0104b00 msgr2=0x7f90f0104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 --2- 172.21.15.138:0/2151245953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f0104b00 0x7f90f0104ee0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009900 tx=0x7f90e402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 -- 172.21.15.138:0/2151245953 shutdown_connections 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 --2- 172.21.15.138:0/2151245953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f010ae90 0x7f90f0111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 --2- 172.21.15.138:0/2151245953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f010a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 --2- 172.21.15.138:0/2151245953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f0104b00 0x7f90f0104ee0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 -- 172.21.15.138:0/2151245953 >> 172.21.15.138:0/2151245953 conn(0x7f90f0100550 msgr2=0x7f90f0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 -- 172.21.15.138:0/2151245953 shutdown_connections 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.713+0000 7f90f6000640 1 -- 172.21.15.138:0/2151245953 wait complete. 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 Processor -- start 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 -- start start 2024-08-20T01:07:01.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f0104b00 0x7f90f019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f010ae90 0x7f90f01a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114b10 con 0x7f90f010a510 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114990 con 0x7f90f010ae90 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.714+0000 7f90f6000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90f0114c90 con 0x7f90f0104b00 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48546/0 (socket says 172.21.15.138:48546) 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 -- 172.21.15.138:0/2369203776 learned_addr learned my addr 172.21.15.138:0/2369203776 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90f4c00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f0104b00 0x7f90f019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90f5600640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f010ae90 0x7f90f01a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 -- 172.21.15.138:0/2369203776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f0104b00 msgr2=0x7f90f019aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f0104b00 0x7f90f019aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 -- 172.21.15.138:0/2369203776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f010ae90 msgr2=0x7f90f01a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f010ae90 0x7f90f01a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 -- 172.21.15.138:0/2369203776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f01a0f90 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.715+0000 7f90efe00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00ea60 tx=0x7f90dc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.716+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc00cde0 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.716+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f01a1280 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.716+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90dc0035f0 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.716+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc010690 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.716+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f01a7b10 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.717+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90f0104ee0 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.719+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f90dc0037a0 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.720+0000 7f90ed600640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 0x7f90d80b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.721+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f90dc0e44f0 con 0x7f90f010a510 2024-08-20T01:07:01.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.721+0000 7f90f4c00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 0x7f90d80b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:01.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.721+0000 7f90f4c00640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 0x7f90d80b62a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90e4031440 tx=0x7f90e4005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:01.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.724+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f90dc0ab510 con 0x7f90f010a510 2024-08-20T01:07:01.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.906+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90f01a7ce0 con 0x7f90f010a510 2024-08-20T01:07:01.905 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:01 smithi138 ceph-mon[18076]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:01.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.908+0000 7f90ed600640 1 -- 172.21.15.138:0/2369203776 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7f90dc0aae10 con 0x7f90f010a510 2024-08-20T01:07:01.907 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:01.907 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 msgr2=0x7f90d80b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 0x7f90d80b62a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f90e4031440 tx=0x7f90e4005b50 comp rx=0 tx=0).stop 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 msgr2=0x7f90f019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f90dc00ea60 tx=0x7f90dc00ef20 comp rx=0 tx=0).stop 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 shutdown_connections 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90d80b3de0 0x7f90d80b62a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.911+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90f010ae90 0x7f90f01a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.912+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f010a510 0x7f90f019b420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.912+0000 7f90f6000640 1 --2- 172.21.15.138:0/2369203776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90f0104b00 0x7f90f019aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:01.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.912+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 >> 172.21.15.138:0/2369203776 conn(0x7f90f0100550 msgr2=0x7f90f0102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:01.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.912+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 shutdown_connections 2024-08-20T01:07:01.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:01.912+0000 7f90f6000640 1 -- 172.21.15.138:0/2369203776 wait complete. 2024-08-20T01:07:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:01 smithi167 ceph-mon[21965]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:01 smithi142 ceph-mon[21799]: pgmap v329: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:02.950 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:02 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2369203776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:02.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:02 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2369203776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:02.988 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-08-20T01:07:02.988 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:07:02.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:02 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2369203776' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:03.356 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:03.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:03 smithi138 ceph-mon[18076]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:03 smithi167 ceph-mon[21965]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 -- 172.21.15.138:0/3648913694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8754075b30 msgr2=0x7f875410e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 --2- 172.21.15.138:0/3648913694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8754075b30 0x7f875410e9b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8744009900 tx=0x7f874402a410 comp rx=0 tx=0).stop 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 -- 172.21.15.138:0/3648913694 shutdown_connections 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 --2- 172.21.15.138:0/3648913694 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f875410eef0 0x7f87541112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 --2- 172.21.15.138:0/3648913694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8754075b30 0x7f875410e9b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 --2- 172.21.15.138:0/3648913694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075210 0x7f87540755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:03.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 -- 172.21.15.138:0/3648913694 >> 172.21.15.138:0/3648913694 conn(0x7f87540fe390 msgr2=0x7f87541007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:03.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 -- 172.21.15.138:0/3648913694 shutdown_connections 2024-08-20T01:07:03.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.985+0000 7f875b200640 1 -- 172.21.15.138:0/3648913694 wait complete. 2024-08-20T01:07:03.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.986+0000 7f875b200640 1 Processor -- start 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 -- start start 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075b30 0x7f875419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 0x7f87541a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8754114b20 con 0x7f875410eef0 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87541149a0 con 0x7f8754075b30 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8754114ca0 con 0x7f8754075210 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 0x7f87541a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f8759e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f8759e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44512/0 (socket says 172.21.15.138:44512) 2024-08-20T01:07:03.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f875a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 0x7f87541a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48562/0 (socket says 172.21.15.138:48562) 2024-08-20T01:07:03.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f8759e00640 1 -- 172.21.15.138:0/846790591 learned_addr learned my addr 172.21.15.138:0/846790591 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:03.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.987+0000 7f8759400640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075b30 0x7f875419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 -- 172.21.15.138:0/846790591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075b30 msgr2=0x7f875419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075b30 0x7f875419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 -- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 msgr2=0x7f87541a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 0x7f87541a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 -- 172.21.15.138:0/846790591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87541a0fe0 con 0x7f8754075210 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f8759e00640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f874800e9f0 tx=0x7f874800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:03.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f874800cd80 con 0x7f8754075210 2024-08-20T01:07:03.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87541a12d0 con 0x7f8754075210 2024-08-20T01:07:03.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87480108e0 con 0x7f8754075210 2024-08-20T01:07:03.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.988+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748018980 con 0x7f8754075210 2024-08-20T01:07:03.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.989+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87541a7b60 con 0x7f8754075210 2024-08-20T01:07:03.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.990+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8754076720 con 0x7f8754075210 2024-08-20T01:07:03.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.992+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f8748010430 con 0x7f8754075210 2024-08-20T01:07:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:03 smithi142 ceph-mon[21799]: pgmap v330: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:03.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.994+0000 7f874ea00640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 0x7f873c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:03.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.994+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f87480e4a10 con 0x7f8754075210 2024-08-20T01:07:03.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.994+0000 7f8759400640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 0x7f873c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:03.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.995+0000 7f8759400640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 0x7f873c0b62a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8744005ab0 tx=0x7f8744005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:03.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:03.995+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f87480ab3e0 con 0x7f8754075210 2024-08-20T01:07:04.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.183+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87541a7d30 con 0x7f8754075210 2024-08-20T01:07:04.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.185+0000 7f874ea00640 1 -- 172.21.15.138:0/846790591 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7f87480af930 con 0x7f8754075210 2024-08-20T01:07:04.184 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:04.184 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 msgr2=0x7f873c0b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 0x7f873c0b62a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8744005ab0 tx=0x7f8744005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 msgr2=0x7f875419af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f874800e9f0 tx=0x7f874800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 shutdown_connections 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f873c0b3de0 0x7f873c0b62a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f875410eef0 0x7f87541a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8754075b30 0x7f875419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 --2- 172.21.15.138:0/846790591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8754075210 0x7f875419af30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.187+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 >> 172.21.15.138:0/846790591 conn(0x7f87540fe390 msgr2=0x7f87540fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:04.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.188+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 shutdown_connections 2024-08-20T01:07:04.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:04.188+0000 7f875b200640 1 -- 172.21.15.138:0/846790591 wait complete. 2024-08-20T01:07:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/846790591' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/846790591' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/846790591' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:05.141 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-08-20T01:07:05.142 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-08-20T01:07:05.142 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-08-20T01:07:05.142 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-08-20T01:07:05.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-08-20T01:07:05.143 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-08-20T01:07:05.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-08-20T01:07:05.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-08-20T01:07:05.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-08-20T01:07:05.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-08-20T01:07:05.145 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-08-20T01:07:05.146 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-08-20T01:07:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:05 smithi167 ceph-mon[21965]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:05 smithi142 ceph-mon[21799]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:06.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:05 smithi138 ceph-mon[18076]: pgmap v331: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:07:06.140 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:06.227 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 -- 172.21.15.138:0/3097232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 msgr2=0x7f1c901116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 --2- 172.21.15.138:0/3097232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901116a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009ca0 tx=0x7f1c8c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 -- 172.21.15.138:0/3097232 shutdown_connections 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 --2- 172.21.15.138:0/3097232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901116a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 --2- 172.21.15.138:0/3097232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 0x7f1c9010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 --2- 172.21.15.138:0/3097232 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c90104b10 0x7f1c90104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.967+0000 7f1c97200640 1 -- 172.21.15.138:0/3097232 >> 172.21.15.138:0/3097232 conn(0x7f1c90100540 msgr2=0x7f1c90102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.968+0000 7f1c97200640 1 -- 172.21.15.138:0/3097232 shutdown_connections 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.968+0000 7f1c97200640 1 -- 172.21.15.138:0/3097232 wait complete. 2024-08-20T01:07:07.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 Processor -- start 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 -- start start 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c90104b10 0x7f1c9019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 0x7f1c9019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90114bb0 con 0x7f1c9010aea0 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90114a30 con 0x7f1c9010a520 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c97200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1c90114d30 con 0x7f1c90104b10 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c96800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c95400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 0x7f1c9019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c96800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48586/0 (socket says 172.21.15.138:48586) 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.969+0000 7f1c96800640 1 -- 172.21.15.138:0/488912124 learned_addr learned my addr 172.21.15.138:0/488912124 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95e00640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c90104b10 0x7f1c9019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 -- 172.21.15.138:0/488912124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c90104b10 msgr2=0x7f1c9019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1c90104b10 0x7f1c9019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 -- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 msgr2=0x7f1c901a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c901a0fb0 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c96800640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c9010aea0 0x7f1c901a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c95400640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 0x7f1c9019b440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000e9f0 tx=0x7f1c8000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.970+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8000ccc0 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.971+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c800108e0 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.971+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c901a12a0 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.971+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80018880 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.971+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c901a7960 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.972+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c90107e30 con 0x7f1c9010a520 2024-08-20T01:07:07.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.975+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f1c80010430 con 0x7f1c9010a520 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c8aa00640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1c780b3d60 0x7f1c780b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1c800e4db0 con 0x7f1c9010a520 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c95e00640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1c780b3d60 0x7f1c780b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c8aa00640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] conn(0x7f1c780bf2b0 0x7f1c780c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c80015e80 con 0x7f1c780bf2b0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.977+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f1c800e5160 con 0x7f1c9010a520 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.978+0000 7f1c96800640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] conn(0x7f1c780bf2b0 0x7f1c780c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.978+0000 7f1c95e00640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1c780b3d60 0x7f1c780b6220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009bd0 tx=0x7f1c84005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.978+0000 7f1c96800640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] conn(0x7f1c780bf2b0 0x7f1c780c1750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.979+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== osd.0 v2:172.21.15.138:6802/2240185671 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c80015e80 con 0x7f1c780bf2b0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.994+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 --> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c90105dc0 con 0x7f1c780bf2b0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.994+0000 7f1c8aa00640 1 -- 172.21.15.138:0/488912124 <== osd.0 v2:172.21.15.138:6802/2240185671 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1c90105dc0 con 0x7f1c780bf2b0 2024-08-20T01:07:07.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] conn(0x7f1c780bf2b0 msgr2=0x7f1c780c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] conn(0x7f1c780bf2b0 0x7f1c780c1750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1c780b3d60 msgr2=0x7f1c780b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1c780b3d60 0x7f1c780b6220 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c84009bd0 tx=0x7f1c84005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 msgr2=0x7f1c9019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 --2- 172.21.15.138:0/488912124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1c9010a520 0x7f1c9019b440 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000e9f0 tx=0x7f1c8000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c96800640 1 -- 172.21.15.138:0/488912124 reap_dead start 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 shutdown_connections 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 >> 172.21.15.138:0/488912124 conn(0x7f1c90100540 msgr2=0x7f1c90103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:07.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 shutdown_connections 2024-08-20T01:07:07.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:06.995+0000 7f1c97200640 1 -- 172.21.15.138:0/488912124 wait complete. 2024-08-20T01:07:07.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 -- 172.21.15.138:0/434713124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108330 msgr2=0x7fbf70108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/434713124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108330 0x7fbf70108770 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009900 tx=0x7fbf5c02a410 comp rx=0 tx=0).stop 2024-08-20T01:07:07.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 -- 172.21.15.138:0/434713124 shutdown_connections 2024-08-20T01:07:07.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/434713124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70108cb0 0x7fbf7010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/434713124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108330 0x7fbf70108770 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/434713124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70102920 0x7fbf70102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 -- 172.21.15.138:0/434713124 >> 172.21.15.138:0/434713124 conn(0x7fbf700fe350 msgr2=0x7fbf70100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:07.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 -- 172.21.15.138:0/434713124 shutdown_connections 2024-08-20T01:07:07.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.490+0000 7fbf6fe00640 1 -- 172.21.15.138:0/434713124 wait complete. 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.492+0000 7fbf6fe00640 1 Processor -- start 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.492+0000 7fbf6fe00640 1 -- start start 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.492+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70102920 0x7fbf70198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.492+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 0x7fbf70199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.492+0000 7fbf6fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108cb0 0x7fbf7019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf701129c0 con 0x7fbf70108cb0 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70112840 con 0x7fbf70102920 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf70112b40 con 0x7fbf70108330 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 0x7fbf70199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70102920 0x7fbf70198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 0x7fbf70199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44546/0 (socket says 172.21.15.138:44546) 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 -- 172.21.15.138:0/3840451530 learned_addr learned my addr 172.21.15.138:0/3840451530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6f400640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108cb0 0x7fbf7019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 -- 172.21.15.138:0/3840451530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70102920 msgr2=0x7fbf70198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf70102920 0x7fbf70198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 -- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108cb0 msgr2=0x7fbf7019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf70108cb0 0x7fbf7019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.493+0000 7fbf6e000640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7019eea0 con 0x7fbf70108330 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf6e000640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 0x7fbf70199330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005ab0 tx=0x7fbf5c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c03f070 con 0x7fbf70108330 2024-08-20T01:07:07.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7019f130 con 0x7fbf70108330 2024-08-20T01:07:07.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf5c031b60 con 0x7fbf70108330 2024-08-20T01:07:07.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c035030 con 0x7fbf70108330 2024-08-20T01:07:07.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.494+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf700750b0 con 0x7fbf70108330 2024-08-20T01:07:07.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.495+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf70105bc0 con 0x7fbf70108330 2024-08-20T01:07:07.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.499+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fbf5c042070 con 0x7fbf70108330 2024-08-20T01:07:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf74a00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf540b3d60 0x7fbf540b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fbf5c105030 con 0x7fbf70108330 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf6ea00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf540b3d60 0x7fbf540b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf74a00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] conn(0x7fbf540bf2b0 0x7fbf540c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf540c1e20 con 0x7fbf540bf2b0 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.501+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fbf5c105480 con 0x7fbf70108330 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.502+0000 7fbf6f400640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] conn(0x7fbf540bf2b0 0x7fbf540c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:07.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.503+0000 7fbf6ea00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf540b3d60 0x7fbf540b6220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009bd0 tx=0x7fbf60005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.503+0000 7fbf6f400640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] conn(0x7fbf540bf2b0 0x7fbf540c1750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:07.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.504+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== osd.3 v2:172.21.15.138:6810/876217371 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf540c1e20 con 0x7fbf540bf2b0 2024-08-20T01:07:07.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.523+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 --> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbf70103bd0 con 0x7fbf540bf2b0 2024-08-20T01:07:07.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.524+0000 7fbf74a00640 1 -- 172.21.15.138:0/3840451530 <== osd.3 v2:172.21.15.138:6810/876217371 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbf70103bd0 con 0x7fbf540bf2b0 2024-08-20T01:07:07.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] conn(0x7fbf540bf2b0 msgr2=0x7fbf540c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] conn(0x7fbf540bf2b0 0x7fbf540c1750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf540b3d60 msgr2=0x7fbf540b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf540b3d60 0x7fbf540b6220 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009bd0 tx=0x7fbf60005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 msgr2=0x7fbf70199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 --2- 172.21.15.138:0/3840451530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf70108330 0x7fbf70199330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005ab0 tx=0x7fbf5c003c10 comp rx=0 tx=0).stop 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6f400640 1 -- 172.21.15.138:0/3840451530 reap_dead start 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 shutdown_connections 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.525+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 >> 172.21.15.138:0/3840451530 conn(0x7fbf700fe350 msgr2=0x7fbf70100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:07.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.526+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 shutdown_connections 2024-08-20T01:07:07.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:07.527+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3840451530 wait complete. 2024-08-20T01:07:07.526 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:07.634 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:07 smithi138 ceph-mon[18076]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:07 smithi167 ceph-mon[21965]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:07 smithi142 ceph-mon[21799]: pgmap v332: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-08-20T01:07:08.511 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:08.518 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:08.518 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:08.518 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:08.530 INFO:teuthology.orchestra.run.smithi138.stdout:111669149771 2024-08-20T01:07:08.530 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-08-20T01:07:09.165 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:09.194 INFO:teuthology.orchestra.run.smithi138.stdout:38654705755 2024-08-20T01:07:09.194 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-08-20T01:07:09.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 -- 172.21.15.138:0/2412867714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40410aea0 msgr2=0x7fd4041116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 --2- 172.21.15.138:0/2412867714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40410aea0 0x7fd4041116a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc009860 tx=0x7fd3fc032b90 comp rx=0 tx=0).stop 2024-08-20T01:07:09.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 -- 172.21.15.138:0/2412867714 shutdown_connections 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 --2- 172.21.15.138:0/2412867714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd40410aea0 0x7fd4041116a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 --2- 172.21.15.138:0/2412867714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 0x7fd40410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 --2- 172.21.15.138:0/2412867714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd404104b10 0x7fd404104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.408+0000 7fd40a200640 1 -- 172.21.15.138:0/2412867714 >> 172.21.15.138:0/2412867714 conn(0x7fd404100540 msgr2=0x7fd404102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.409+0000 7fd40a200640 1 -- 172.21.15.138:0/2412867714 shutdown_connections 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.409+0000 7fd40a200640 1 -- 172.21.15.138:0/2412867714 wait complete. 2024-08-20T01:07:09.408 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:09.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.411+0000 7fd40a200640 1 Processor -- start 2024-08-20T01:07:09.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.412+0000 7fd40a200640 1 -- start start 2024-08-20T01:07:09.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd404104b10 0x7fd404197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:09.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 0x7fd404197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd40410aea0 0x7fd404199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd404079b50 con 0x7fd40410aea0 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd4040799d0 con 0x7fd40410a520 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.413+0000 7fd40a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd404079cd0 con 0x7fd404104b10 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.414+0000 7fd403e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 0x7fd404197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.414+0000 7fd403e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 0x7fd404197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34342/0 (socket says 172.21.15.138:34342) 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.414+0000 7fd403e00640 1 -- 172.21.15.138:0/347304010 learned_addr learned my addr 172.21.15.138:0/347304010 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:09.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.414+0000 7fd409800640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd40410aea0 0x7fd404199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:09.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 -- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd404104b10 msgr2=0x7fd404197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd408e00640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd404104b10 0x7fd404197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd404104b10 0x7fd404197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 -- 172.21.15.138:0/347304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 msgr2=0x7fd404197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd40410a520 0x7fd404197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4041998e0 con 0x7fd40410aea0 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.415+0000 7fd409800640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd40410aea0 0x7fd404199160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc03e040 tx=0x7fd3fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.416+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc04a070 con 0x7fd40410aea0 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.416+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3fc005500 con 0x7fd40410aea0 2024-08-20T01:07:09.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.416+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc004920 con 0x7fd40410aea0 2024-08-20T01:07:09.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.416+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd404199b70 con 0x7fd40410aea0 2024-08-20T01:07:09.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.416+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4041a3520 con 0x7fd40410aea0 2024-08-20T01:07:09.416 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:09.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.417+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3e4001060 con 0x7fd40410aea0 2024-08-20T01:07:09.416 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:09.417 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:09.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.423+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fd3fc034b60 con 0x7fd40410aea0 2024-08-20T01:07:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.427+0000 7fd401600640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd3e80b3e60 0x7fd3e80b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.427+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd3fc10d4c0 con 0x7fd40410aea0 2024-08-20T01:07:09.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.429+0000 7fd408e00640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd3e80b3e60 0x7fd3e80b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:09.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.429+0000 7fd401600640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] conn(0x7fd3e80bf3b0 0x7fd3e80c1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:09.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.430+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3fc056e70 con 0x7fd3e80bf3b0 2024-08-20T01:07:09.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.430+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fd3fc10d8b0 con 0x7fd40410aea0 2024-08-20T01:07:09.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.430+0000 7fd403e00640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] conn(0x7fd3e80bf3b0 0x7fd3e80c1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:09.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.431+0000 7fd408e00640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd3e80b3e60 0x7fd3e80b6320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4005fd0 tx=0x7fd3f40095c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:09.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.431+0000 7fd403e00640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] conn(0x7fd3e80bf3b0 0x7fd3e80c1850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:09.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.433+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== osd.5 v2:172.21.15.167:6808/808331745 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3fc056e70 con 0x7fd3e80bf3b0 2024-08-20T01:07:09.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.462+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 --> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd3e4002e00 con 0x7fd3e80bf3b0 2024-08-20T01:07:09.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.463+0000 7fd401600640 1 -- 172.21.15.138:0/347304010 <== osd.5 v2:172.21.15.167:6808/808331745 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd3e4002e00 con 0x7fd3e80bf3b0 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] conn(0x7fd3e80bf3b0 msgr2=0x7fd3e80c1850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] conn(0x7fd3e80bf3b0 0x7fd3e80c1850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd3e80b3e60 msgr2=0x7fd3e80b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd3e80b3e60 0x7fd3e80b6320 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4005fd0 tx=0x7fd3f40095c0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd40410aea0 msgr2=0x7fd404199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 --2- 172.21.15.138:0/347304010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd40410aea0 0x7fd404199160 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc03e040 tx=0x7fd3fc0045e0 comp rx=0 tx=0).stop 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd409800640 1 -- 172.21.15.138:0/347304010 reap_dead start 2024-08-20T01:07:09.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 shutdown_connections 2024-08-20T01:07:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.464+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 >> 172.21.15.138:0/347304010 conn(0x7fd404100540 msgr2=0x7fd4041072e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.465+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 shutdown_connections 2024-08-20T01:07:09.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:09.465+0000 7fd40a200640 1 -- 172.21.15.138:0/347304010 wait complete. 2024-08-20T01:07:09.749 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:09 smithi138 ceph-mon[18076]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:09 smithi167 ceph-mon[21965]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:09 smithi142 ceph-mon[21799]: pgmap v333: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 -- 172.21.15.138:0/746628081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2410aea0 msgr2=0x7feb241116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 --2- 172.21.15.138:0/746628081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2410aea0 0x7feb241116a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb1c009ca0 tx=0x7feb1c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 -- 172.21.15.138:0/746628081 shutdown_connections 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 --2- 172.21.15.138:0/746628081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb2410aea0 0x7feb241116a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 --2- 172.21.15.138:0/746628081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2410a520 0x7feb2410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 --2- 172.21.15.138:0/746628081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb24104b10 0x7feb24104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 -- 172.21.15.138:0/746628081 >> 172.21.15.138:0/746628081 conn(0x7feb24100540 msgr2=0x7feb24102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 -- 172.21.15.138:0/746628081 shutdown_connections 2024-08-20T01:07:10.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.268+0000 7feb29a00640 1 -- 172.21.15.138:0/746628081 wait complete. 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.269+0000 7feb29a00640 1 Processor -- start 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.269+0000 7feb29a00640 1 -- start start 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 0x7feb240773e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2410a520 0x7feb24077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2410aea0 0x7feb24077e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb24114b20 con 0x7feb24104b10 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb241149a0 con 0x7feb2410aea0 2024-08-20T01:07:10.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb24114ca0 con 0x7feb2410a520 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb23e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 0x7feb240773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb23e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 0x7feb240773e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54888/0 (socket says 172.21.15.138:54888) 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb23e00640 1 -- 172.21.15.138:0/4135705102 learned_addr learned my addr 172.21.15.138:0/4135705102 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb29000640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2410aea0 0x7feb24077e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.270+0000 7feb23400640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2410a520 0x7feb24077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 -- 172.21.15.138:0/4135705102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2410a520 msgr2=0x7feb24077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb2410a520 0x7feb24077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 -- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2410aea0 msgr2=0x7feb24077e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb2410aea0 0x7feb24077e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb241ab520 con 0x7feb24104b10 2024-08-20T01:07:10.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb23e00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 0x7feb240773e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:10.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1000cd80 con 0x7feb24104b10 2024-08-20T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb100108e0 con 0x7feb24104b10 2024-08-20T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb10018980 con 0x7feb24104b10 2024-08-20T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb241ab810 con 0x7feb24104b10 2024-08-20T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.271+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb241abd50 con 0x7feb24104b10 2024-08-20T01:07:10.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.272+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb24101710 con 0x7feb24104b10 2024-08-20T01:07:10.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.275+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7feb10010430 con 0x7feb24104b10 2024-08-20T01:07:10.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.276+0000 7feb20c00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb080b3d60 0x7feb080b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:10.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7feb100e4db0 con 0x7feb24104b10 2024-08-20T01:07:10.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb23400640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb080b3d60 0x7feb080b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:10.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb20c00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] conn(0x7feb080bf2b0 0x7feb080c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:10.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb080c1e20 con 0x7feb080bf2b0 2024-08-20T01:07:10.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7feb100e5160 con 0x7feb24104b10 2024-08-20T01:07:10.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.277+0000 7feb29000640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] conn(0x7feb080bf2b0 0x7feb080c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:10.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.278+0000 7feb23400640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb080b3d60 0x7feb080b6220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb2407aef0 tx=0x7feb14005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:10.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.278+0000 7feb29000640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] conn(0x7feb080bf2b0 0x7feb080c1750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:10.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.279+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== osd.10 v2:172.21.15.142:6824/1588947901 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb080c1e20 con 0x7feb080bf2b0 2024-08-20T01:07:10.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.293+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 --> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb24105dc0 con 0x7feb080bf2b0 2024-08-20T01:07:10.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb20c00640 1 -- 172.21.15.138:0/4135705102 <== osd.10 v2:172.21.15.142:6824/1588947901 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feb24105dc0 con 0x7feb080bf2b0 2024-08-20T01:07:10.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] conn(0x7feb080bf2b0 msgr2=0x7feb080c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] conn(0x7feb080bf2b0 0x7feb080c1750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb080b3d60 msgr2=0x7feb080b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb080b3d60 0x7feb080b6220 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb2407aef0 tx=0x7feb14005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:10.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 msgr2=0x7feb240773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29a00640 1 --2- 172.21.15.138:0/4135705102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb24104b10 0x7feb240773e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb1000e9f0 tx=0x7feb1000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.294+0000 7feb29000640 1 -- 172.21.15.138:0/4135705102 reap_dead start 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.295+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 shutdown_connections 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.295+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 >> 172.21.15.138:0/4135705102 conn(0x7feb24100540 msgr2=0x7feb24100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.295+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 shutdown_connections 2024-08-20T01:07:10.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:10.295+0000 7feb29a00640 1 -- 172.21.15.138:0/4135705102 wait complete. 2024-08-20T01:07:11.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 -- 172.21.15.138:0/597449061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108330 msgr2=0x7f5a30108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 --2- 172.21.15.138:0/597449061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108330 0x7f5a30108770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009900 tx=0x7f5a1c02a410 comp rx=0 tx=0).stop 2024-08-20T01:07:11.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 -- 172.21.15.138:0/597449061 shutdown_connections 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 --2- 172.21.15.138:0/597449061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30108cb0 0x7f5a3010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 --2- 172.21.15.138:0/597449061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108330 0x7f5a30108770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 --2- 172.21.15.138:0/597449061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30102920 0x7f5a30102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 -- 172.21.15.138:0/597449061 >> 172.21.15.138:0/597449061 conn(0x7f5a300fe350 msgr2=0x7f5a30100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 -- 172.21.15.138:0/597449061 shutdown_connections 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.243+0000 7f5a34c00640 1 -- 172.21.15.138:0/597449061 wait complete. 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 Processor -- start 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 -- start start 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30102920 0x7f5a3019aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 0x7f5a3019b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108cb0 0x7f5a301a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5a301129c0 con 0x7f5a30108cb0 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30112840 con 0x7f5a30102920 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.244+0000 7f5a34c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5a30112b40 con 0x7f5a30108330 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 0x7f5a3019b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30102920 0x7f5a3019aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108cb0 0x7f5a301a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 0x7f5a3019b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37302/0 (socket says 172.21.15.138:37302) 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 -- 172.21.15.138:0/1451638475 learned_addr learned my addr 172.21.15.138:0/1451638475 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 -- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30102920 msgr2=0x7f5a3019aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30102920 0x7f5a3019aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 -- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108cb0 msgr2=0x7f5a301a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108cb0 0x7f5a301a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2ea00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a301a10a0 con 0x7f5a30108330 2024-08-20T01:07:11.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2fe00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a30108cb0 0x7f5a301a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.245+0000 7f5a2f400640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30102920 0x7f5a3019aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a2ea00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 0x7f5a3019b530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005ab0 tx=0x7f5a1c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c03f070 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c031b60 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a301a1330 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c035030 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.246+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a301a7a50 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.247+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a300635d0 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.250+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5a1c042070 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a27e00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a140b3d60 0x7f5a140b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5a1c104fe0 con 0x7f5a30108330 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a2f400640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a140b3d60 0x7f5a140b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a27e00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] conn(0x7f5a140bf2b0 0x7f5a140c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a140c1e20 con 0x7f5a140bf2b0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.251+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c105390 con 0x7f5a30108330 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.252+0000 7f5a2fe00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] conn(0x7f5a140bf2b0 0x7f5a140c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.252+0000 7f5a2f400640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a140b3d60 0x7f5a140b6220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009fc0 tx=0x7f5a20005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.252+0000 7f5a2fe00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] conn(0x7f5a140bf2b0 0x7f5a140c1750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.253+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== osd.4 v2:172.21.15.142:6808/2294964237 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a140c1e20 con 0x7f5a140bf2b0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.267+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 --> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5a30063a50 con 0x7f5a140bf2b0 2024-08-20T01:07:11.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a27e00640 1 -- 172.21.15.138:0/1451638475 <== osd.4 v2:172.21.15.142:6808/2294964237 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5a30063a50 con 0x7f5a140bf2b0 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] conn(0x7f5a140bf2b0 msgr2=0x7f5a140c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] conn(0x7f5a140bf2b0 0x7f5a140c1750 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a140b3d60 msgr2=0x7f5a140b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a140b3d60 0x7f5a140b6220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009fc0 tx=0x7f5a20005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 msgr2=0x7f5a3019b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 --2- 172.21.15.138:0/1451638475 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5a30108330 0x7f5a3019b530 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c005ab0 tx=0x7f5a1c003c10 comp rx=0 tx=0).stop 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a2fe00640 1 -- 172.21.15.138:0/1451638475 reap_dead start 2024-08-20T01:07:11.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 shutdown_connections 2024-08-20T01:07:11.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.268+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 >> 172.21.15.138:0/1451638475 conn(0x7f5a300fe350 msgr2=0x7f5a301048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:11.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.269+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 shutdown_connections 2024-08-20T01:07:11.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.269+0000 7f5a34c00640 1 -- 172.21.15.138:0/1451638475 wait complete. 2024-08-20T01:07:11.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 -- 172.21.15.138:0/3949705603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb0104cb0 msgr2=0x7f4bb0105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 --2- 172.21.15.138:0/3949705603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb0104cb0 0x7f4bb0105090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009900 tx=0x7f4ba402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 -- 172.21.15.138:0/3949705603 shutdown_connections 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 --2- 172.21.15.138:0/3949705603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb010b040 0x7f4bb0111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 --2- 172.21.15.138:0/3949705603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb010a6c0 0x7f4bb010ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 --2- 172.21.15.138:0/3949705603 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb0104cb0 0x7f4bb0105090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 -- 172.21.15.138:0/3949705603 >> 172.21.15.138:0/3949705603 conn(0x7f4bb0100720 msgr2=0x7f4bb0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 -- 172.21.15.138:0/3949705603 shutdown_connections 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.702+0000 7f4bb8400640 1 -- 172.21.15.138:0/3949705603 wait complete. 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 Processor -- start 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 -- start start 2024-08-20T01:07:11.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 0x7f4bb019b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb010a6c0 0x7f4bb019b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb010b040 0x7f4bb01a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0114d50 con 0x7f4bb0104cb0 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0114bd0 con 0x7f4bb010a6c0 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb8400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb0114ed0 con 0x7f4bb010b040 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 0x7f4bb019b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb5a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 0x7f4bb019b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54918/0 (socket says 172.21.15.138:54918) 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.703+0000 7f4bb5a00640 1 -- 172.21.15.138:0/470583155 learned_addr learned my addr 172.21.15.138:0/470583155 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb6400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb010b040 0x7f4bb01a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5000640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb010a6c0 0x7f4bb019b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 -- 172.21.15.138:0/470583155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb010b040 msgr2=0x7f4bb01a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4bb010b040 0x7f4bb01a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 -- 172.21.15.138:0/470583155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb010a6c0 msgr2=0x7f4bb019b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4bb010a6c0 0x7f4bb019b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb01a12a0 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.704+0000 7f4bb5a00640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 0x7f4bb019b1f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005ab0 tx=0x7f4ba4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.705+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba403f070 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.705+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ba4031b60 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.705+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb01a1530 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.705+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4035030 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.705+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb01a7c50 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.706+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bb00635d0 con 0x7f4bb0104cb0 2024-08-20T01:07:11.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.708+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f4ba4042070 con 0x7f4bb0104cb0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.709+0000 7f4bae000640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b980b3cb0 0x7f4b980b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.709+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4ba4104fe0 con 0x7f4bb0104cb0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bb5000640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b980b3cb0 0x7f4b980b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bae000640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] conn(0x7f4b980bf200 0x7f4b980c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b980c1d70 con 0x7f4b980bf200 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f4ba4105430 con 0x7f4bb0104cb0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bb6400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] conn(0x7f4b980bf200 0x7f4b980c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.710+0000 7f4bb5000640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b980b3cb0 0x7f4b980b6170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0103c30 tx=0x7f4b9c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.711+0000 7f4bb6400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] conn(0x7f4b980bf200 0x7f4b980c16a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:11.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.711+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== osd.9 v2:172.21.15.138:6826/1914094697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b980c1d70 con 0x7f4b980bf200 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 --> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4bb0063a50 con 0x7f4b980bf200 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bae000640 1 -- 172.21.15.138:0/470583155 <== osd.9 v2:172.21.15.138:6826/1914094697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4bb0063a50 con 0x7f4b980bf200 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] conn(0x7f4b980bf200 msgr2=0x7f4b980c16a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bb8400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] conn(0x7f4b980bf200 0x7f4b980c16a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b980b3cb0 msgr2=0x7f4b980b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.726+0000 7f4bb8400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b980b3cb0 0x7f4b980b6170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0103c30 tx=0x7f4b9c005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 msgr2=0x7f4bb019b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 --2- 172.21.15.138:0/470583155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bb0104cb0 0x7f4bb019b1f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4005ab0 tx=0x7f4ba4003c10 comp rx=0 tx=0).stop 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb6400640 1 -- 172.21.15.138:0/470583155 reap_dead start 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 shutdown_connections 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 >> 172.21.15.138:0/470583155 conn(0x7f4bb0100720 msgr2=0x7f4bb0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 shutdown_connections 2024-08-20T01:07:11.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:11.727+0000 7f4bb8400640 1 -- 172.21.15.138:0/470583155 wait complete. 2024-08-20T01:07:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:11 smithi167 ceph-mon[21965]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-08-20T01:07:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:11 smithi142 ceph-mon[21799]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-08-20T01:07:12.034 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:11 smithi138 ceph-mon[18076]: pgmap v334: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 7 op/s 2024-08-20T01:07:12.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 -- 172.21.15.138:0/2353147640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc104cb0 msgr2=0x7f91fc105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 --2- 172.21.15.138:0/2353147640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc104cb0 0x7f91fc105090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009900 tx=0x7f91f002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 -- 172.21.15.138:0/2353147640 shutdown_connections 2024-08-20T01:07:12.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 --2- 172.21.15.138:0/2353147640 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc10b040 0x7f91fc111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 --2- 172.21.15.138:0/2353147640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91fc10a6c0 0x7f91fc10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 --2- 172.21.15.138:0/2353147640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc104cb0 0x7f91fc105090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 -- 172.21.15.138:0/2353147640 >> 172.21.15.138:0/2353147640 conn(0x7f91fc100720 msgr2=0x7f91fc102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 -- 172.21.15.138:0/2353147640 shutdown_connections 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.681+0000 7f9204800640 1 -- 172.21.15.138:0/2353147640 wait complete. 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 Processor -- start 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 -- start start 2024-08-20T01:07:12.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 0x7f91fc19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91fc10a6c0 0x7f91fc19b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc10b040 0x7f91fc1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc114d50 con 0x7f91fc10b040 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc114bd0 con 0x7f91fc10a6c0 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.682+0000 7f9204800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f91fc114ed0 con 0x7f91fc104cb0 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 0x7f91fc19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 0x7f91fc19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37348/0 (socket says 172.21.15.138:37348) 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 -- 172.21.15.138:0/846660782 learned_addr learned my addr 172.21.15.138:0/846660782 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9202800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc10b040 0x7f91fc1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201400640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91fc10a6c0 0x7f91fc19b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 -- 172.21.15.138:0/846660782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91fc10a6c0 msgr2=0x7f91fc19b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91fc10a6c0 0x7f91fc19b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 -- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc10b040 msgr2=0x7f91fc1a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91fc10b040 0x7f91fc1a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91fc1a1220 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.683+0000 7f9201e00640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 0x7f91fc19b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ab0 tx=0x7f91f0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.684+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f003f070 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.684+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91fc1a14b0 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.684+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91f0031b60 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.684+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f0035030 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.684+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91fc1a7bd0 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.685+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f91fc063610 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.687+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f91f0042070 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f91f6a00640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91e40b3d30 0x7f91e40b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f91f0104f50 con 0x7f91fc104cb0 2024-08-20T01:07:12.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f9201400640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91e40b3d30 0x7f91e40b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f91f6a00640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] conn(0x7f91e40bf280 0x7f91e40c1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91e40c1df0 con 0x7f91e40bf280 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.689+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f91f0105300 con 0x7f91fc104cb0 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.690+0000 7f9202800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] conn(0x7f91e40bf280 0x7f91e40c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.690+0000 7f9201400640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91e40b3d30 0x7f91e40b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0095f0 tx=0x7f91ec005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.690+0000 7f9202800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] conn(0x7f91e40bf280 0x7f91e40c1720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.691+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== osd.11 v2:172.21.15.167:6824/3808767407 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f91e40c1df0 con 0x7f91e40bf280 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.706+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 --> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f91fc063a10 con 0x7f91e40bf280 2024-08-20T01:07:12.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.706+0000 7f91f6a00640 1 -- 172.21.15.138:0/846660782 <== osd.11 v2:172.21.15.167:6824/3808767407 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f91fc063a10 con 0x7f91e40bf280 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] conn(0x7f91e40bf280 msgr2=0x7f91e40c1720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] conn(0x7f91e40bf280 0x7f91e40c1720 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91e40b3d30 msgr2=0x7f91e40b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91e40b3d30 0x7f91e40b61f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91ec0095f0 tx=0x7f91ec005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 msgr2=0x7f91fc19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 --2- 172.21.15.138:0/846660782 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f91fc104cb0 0x7f91fc19b170 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005ab0 tx=0x7f91f0003c10 comp rx=0 tx=0).stop 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9202800640 1 -- 172.21.15.138:0/846660782 reap_dead start 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 shutdown_connections 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 >> 172.21.15.138:0/846660782 conn(0x7f91fc100720 msgr2=0x7f91fc102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 shutdown_connections 2024-08-20T01:07:12.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.707+0000 7f9204800640 1 -- 172.21.15.138:0/846660782 wait complete. 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 -- 172.21.15.138:0/639860174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 msgr2=0x7f44d8102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 --2- 172.21.15.138:0/639860174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8102d00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f44cc009900 tx=0x7f44cc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 -- 172.21.15.138:0/639860174 shutdown_connections 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 --2- 172.21.15.138:0/639860174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44d8108cb0 0x7f44d810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 --2- 172.21.15.138:0/639860174 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d8108330 0x7f44d8108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 --2- 172.21.15.138:0/639860174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8102d00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 -- 172.21.15.138:0/639860174 >> 172.21.15.138:0/639860174 conn(0x7f44d80fe350 msgr2=0x7f44d8100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 -- 172.21.15.138:0/639860174 shutdown_connections 2024-08-20T01:07:13.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.394+0000 7f44de200640 1 -- 172.21.15.138:0/639860174 wait complete. 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.395+0000 7f44de200640 1 Processor -- start 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.395+0000 7f44de200640 1 -- start start 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d8108330 0x7f44d8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44d8108cb0 0x7f44d819e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8112970 con 0x7f44d8108cb0 2024-08-20T01:07:13.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44d81127f0 con 0x7f44d8102920 2024-08-20T01:07:13.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44de200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44d8112af0 con 0x7f44d8108330 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44dd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44d8108cb0 0x7f44d819e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34462/0 (socket says 172.21.15.138:34462) 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44dce00640 1 -- 172.21.15.138:0/2532816746 learned_addr learned my addr 172.21.15.138:0/2532816746 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.396+0000 7f44d7e00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d8108330 0x7f44d8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 -- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d8108330 msgr2=0x7f44d8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f44d8108330 0x7f44d8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 -- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44d8108cb0 msgr2=0x7f44d819e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44d8108cb0 0x7f44d819e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d819edf0 con 0x7f44d8102920 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44dce00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8198d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005940 tx=0x7f44cc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc03f030 con 0x7f44d8102920 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d819f080 con 0x7f44d8102920 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.397+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44cc0044c0 con 0x7f44d8102920 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.398+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44cc035070 con 0x7f44d8102920 2024-08-20T01:07:13.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.398+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d81a57a0 con 0x7f44d8102920 2024-08-20T01:07:13.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.398+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44d8105c40 con 0x7f44d8102920 2024-08-20T01:07:13.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.402+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f44cc042030 con 0x7f44d8102920 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.403+0000 7f44d5600640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44c00b3de0 0x7f44c00b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f44cc104c20 con 0x7f44d8102920 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d7e00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44c00b3de0 0x7f44c00b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d5600640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] conn(0x7f44c00bf330 0x7f44c00c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44cc037ea0 con 0x7f44c00bf330 2024-08-20T01:07:13.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f44cc104fd0 con 0x7f44d8102920 2024-08-20T01:07:13.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44d7e00640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44c00b3de0 0x7f44c00b62a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44d8101840 tx=0x7f44c4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:13.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.404+0000 7f44dd800640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] conn(0x7f44c00bf330 0x7f44c00c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:13.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.405+0000 7f44dd800640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] conn(0x7f44c00bf330 0x7f44c00c17d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:13.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.406+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== osd.8 v2:172.21.15.167:6816/3511497686 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f44cc037ea0 con 0x7f44c00bf330 2024-08-20T01:07:13.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.420+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 --> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44d8103bd0 con 0x7f44c00bf330 2024-08-20T01:07:13.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44d5600640 1 -- 172.21.15.138:0/2532816746 <== osd.8 v2:172.21.15.167:6816/3511497686 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f44d8103bd0 con 0x7f44c00bf330 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] conn(0x7f44c00bf330 msgr2=0x7f44c00c17d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] conn(0x7f44c00bf330 0x7f44c00c17d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44c00b3de0 msgr2=0x7f44c00b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44c00b3de0 0x7f44c00b62a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f44d8101840 tx=0x7f44c4005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 msgr2=0x7f44d8198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44de200640 1 --2- 172.21.15.138:0/2532816746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44d8102920 0x7f44d8198d40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44cc005940 tx=0x7f44cc0040c0 comp rx=0 tx=0).stop 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.421+0000 7f44dd800640 1 -- 172.21.15.138:0/2532816746 reap_dead start 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.422+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 shutdown_connections 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.422+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 >> 172.21.15.138:0/2532816746 conn(0x7f44d80fe350 msgr2=0x7f44d8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.422+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 shutdown_connections 2024-08-20T01:07:13.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:13.422+0000 7f44de200640 1 -- 172.21.15.138:0/2532816746 wait complete. 2024-08-20T01:07:13.899 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:13 smithi138 ceph-mon[18076]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:13 smithi167 ceph-mon[21965]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:13 smithi142 ceph-mon[21799]: pgmap v335: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:14.404 INFO:teuthology.orchestra.run.smithi138.stdout:304942678051 2024-08-20T01:07:14.404 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-08-20T01:07:15.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.793+0000 7f0d3b200640 1 -- 172.21.15.138:0/2440607865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 msgr2=0x7f0d34102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.793+0000 7f0d3b200640 1 --2- 172.21.15.138:0/2440607865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d34102e90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0d24009900 tx=0x7f0d2402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 -- 172.21.15.138:0/2440607865 shutdown_connections 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 --2- 172.21.15.138:0/2440607865 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d34108e40 0x7f0d3410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 --2- 172.21.15.138:0/2440607865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d341084c0 0x7f0d34108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 --2- 172.21.15.138:0/2440607865 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d34102e90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 -- 172.21.15.138:0/2440607865 >> 172.21.15.138:0/2440607865 conn(0x7f0d340fe500 msgr2=0x7f0d34100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 -- 172.21.15.138:0/2440607865 shutdown_connections 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.794+0000 7f0d3b200640 1 -- 172.21.15.138:0/2440607865 wait complete. 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 Processor -- start 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 -- start start 2024-08-20T01:07:15.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d3419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d341084c0 0x7f0d3419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d34108e40 0x7f0d341a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34112b50 con 0x7f0d34102ab0 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0d341129d0 con 0x7f0d34108e40 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d3b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0d34112cd0 con 0x7f0d341084c0 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d3419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d3419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54946/0 (socket says 172.21.15.138:54946) 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.795+0000 7f0d33e00640 1 -- 172.21.15.138:0/1447074383 learned_addr learned my addr 172.21.15.138:0/1447074383 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d39200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d34108e40 0x7f0d341a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33400640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d341084c0 0x7f0d3419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 -- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d341084c0 msgr2=0x7f0d3419b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d341084c0 0x7f0d3419b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 -- 172.21.15.138:0/1447074383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d34108e40 msgr2=0x7f0d341a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d34108e40 0x7f0d341a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d341a1290 con 0x7f0d34102ab0 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d39200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d34108e40 0x7f0d341a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d33e00640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d3419b1e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005ab0 tx=0x7f0d24003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d2403f070 con 0x7f0d34102ab0 2024-08-20T01:07:15.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d341a1520 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d24031b60 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d24035030 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.796+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d341a7c40 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.797+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d34100610 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.800+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f0d24042070 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.801+0000 7f0d30c00640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0d180b3d30 0x7f0d180b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.801+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f0d24105100 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.801+0000 7f0d33400640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0d180b3d30 0x7f0d180b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.802+0000 7f0d30c00640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] conn(0x7f0d180bf280 0x7f0d180c1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.802+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d180c1df0 con 0x7f0d180bf280 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.802+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7f0d241054b0 con 0x7f0d34102ab0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.802+0000 7f0d33400640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0d180b3d30 0x7f0d180b61f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d34101ac0 tx=0x7f0d1c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:15.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.802+0000 7f0d39200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] conn(0x7f0d180bf280 0x7f0d180c1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.803+0000 7f0d39200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] conn(0x7f0d180bf280 0x7f0d180c1720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.803+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== osd.2 v2:172.21.15.167:6800/3518437214 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d180c1df0 con 0x7f0d180bf280 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.818+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 --> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d34103d60 con 0x7f0d180bf280 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d30c00640 1 -- 172.21.15.138:0/1447074383 <== osd.2 v2:172.21.15.167:6800/3518437214 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d34103d60 con 0x7f0d180bf280 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] conn(0x7f0d180bf280 msgr2=0x7f0d180c1720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] conn(0x7f0d180bf280 0x7f0d180c1720 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0d180b3d30 msgr2=0x7f0d180b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0d180b3d30 0x7f0d180b61f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0d34101ac0 tx=0x7f0d1c009500 comp rx=0 tx=0).stop 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 msgr2=0x7f0d3419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 --2- 172.21.15.138:0/1447074383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d34102ab0 0x7f0d3419b1e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0d24005ab0 tx=0x7f0d24003c10 comp rx=0 tx=0).stop 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d39200640 1 -- 172.21.15.138:0/1447074383 reap_dead start 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 shutdown_connections 2024-08-20T01:07:15.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.819+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 >> 172.21.15.138:0/1447074383 conn(0x7f0d340fe500 msgr2=0x7f0d340ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:15.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.820+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 shutdown_connections 2024-08-20T01:07:15.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:12.820+0000 7f0d3b200640 1 -- 172.21.15.138:0/1447074383 wait complete. 2024-08-20T01:07:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:15 smithi167 ceph-mon[21965]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:15 smithi142 ceph-mon[21799]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:15 smithi138 ceph-mon[18076]: pgmap v336: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-08-20T01:07:16.254 INFO:teuthology.orchestra.run.smithi138.stdout:137438953544 2024-08-20T01:07:16.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-08-20T01:07:16.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:16 smithi138 ceph-mon[18076]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:16 smithi167 ceph-mon[21965]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:16.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:16 smithi142 ceph-mon[21799]: pgmap v337: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:17.385 INFO:teuthology.orchestra.run.smithi138.stdout:279172874280 2024-08-20T01:07:17.386 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-08-20T01:07:18.250 INFO:teuthology.orchestra.run.smithi138.stdout:257698037805 2024-08-20T01:07:18.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-08-20T01:07:18.402 INFO:teuthology.orchestra.run.smithi138.stdout:81604378707 2024-08-20T01:07:18.402 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-08-20T01:07:18.955 INFO:teuthology.orchestra.run.smithi138.stdout:158913790018 2024-08-20T01:07:18.956 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-08-20T01:07:19.105 INFO:teuthology.orchestra.run.smithi138.stdout:231928234035 2024-08-20T01:07:19.105 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-08-20T01:07:19.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 -- 172.21.15.138:0/3407624488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 msgr2=0x7fc7040755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 --2- 172.21.15.138:0/3407624488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc7040755f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009900 tx=0x7fc6f402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 -- 172.21.15.138:0/3407624488 shutdown_connections 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 --2- 172.21.15.138:0/3407624488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70410ef10 0x7fc704111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 --2- 172.21.15.138:0/3407624488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 0x7fc70410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 --2- 172.21.15.138:0/3407624488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc7040755f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 -- 172.21.15.138:0/3407624488 >> 172.21.15.138:0/3407624488 conn(0x7fc7040fe350 msgr2=0x7fc704100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 -- 172.21.15.138:0/3407624488 shutdown_connections 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.322+0000 7fc708c00640 1 -- 172.21.15.138:0/3407624488 wait complete. 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.323+0000 7fc708c00640 1 Processor -- start 2024-08-20T01:07:19.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 -- start start 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc70419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 0x7fc70419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70410ef10 0x7fc7041a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc704114ab0 con 0x7fc704075b30 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc704114930 con 0x7fc70410ef10 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc708c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc704114c30 con 0x7fc704075210 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc703400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc70419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc702a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 0x7fc70419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc703400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc70419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37778/0 (socket says 172.21.15.138:37778) 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc703400640 1 -- 172.21.15.138:0/2474983177 learned_addr learned my addr 172.21.15.138:0/2474983177 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.324+0000 7fc703e00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70410ef10 0x7fc7041a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 -- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70410ef10 msgr2=0x7fc7041a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc70410ef10 0x7fc7041a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 -- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 msgr2=0x7fc70419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 0x7fc70419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7041a0ff0 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc702a00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc704075b30 0x7fc70419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.325+0000 7fc703400640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc70419af40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005940 tx=0x7fc6f4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.326+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f403f030 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.326+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6f4003aa0 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.326+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4035070 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.326+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7041a1280 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.326+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7041a79a0 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.327+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc704100940 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.331+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fc6f4042030 con 0x7fc704075210 2024-08-20T01:07:19.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.332+0000 7fc6fbe00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6e80b3d60 0x7fc6e80b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fc6f4104fc0 con 0x7fc704075210 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc702a00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6e80b3d60 0x7fc6e80b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc6fbe00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] conn(0x7fc6e80bf2b0 0x7fc6e80c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc6f403ce50 con 0x7fc6e80bf2b0 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fc6f4105370 con 0x7fc704075210 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc703e00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] conn(0x7fc6e80bf2b0 0x7fc6e80c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.333+0000 7fc702a00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6e80b3d60 0x7fc6e80b6220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00095f0 tx=0x7fc6f0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.334+0000 7fc703e00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] conn(0x7fc6e80bf2b0 0x7fc6e80c1750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.335+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== osd.1 v2:172.21.15.142:6800/1595380391 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc6f403ce50 con 0x7fc6e80bf2b0 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.349+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 --> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc704076720 con 0x7fc6e80bf2b0 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc6fbe00640 1 -- 172.21.15.138:0/2474983177 <== osd.1 v2:172.21.15.142:6800/1595380391 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc704076720 con 0x7fc6e80bf2b0 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] conn(0x7fc6e80bf2b0 msgr2=0x7fc6e80c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] conn(0x7fc6e80bf2b0 0x7fc6e80c1750 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6e80b3d60 msgr2=0x7fc6e80b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6e80b3d60 0x7fc6e80b6220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6f00095f0 tx=0x7fc6f0005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 msgr2=0x7fc70419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc708c00640 1 --2- 172.21.15.138:0/2474983177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc704075210 0x7fc70419af40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4005940 tx=0x7fc6f4004270 comp rx=0 tx=0).stop 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.350+0000 7fc703e00640 1 -- 172.21.15.138:0/2474983177 reap_dead start 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.351+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 shutdown_connections 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.351+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 >> 172.21.15.138:0/2474983177 conn(0x7fc7040fe350 msgr2=0x7fc7040fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.351+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 shutdown_connections 2024-08-20T01:07:19.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:19.351+0000 7fc708c00640 1 -- 172.21.15.138:0/2474983177 wait complete. 2024-08-20T01:07:19.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 -- 172.21.15.138:0/2235728269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108330 msgr2=0x7fdc78108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 --2- 172.21.15.138:0/2235728269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108330 0x7fdc78108770 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009900 tx=0x7fdc6402a410 comp rx=0 tx=0).stop 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 -- 172.21.15.138:0/2235728269 shutdown_connections 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 --2- 172.21.15.138:0/2235728269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc78108cb0 0x7fdc7810f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 --2- 172.21.15.138:0/2235728269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108330 0x7fdc78108770 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 --2- 172.21.15.138:0/2235728269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78102920 0x7fdc78102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 -- 172.21.15.138:0/2235728269 >> 172.21.15.138:0/2235728269 conn(0x7fdc780fe350 msgr2=0x7fdc78100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 -- 172.21.15.138:0/2235728269 shutdown_connections 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.222+0000 7fdc7e000640 1 -- 172.21.15.138:0/2235728269 wait complete. 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.223+0000 7fdc7e000640 1 Processor -- start 2024-08-20T01:07:19.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.223+0000 7fdc7e000640 1 -- start start 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc78102920 0x7fdc78198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78108330 0x7fdc78199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 0x7fdc7819e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78112980 con 0x7fdc78108cb0 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78112800 con 0x7fdc78108330 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7e000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78112b00 con 0x7fdc78102920 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 0x7fdc7819e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 0x7fdc7819e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54990/0 (socket says 172.21.15.138:54990) 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc7d600640 1 -- 172.21.15.138:0/1512277496 learned_addr learned my addr 172.21.15.138:0/1512277496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.224+0000 7fdc77e00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78108330 0x7fdc78199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 -- 172.21.15.138:0/1512277496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc78102920 msgr2=0x7fdc78198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc78102920 0x7fdc78198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 -- 172.21.15.138:0/1512277496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78108330 msgr2=0x7fdc78199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78108330 0x7fdc78199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7819edc0 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc77e00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc78108330 0x7fdc78199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7d600640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 0x7fdc7819e5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000eb10 tx=0x7fdc7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70004b20 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7819f0b0 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc70004cc0 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70010690 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.225+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc781a5940 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.226+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc78105c60 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.228+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fdc70005000 con 0x7fdc78108cb0 2024-08-20T01:07:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc75600640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc600b3de0 0x7fdc600b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fdc700e4cc0 con 0x7fdc78108cb0 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc7cc00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc600b3de0 0x7fdc600b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc75600640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] conn(0x7fdc600bf330 0x7fdc600c17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc70015e80 con 0x7fdc600bf330 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7fdc700e5070 con 0x7fdc78108cb0 2024-08-20T01:07:19.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.230+0000 7fdc77e00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] conn(0x7fdc600bf330 0x7fdc600c17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.231+0000 7fdc7cc00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc600b3de0 0x7fdc600b62a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00ba80 tx=0x7fdc6c005ed0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.231+0000 7fdc77e00640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] conn(0x7fdc600bf330 0x7fdc600c17d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.232+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== osd.6 v2:172.21.15.138:6818/2049480484 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc70015e80 con 0x7fdc600bf330 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.246+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 --> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdc78103bd0 con 0x7fdc600bf330 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.246+0000 7fdc75600640 1 -- 172.21.15.138:0/1512277496 <== osd.6 v2:172.21.15.138:6818/2049480484 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdc78103bd0 con 0x7fdc600bf330 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] conn(0x7fdc600bf330 msgr2=0x7fdc600c17d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] conn(0x7fdc600bf330 0x7fdc600c17d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc600b3de0 msgr2=0x7fdc600b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc600b3de0 0x7fdc600b62a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00ba80 tx=0x7fdc6c005ed0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 msgr2=0x7fdc7819e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 --2- 172.21.15.138:0/1512277496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc78108cb0 0x7fdc7819e5f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000eb10 tx=0x7fdc7000c2d0 comp rx=0 tx=0).stop 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7d600640 1 -- 172.21.15.138:0/1512277496 reap_dead start 2024-08-20T01:07:19.853 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 shutdown_connections 2024-08-20T01:07:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 >> 172.21.15.138:0/1512277496 conn(0x7fdc780fe350 msgr2=0x7fdc78100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 shutdown_connections 2024-08-20T01:07:19.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:15.247+0000 7fdc7e000640 1 -- 172.21.15.138:0/1512277496 wait complete. 2024-08-20T01:07:19.854 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:19 smithi138 ceph-mon[18076]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:19.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:19 smithi167 ceph-mon[21965]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:19 smithi142 ceph-mon[21799]: pgmap v338: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:20.084 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:20.087 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:20.756 INFO:teuthology.orchestra.run.smithi138.stdout:184683593791 2024-08-20T01:07:20.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-08-20T01:07:21.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 -- 172.21.15.138:0/2091888150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 msgr2=0x7ff40410f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 --2- 172.21.15.138:0/2091888150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff40410f490 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009ca0 tx=0x7ff3fc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 -- 172.21.15.138:0/2091888150 shutdown_connections 2024-08-20T01:07:21.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 --2- 172.21.15.138:0/2091888150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff40410f490 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 --2- 172.21.15.138:0/2091888150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 0x7ff404108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 --2- 172.21.15.138:0/2091888150 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404102900 0x7ff404102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 -- 172.21.15.138:0/2091888150 >> 172.21.15.138:0/2091888150 conn(0x7ff4040fe390 msgr2=0x7ff4041007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 -- 172.21.15.138:0/2091888150 shutdown_connections 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.380+0000 7ff40ac00640 1 -- 172.21.15.138:0/2091888150 wait complete. 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.381+0000 7ff40ac00640 1 Processor -- start 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.381+0000 7ff40ac00640 1 -- start start 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404102900 0x7ff404076c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 0x7ff4040751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff404075800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112960 con 0x7ff404108c90 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4041127e0 con 0x7ff404108310 2024-08-20T01:07:21.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff404112ae0 con 0x7ff404102900 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff404075800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff404075800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43642/0 (socket says 172.21.15.138:43642) 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff40a200640 1 -- 172.21.15.138:0/1978654499 learned_addr learned my addr 172.21.15.138:0/1978654499 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff408e00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 0x7ff4040751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.382+0000 7ff409800640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404102900 0x7ff404076c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 -- 172.21.15.138:0/1978654499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404102900 msgr2=0x7ff404076c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff404102900 0x7ff404076c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 -- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 msgr2=0x7ff4040751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 0x7ff4040751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4041ab670 con 0x7ff404108c90 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff408e00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff404108310 0x7ff4040751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40a200640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff404075800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009430 tx=0x7ff3fc0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:21.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc003b70 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3fc003d10 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc02fa30 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.383+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4041ab840 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.384+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4041abca0 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.384+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4040ff450 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.387+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff3fc02fbd0 con 0x7ff404108c90 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.388+0000 7ff402000640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3ec0b3d60 0x7ff3ec0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:21.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.388+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff3fc105850 con 0x7ff404108c90 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.388+0000 7ff409800640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3ec0b3d60 0x7ff3ec0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff402000640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] conn(0x7ff3ec0bf2b0 0x7ff3ec0c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3fc044e50 con 0x7ff3ec0bf2b0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_get_version_reply(handle=1 version=88) ==== 24+0+0 (secure 0 0 0) 0x7ff3fc105c00 con 0x7ff404108c90 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff408e00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] conn(0x7ff3ec0bf2b0 0x7ff3ec0c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff409800640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3ec0b3d60 0x7ff3ec0b6220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bbe0 tx=0x7ff3f800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.389+0000 7ff408e00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] conn(0x7ff3ec0bf2b0 0x7ff3ec0c1750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.390+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== osd.7 v2:172.21.15.142:6816/2538307187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3fc044e50 con 0x7ff3ec0bf2b0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.404+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 --> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff404103bb0 con 0x7ff3ec0bf2b0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.405+0000 7ff402000640 1 -- 172.21.15.138:0/1978654499 <== osd.7 v2:172.21.15.142:6816/2538307187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff404103bb0 con 0x7ff3ec0bf2b0 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] conn(0x7ff3ec0bf2b0 msgr2=0x7ff3ec0c1750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] conn(0x7ff3ec0bf2b0 0x7ff3ec0c1750 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3ec0b3d60 msgr2=0x7ff3ec0b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff3ec0b3d60 0x7ff3ec0b6220 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bbe0 tx=0x7ff3f800b390 comp rx=0 tx=0).stop 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 msgr2=0x7ff404075800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 --2- 172.21.15.138:0/1978654499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff404108c90 0x7ff404075800 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009430 tx=0x7ff3fc0317c0 comp rx=0 tx=0).stop 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40a200640 1 -- 172.21.15.138:0/1978654499 reap_dead start 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 shutdown_connections 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 >> 172.21.15.138:0/1978654499 conn(0x7ff4040fe390 msgr2=0x7ff4040fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 shutdown_connections 2024-08-20T01:07:21.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:21.406+0000 7ff40ac00640 1 -- 172.21.15.138:0/1978654499 wait complete. 2024-08-20T01:07:21.808 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:21.809 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:21 smithi167 ceph-mon[21965]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:21 smithi142 ceph-mon[21799]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:22.022 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:21 smithi138 ceph-mon[18076]: pgmap v339: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 -- 172.21.15.138:0/3718868273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c1134f0 msgr2=0x7f5c7c1158e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 --2- 172.21.15.138:0/3718868273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1158e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5c74009ca0 tx=0x7f5c7402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 -- 172.21.15.138:0/3718868273 shutdown_connections 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 --2- 172.21.15.138:0/3718868273 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1158e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 --2- 172.21.15.138:0/3718868273 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c077d20 0x7f5c7c112fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 --2- 172.21.15.138:0/3718868273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c0777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 -- 172.21.15.138:0/3718868273 >> 172.21.15.138:0/3718868273 conn(0x7f5c7c100550 msgr2=0x7f5c7c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:23.120 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 -- 172.21.15.138:0/3718868273 shutdown_connections 2024-08-20T01:07:23.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.120+0000 7f5c7be00640 1 -- 172.21.15.138:0/3718868273 wait complete. 2024-08-20T01:07:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.121+0000 7f5c7be00640 1 Processor -- start 2024-08-20T01:07:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.121+0000 7f5c7be00640 1 -- start start 2024-08-20T01:07:23.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c077d20 0x7f5c7c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1a4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c118f50 con 0x7f5c7c077d20 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c118dd0 con 0x7f5c7c077400 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7be00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5c7c1190d0 con 0x7f5c7c1134f0 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39076/0 (socket says 172.21.15.138:39076) 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 -- 172.21.15.138:0/1189250518 learned_addr learned my addr 172.21.15.138:0/1189250518 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c73e00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c077d20 0x7f5c7c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7b400640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1a4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 -- 172.21.15.138:0/1189250518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 msgr2=0x7f5c7c1a4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1a4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 -- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c077d20 msgr2=0x7f5c7c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c077d20 0x7f5c7c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.122+0000 7f5c7aa00640 1 -- 172.21.15.138:0/1189250518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7c1a5350 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c7b400640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c7aa00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9f0 tx=0x7f5c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c00cd80 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7c1a5640 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c0108e0 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c018980 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.123+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7c1abed0 con 0x7f5c7c077400 2024-08-20T01:07:23.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.124+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c7c0777e0 con 0x7f5c7c077400 2024-08-20T01:07:23.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.127+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f5c6c010430 con 0x7f5c7c077400 2024-08-20T01:07:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.128+0000 7f5c80a00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 0x7f5c600b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.128+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5c6c0e4ac0 con 0x7f5c7c077400 2024-08-20T01:07:23.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.128+0000 7f5c73e00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 0x7f5c600b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:23.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.129+0000 7f5c73e00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 0x7f5c600b62a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c079d40 tx=0x7f5c64005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:23.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.130+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5c6c0abae0 con 0x7f5c7c077400 2024-08-20T01:07:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.329+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5c7c1ac0a0 con 0x7f5c7c077400 2024-08-20T01:07:23.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.330+0000 7f5c80a00640 1 -- 172.21.15.138:0/1189250518 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c6c0ab3e0 con 0x7f5c7c077400 2024-08-20T01:07:23.330 INFO:teuthology.orchestra.run.smithi138.stdout:38654705758 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.332+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 msgr2=0x7f5c600b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.332+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 0x7f5c600b62a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c079d40 tx=0x7f5c64005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.332+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 msgr2=0x7f5c7c19f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.332+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00e9f0 tx=0x7f5c6c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 shutdown_connections 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5c600b3de0 0x7f5c600b62a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5c7c1134f0 0x7f5c7c1a4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c7c077d20 0x7f5c7c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 --2- 172.21.15.138:0/1189250518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c7c077400 0x7f5c7c19f2a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 >> 172.21.15.138:0/1189250518 conn(0x7f5c7c100550 msgr2=0x7f5c7c102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 shutdown_connections 2024-08-20T01:07:23.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:23.333+0000 7f5c7be00640 1 -- 172.21.15.138:0/1189250518 wait complete. 2024-08-20T01:07:23.792 INFO:teuthology.orchestra.run.smithi138.stdout:206158430265 2024-08-20T01:07:23.792 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-08-20T01:07:23.841 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:23 smithi138 ceph-mon[18076]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-20T01:07:23.841 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:23 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1189250518' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T01:07:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:23 smithi167 ceph-mon[21965]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-20T01:07:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:23 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1189250518' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T01:07:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:23 smithi142 ceph-mon[21799]: pgmap v340: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 5 op/s 2024-08-20T01:07:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:23 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1189250518' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-08-20T01:07:24.379 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705755 got 38654705758 for osd.0 2024-08-20T01:07:24.379 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:24.604 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:24 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/80361140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T01:07:24.604 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:24 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:07:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:24 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/80361140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T01:07:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:24 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:07:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:24 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/80361140' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-08-20T01:07:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:24 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:07:25.060 INFO:teuthology.orchestra.run.smithi138.stdout:60129542232 2024-08-20T01:07:25.060 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-08-20T01:07:25.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 -- 172.21.15.138:0/2551854310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f4104cb0 msgr2=0x7fc6f4105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 --2- 172.21.15.138:0/2551854310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f4104cb0 0x7fc6f4105090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009900 tx=0x7fc6e402a410 comp rx=0 tx=0).stop 2024-08-20T01:07:25.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 -- 172.21.15.138:0/2551854310 shutdown_connections 2024-08-20T01:07:25.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 --2- 172.21.15.138:0/2551854310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f410b040 0x7fc6f4111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 --2- 172.21.15.138:0/2551854310 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410a6c0 0x7fc6f410ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 --2- 172.21.15.138:0/2551854310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f4104cb0 0x7fc6f4105090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.720+0000 7fc6fba00640 1 -- 172.21.15.138:0/2551854310 >> 172.21.15.138:0/2551854310 conn(0x7fc6f4100720 msgr2=0x7fc6f4102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.721+0000 7fc6fba00640 1 -- 172.21.15.138:0/2551854310 shutdown_connections 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.721+0000 7fc6fba00640 1 -- 172.21.15.138:0/2551854310 wait complete. 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.721+0000 7fc6fba00640 1 Processor -- start 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 -- start start 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f4104cb0 0x7fc6f419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410b040 0x7fc6f41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4114d50 con 0x7fc6f4104cb0 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4114bd0 con 0x7fc6f410a6c0 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6fba00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4114ed0 con 0x7fc6f410b040 2024-08-20T01:07:25.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6f9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f4104cb0 0x7fc6f419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39102/0 (socket says 172.21.15.138:39102) 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6ebe00640 1 -- 172.21.15.138:0/4173769900 learned_addr learned my addr 172.21.15.138:0/4173769900 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.722+0000 7fc6f9a00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410b040 0x7fc6f41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 -- 172.21.15.138:0/4173769900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410b040 msgr2=0x7fc6f41a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410b040 0x7fc6f41a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 -- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f4104cb0 msgr2=0x7fc6f419b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f4104cb0 0x7fc6f419b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 -- 172.21.15.138:0/4173769900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f41a12b0 con 0x7fc6f410a6c0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6ebe00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00e9f0 tx=0x7fc6dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00cd80 con 0x7fc6f410a6c0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6dc0054d0 con 0x7fc6f410a6c0 2024-08-20T01:07:25.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc010690 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41a15a0 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.723+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41a7c60 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.725+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6f4105090 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.726+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fc6dc004e90 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.728+0000 7fc6f2000640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 0x7fc6d80b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stdout:279172874282 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.728+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fc6dc0e4640 con 0x7fc6f410a6c0 2024-08-20T01:07:25.134 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.728+0000 7fc6f9000640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 0x7fc6d80b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.729+0000 7fc6f9000640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 0x7fc6d80b61f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005ab0 tx=0x7fc6e4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.731+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc6dc0ab660 con 0x7fc6f410a6c0 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.926+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc6f410c890 con 0x7fc6f410a6c0 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.926+0000 7fc6f2000640 1 -- 172.21.15.138:0/4173769900 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc6dc0aaf60 con 0x7fc6f410a6c0 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 msgr2=0x7fc6d80b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 0x7fc6d80b61f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4005ab0 tx=0x7fc6e4005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 msgr2=0x7fc6f419b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc00e9f0 tx=0x7fc6dc00edf0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 shutdown_connections 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc6d80b3d30 0x7fc6d80b61f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc6f410b040 0x7fc6f41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.135 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc6f410a6c0 0x7fc6f419b740 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 --2- 172.21.15.138:0/4173769900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc6f4104cb0 0x7fc6f419b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 >> 172.21.15.138:0/4173769900 conn(0x7fc6f4100720 msgr2=0x7fc6f4102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 shutdown_connections 2024-08-20T01:07:25.136 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.929+0000 7fc6fba00640 1 -- 172.21.15.138:0/4173769900 wait complete. 2024-08-20T01:07:25.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 -- 172.21.15.138:0/1381464406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10b030 msgr2=0x7f3eac111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 --2- 172.21.15.138:0/1381464406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10b030 0x7f3eac111830 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4009be0 tx=0x7f3ea402a6c0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 -- 172.21.15.138:0/1381464406 shutdown_connections 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 --2- 172.21.15.138:0/1381464406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10b030 0x7f3eac111830 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 --2- 172.21.15.138:0/1381464406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac10a6b0 0x7f3eac10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 --2- 172.21.15.138:0/1381464406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac104ca0 0x7f3eac105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 -- 172.21.15.138:0/1381464406 >> 172.21.15.138:0/1381464406 conn(0x7f3eac100730 msgr2=0x7f3eac102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 -- 172.21.15.138:0/1381464406 shutdown_connections 2024-08-20T01:07:25.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.574+0000 7f3eb4200640 1 -- 172.21.15.138:0/1381464406 wait complete. 2024-08-20T01:07:25.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.575+0000 7f3eb4200640 1 Processor -- start 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.575+0000 7f3eb4200640 1 -- start start 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.575+0000 7f3eb4200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb4200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10a6b0 0x7f3eac19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb4200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac10b030 0x7f3eac1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb4200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114cf0 con 0x7f3eac10a6b0 2024-08-20T01:07:25.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb4200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114b70 con 0x7f3eac104ca0 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb4200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3eac114e70 con 0x7f3eac10b030 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10a6b0 0x7f3eac19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39134/0 (socket says 172.21.15.138:39134) 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 -- 172.21.15.138:0/183372666 learned_addr learned my addr 172.21.15.138:0/183372666 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac10b030 0x7f3eac1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 -- 172.21.15.138:0/183372666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac10b030 msgr2=0x7f3eac1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac10b030 0x7f3eac1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 -- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10a6b0 msgr2=0x7f3eac19b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10a6b0 0x7f3eac19b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.576+0000 7f3eb1800640 1 -- 172.21.15.138:0/183372666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eac1a11b0 con 0x7f3eac104ca0 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eb1800640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9c0 tx=0x7f3ea000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000ccc0 con 0x7f3eac104ca0 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eac1a14a0 con 0x7f3eac104ca0 2024-08-20T01:07:25.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea00054d0 con 0x7f3eac104ca0 2024-08-20T01:07:25.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0010430 con 0x7f3eac104ca0 2024-08-20T01:07:25.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.577+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eac1a7b60 con 0x7f3eac104ca0 2024-08-20T01:07:25.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.578+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eac105080 con 0x7f3eac104ca0 2024-08-20T01:07:25.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.580+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f3ea0004f80 con 0x7f3eac104ca0 2024-08-20T01:07:25.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.581+0000 7f3eaa000640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 0x7f3e940b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.582+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3ea00e46c0 con 0x7f3eac104ca0 2024-08-20T01:07:25.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.582+0000 7f3eb0e00640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 0x7f3e940b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.582+0000 7f3eb0e00640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 0x7f3e940b6170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3eac10a2f0 tx=0x7f3e98005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.585+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3ea00ab6e0 con 0x7f3eac104ca0 2024-08-20T01:07:25.709 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:25 smithi138 ceph-mon[18076]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-20T01:07:25.709 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:25 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/4173769900' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T01:07:25.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.804+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3eac02e340 con 0x7f3eac104ca0 2024-08-20T01:07:25.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.804+0000 7f3eaa000640 1 -- 172.21.15.138:0/183372666 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3ea00aafe0 con 0x7f3eac104ca0 2024-08-20T01:07:25.804 INFO:teuthology.orchestra.run.smithi138.stdout:111669149775 2024-08-20T01:07:25.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 msgr2=0x7f3e940b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 0x7f3e940b6170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3eac10a2f0 tx=0x7f3e98005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 msgr2=0x7f3eac19b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000e9c0 tx=0x7f3ea000ee80 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 shutdown_connections 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e940b3cb0 0x7f3e940b6170 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3eac10b030 0x7f3eac1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3eac10a6b0 0x7f3eac19b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 --2- 172.21.15.138:0/183372666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3eac104ca0 0x7f3eac19b100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 >> 172.21.15.138:0/183372666 conn(0x7f3eac100730 msgr2=0x7f3eac102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 shutdown_connections 2024-08-20T01:07:25.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:25.807+0000 7f3eb4200640 1 -- 172.21.15.138:0/183372666 wait complete. 2024-08-20T01:07:25.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 -- 172.21.15.138:0/2720857355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 msgr2=0x7f4530111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 --2- 172.21.15.138:0/2720857355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f4530111840 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4528009ca0 tx=0x7f452802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 -- 172.21.15.138:0/2720857355 shutdown_connections 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 --2- 172.21.15.138:0/2720857355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f4530111840 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 --2- 172.21.15.138:0/2720857355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 0x7f453010ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 --2- 172.21.15.138:0/2720857355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 0x7f4530105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 -- 172.21.15.138:0/2720857355 >> 172.21.15.138:0/2720857355 conn(0x7f4530100720 msgr2=0x7f4530102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 -- 172.21.15.138:0/2720857355 shutdown_connections 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.096+0000 7f4534c00640 1 -- 172.21.15.138:0/2720857355 wait complete. 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 Processor -- start 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 -- start start 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 0x7f453019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 0x7f453019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4530114d00 con 0x7f453010b040 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4530114b80 con 0x7f4530104cb0 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.097+0000 7f4534c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4530114e80 con 0x7f453010a6c0 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43666/0 (socket says 172.21.15.138:43666) 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 -- 172.21.15.138:0/80361140 learned_addr learned my addr 172.21.15.138:0/80361140 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452d400640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 0x7f453019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452de00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 0x7f453019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 -- 172.21.15.138:0/80361140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 msgr2=0x7f453019b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 0x7f453019b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 -- 172.21.15.138:0/80361140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 msgr2=0x7f453019b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 0x7f453019b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.098+0000 7f452e800640 1 -- 172.21.15.138:0/80361140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45301a11c0 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f452e800640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4528005cc0 tx=0x7f452802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528003bf0 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4528003d90 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45301a1450 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452802fbc0 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.099+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301a7b70 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.100+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4530105090 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.102+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f452802fd60 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.103+0000 7f4526a00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 0x7f45140b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.104+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4528105800 con 0x7f453010b040 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.104+0000 7f452de00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 0x7f45140b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:25.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.104+0000 7f452de00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 0x7f45140b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4520005d00 tx=0x7f452000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.108+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f45280cc930 con 0x7f453010b040 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.300+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f453010c890 con 0x7f453010b040 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.300+0000 7f4526a00640 1 -- 172.21.15.138:0/80361140 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f45280098b0 con 0x7f453010b040 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stdout:304942678053 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.302+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 msgr2=0x7f45140b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.302+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 0x7f45140b6170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4520005d00 tx=0x7f452000b410 comp rx=0 tx=0).stop 2024-08-20T01:07:25.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.302+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 msgr2=0x7f45301a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.302+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4528005cc0 tx=0x7f452802f380 comp rx=0 tx=0).stop 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 shutdown_connections 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f45140b3cb0 0x7f45140b6170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f453010b040 0x7f45301a09f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f453010a6c0 0x7f453019b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 --2- 172.21.15.138:0/80361140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4530104cb0 0x7f453019b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 >> 172.21.15.138:0/80361140 conn(0x7f4530100720 msgr2=0x7f4530102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 shutdown_connections 2024-08-20T01:07:25.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:24.303+0000 7f4534c00640 1 -- 172.21.15.138:0/80361140 wait complete. 2024-08-20T01:07:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:25 smithi167 ceph-mon[21965]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-20T01:07:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:25 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/4173769900' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T01:07:25.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:25 smithi142 ceph-mon[21799]: pgmap v341: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 4 op/s 2024-08-20T01:07:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:25 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/4173769900' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-08-20T01:07:26.537 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874280 got 279172874282 for osd.10 2024-08-20T01:07:26.538 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/183372666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:26 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/183372666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:26 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/183372666' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:26 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:27.443 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:27.473 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678051 got 304942678053 for osd.11 2024-08-20T01:07:27.473 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:27.772 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:27 smithi138 ceph-mon[18076]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-08-20T01:07:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:27 smithi167 ceph-mon[21965]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-08-20T01:07:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:27 smithi142 ceph-mon[21799]: pgmap v342: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 -- 172.21.15.138:0/1199668798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 msgr2=0x7ff0bc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 --2- 172.21.15.138:0/1199668798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc108900 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4009900 tx=0x7ff0a402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 -- 172.21.15.138:0/1199668798 shutdown_connections 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 --2- 172.21.15.138:0/1199668798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc108e40 0x7ff0bc10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 --2- 172.21.15.138:0/1199668798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc108900 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 --2- 172.21.15.138:0/1199668798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc102ab0 0x7ff0bc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.030+0000 7ff0c2800640 1 -- 172.21.15.138:0/1199668798 >> 172.21.15.138:0/1199668798 conn(0x7ff0bc0fe500 msgr2=0x7ff0bc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.031+0000 7ff0c2800640 1 -- 172.21.15.138:0/1199668798 shutdown_connections 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.031+0000 7ff0c2800640 1 -- 172.21.15.138:0/1199668798 wait complete. 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.031+0000 7ff0c2800640 1 Processor -- start 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 -- start start 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc102ab0 0x7ff0bc19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc108e40 0x7ff0bc1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc112ae0 con 0x7ff0bc1084c0 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc112960 con 0x7ff0bc102ab0 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0c2800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff0bc112c60 con 0x7ff0bc108e40 2024-08-20T01:07:28.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0baa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42864/0 (socket says 172.21.15.138:42864) 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0baa00640 1 -- 172.21.15.138:0/3950661972 learned_addr learned my addr 172.21.15.138:0/3950661972 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0bb400640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc102ab0 0x7ff0bc19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.032+0000 7ff0bbe00640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc108e40 0x7ff0bc1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 -- 172.21.15.138:0/3950661972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc108e40 msgr2=0x7ff0bc1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc108e40 0x7ff0bc1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 -- 172.21.15.138:0/3950661972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc102ab0 msgr2=0x7ff0bc19b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc102ab0 0x7ff0bc19b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 -- 172.21.15.138:0/3950661972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0bc1a1260 con 0x7ff0bc1084c0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0baa00640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005ab0 tx=0x7ff0a4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a403f070 con 0x7ff0bc1084c0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0bc1a14f0 con 0x7ff0bc1084c0 2024-08-20T01:07:28.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0a4031d60 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.033+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4035030 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.034+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0bc1a7c10 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.035+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0bc102e90 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.037+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7ff0a4042070 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.038+0000 7ff0b3e00640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 0x7ff0a00b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.038+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff0a4104ae0 con 0x7ff0bc1084c0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.038+0000 7ff0bb400640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 0x7ff0a00b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.039+0000 7ff0bb400640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 0x7ff0a00b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005fd0 tx=0x7ff0ac009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:28.086 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.042+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff0a40cb4b0 con 0x7ff0bc1084c0 2024-08-20T01:07:28.137 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.139 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.148 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.153 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.155 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.156 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.289+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7ff0bc10f030 con 0x7ff0bc1084c0 2024-08-20T01:07:28.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.290+0000 7ff0b3e00640 1 -- 172.21.15.138:0/3950661972 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff0a40cfa00 con 0x7ff0bc1084c0 2024-08-20T01:07:28.289 INFO:teuthology.orchestra.run.smithi138.stdout:81604378710 2024-08-20T01:07:28.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 msgr2=0x7ff0a00b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:28.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 0x7ff0a00b6170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac005fd0 tx=0x7ff0ac009760 comp rx=0 tx=0).stop 2024-08-20T01:07:28.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 msgr2=0x7ff0bc19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:28.293 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4005ab0 tx=0x7ff0a4003cd0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 shutdown_connections 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0a00b3cb0 0x7ff0a00b6170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff0bc108e40 0x7ff0bc1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0bc1084c0 0x7ff0bc19b6f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 --2- 172.21.15.138:0/3950661972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0bc102ab0 0x7ff0bc19b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.293+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 >> 172.21.15.138:0/3950661972 conn(0x7ff0bc0fe500 msgr2=0x7ff0bc105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.294+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 shutdown_connections 2024-08-20T01:07:28.294 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:28.294+0000 7ff0c2800640 1 -- 172.21.15.138:0/3950661972 wait complete. 2024-08-20T01:07:28.939 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:28 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3950661972' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T01:07:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:28 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3950661972' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T01:07:28.981 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149771 got 111669149775 for osd.3 2024-08-20T01:07:28.981 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:28.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:28 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3950661972' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-08-20T01:07:29.874 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:29 smithi138 ceph-mon[18076]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:29 smithi167 ceph-mon[21965]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:29 smithi142 ceph-mon[21799]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:30.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.316+0000 7fe2eec00640 1 -- 172.21.15.138:0/4076514152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 msgr2=0x7fe2e8105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.316+0000 7fe2eec00640 1 --2- 172.21.15.138:0/4076514152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e8105080 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8009900 tx=0x7fe2d802a410 comp rx=0 tx=0).stop 2024-08-20T01:07:30.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 -- 172.21.15.138:0/4076514152 shutdown_connections 2024-08-20T01:07:30.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 --2- 172.21.15.138:0/4076514152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810b030 0x7fe2e8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 --2- 172.21.15.138:0/4076514152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810a6b0 0x7fe2e810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 --2- 172.21.15.138:0/4076514152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e8105080 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 -- 172.21.15.138:0/4076514152 >> 172.21.15.138:0/4076514152 conn(0x7fe2e8100730 msgr2=0x7fe2e8102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 -- 172.21.15.138:0/4076514152 shutdown_connections 2024-08-20T01:07:30.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.317+0000 7fe2eec00640 1 -- 172.21.15.138:0/4076514152 wait complete. 2024-08-20T01:07:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 Processor -- start 2024-08-20T01:07:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 -- start start 2024-08-20T01:07:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 0x7fe2e819b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 0x7fe2e81a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8114cf0 con 0x7fe2e8104ca0 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8114b70 con 0x7fe2e810a6b0 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2eec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2e8114e70 con 0x7fe2e810b030 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.318+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42908/0 (socket says 172.21.15.138:42908) 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 -- 172.21.15.138:0/1976743284 learned_addr learned my addr 172.21.15.138:0/1976743284 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2ecc00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 0x7fe2e81a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2df400640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 0x7fe2e819b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 -- 172.21.15.138:0/1976743284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 msgr2=0x7fe2e81a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 0x7fe2e81a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 -- 172.21.15.138:0/1976743284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 msgr2=0x7fe2e819b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 0x7fe2e819b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 -- 172.21.15.138:0/1976743284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2e81a1270 con 0x7fe2e8104ca0 2024-08-20T01:07:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2ecc00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 0x7fe2e81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2df400640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 0x7fe2e819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.319+0000 7fe2e7e00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005940 tx=0x7fe2d80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.320+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d803f030 con 0x7fe2e8104ca0 2024-08-20T01:07:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.320+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2d80044c0 con 0x7fe2e8104ca0 2024-08-20T01:07:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.320+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2d8035070 con 0x7fe2e8104ca0 2024-08-20T01:07:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.320+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2e81a1500 con 0x7fe2e8104ca0 2024-08-20T01:07:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.320+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2e81a7c20 con 0x7fe2e8104ca0 2024-08-20T01:07:30.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.321+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2e8105080 con 0x7fe2e8104ca0 2024-08-20T01:07:30.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.323+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fe2d8042030 con 0x7fe2e8104ca0 2024-08-20T01:07:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.324+0000 7fe2e5600640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 0x7fe2cc0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.325+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe2d81044f0 con 0x7fe2e8104ca0 2024-08-20T01:07:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.325+0000 7fe2df400640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 0x7fe2cc0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.326+0000 7fe2df400640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 0x7fe2cc0b61f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8109d10 tx=0x7fe2d0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:30.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.328+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe2d80caf20 con 0x7fe2e8104ca0 2024-08-20T01:07:30.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.186+0000 7f7a21600640 1 -- 172.21.15.138:0/1635597201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c10a6a0 msgr2=0x7f7a1c10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.186+0000 7f7a21600640 1 --2- 172.21.15.138:0/1635597201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c10aae0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7a04009900 tx=0x7f7a0402a410 comp rx=0 tx=0).stop 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 -- 172.21.15.138:0/1635597201 shutdown_connections 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 --2- 172.21.15.138:0/1635597201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c111820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 --2- 172.21.15.138:0/1635597201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c10aae0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 --2- 172.21.15.138:0/1635597201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c104c90 0x7f7a1c105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 -- 172.21.15.138:0/1635597201 >> 172.21.15.138:0/1635597201 conn(0x7f7a1c1006c0 msgr2=0x7f7a1c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 -- 172.21.15.138:0/1635597201 shutdown_connections 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.187+0000 7f7a21600640 1 -- 172.21.15.138:0/1635597201 wait complete. 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 Processor -- start 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 -- start start 2024-08-20T01:07:30.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c114d30 con 0x7f7a1c104c90 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c114bb0 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a21600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a1c114eb0 con 0x7f7a1c10b020 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42886/0 (socket says 172.21.15.138:42886) 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.188+0000 7f7a1aa00640 1 -- 172.21.15.138:0/3144828768 learned_addr learned my addr 172.21.15.138:0/3144828768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:30.414 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a1b400640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 -- 172.21.15.138:0/3144828768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 msgr2=0x7f7a1c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 -- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 msgr2=0x7f7a1c19b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 -- 172.21.15.138:0/3144828768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a1c1a1270 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a1aa00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a1b400640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.189+0000 7f7a13e00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c19b700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a0402aa00 tx=0x7f7a04003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.190+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a0403f070 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.190+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a04031b50 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.415 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.190+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a1c1a1500 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.190+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a04035030 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.190+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a1c1a7c20 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.191+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a1c105070 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.193+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f7a04042070 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.194+0000 7f7a13400640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 0x7f7a000b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.195+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f7a04104960 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.195+0000 7f7a1aa00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 0x7f7a000b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.195+0000 7f7a1aa00640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 0x7f7a000b61f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c011550 tx=0x7f7a0c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.196+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7a04031d00 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.395+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f7a1c02e340 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.395+0000 7f7a13400640 1 -- 172.21.15.138:0/3144828768 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7a040cb330 con 0x7f7a1c10a6a0 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stdout:257698037808 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 msgr2=0x7f7a000b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 0x7f7a000b61f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c011550 tx=0x7f7a0c005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 msgr2=0x7f7a1c19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c19b700 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7a0402aa00 tx=0x7f7a04003b90 comp rx=0 tx=0).stop 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 shutdown_connections 2024-08-20T01:07:30.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a000b3d30 0x7f7a000b61f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a1c10b020 0x7f7a1c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a1c10a6a0 0x7f7a1c19b700 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 --2- 172.21.15.138:0/3144828768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a1c104c90 0x7f7a1c19b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 >> 172.21.15.138:0/3144828768 conn(0x7f7a1c1006c0 msgr2=0x7f7a1c102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 shutdown_connections 2024-08-20T01:07:30.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.398+0000 7f7a21600640 1 -- 172.21.15.138:0/3144828768 wait complete. 2024-08-20T01:07:30.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.523+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe2e819c2f0 con 0x7fe2e8104ca0 2024-08-20T01:07:30.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.524+0000 7fe2e5600640 1 -- 172.21.15.138:0/1976743284 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe2d80cf470 con 0x7fe2e8104ca0 2024-08-20T01:07:30.523 INFO:teuthology.orchestra.run.smithi138.stdout:184683593793 2024-08-20T01:07:30.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 msgr2=0x7fe2cc0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 0x7fe2cc0b61f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8109d10 tx=0x7fe2d0005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 msgr2=0x7fe2e819b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe2d8005940 tx=0x7fe2d80040c0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 shutdown_connections 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2cc0b3d30 0x7fe2cc0b61f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2e810b030 0x7fe2e81a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2e810a6b0 0x7fe2e819b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 --2- 172.21.15.138:0/1976743284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2e8104ca0 0x7fe2e819b1c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 >> 172.21.15.138:0/1976743284 conn(0x7fe2e8100730 msgr2=0x7fe2e8102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 shutdown_connections 2024-08-20T01:07:30.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:30.526+0000 7fe2eec00640 1 -- 172.21.15.138:0/1976743284 wait complete. 2024-08-20T01:07:30.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:30 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3144828768' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T01:07:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:30 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1976743284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T01:07:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:30 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3144828768' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T01:07:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:30 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1976743284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T01:07:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:30 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3144828768' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-08-20T01:07:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:30 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1976743284' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-08-20T01:07:31.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.211+0000 7f7c03e00640 1 -- 172.21.15.138:0/3070179251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04108310 msgr2=0x7f7c04108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.211+0000 7f7c03e00640 1 --2- 172.21.15.138:0/3070179251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04108310 0x7f7c04108750 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009900 tx=0x7f7bf002a410 comp rx=0 tx=0).stop 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.211+0000 7f7c03e00640 1 -- 172.21.15.138:0/3070179251 shutdown_connections 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 --2- 172.21.15.138:0/3070179251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 0x7f7c0410f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 --2- 172.21.15.138:0/3070179251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04108310 0x7f7c04108750 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 --2- 172.21.15.138:0/3070179251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04102900 0x7f7c04102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 -- 172.21.15.138:0/3070179251 >> 172.21.15.138:0/3070179251 conn(0x7f7c040fe390 msgr2=0x7f7c041007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 -- 172.21.15.138:0/3070179251 shutdown_connections 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 -- 172.21.15.138:0/3070179251 wait complete. 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.212+0000 7f7c03e00640 1 Processor -- start 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 -- start start 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04108310 0x7f7c04199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 0x7f7c0419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7c041129a0 con 0x7f7c04102900 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04112820 con 0x7f7c04108310 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7c04112b20 con 0x7f7c04108c90 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c02a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c02a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42928/0 (socket says 172.21.15.138:42928) 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c02a00640 1 -- 172.21.15.138:0/1844953747 learned_addr learned my addr 172.21.15.138:0/1844953747 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c02000640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04108310 0x7f7c04199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.213+0000 7f7c03400640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 0x7f7c0419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 -- 172.21.15.138:0/1844953747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 msgr2=0x7f7c0419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 0x7f7c0419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 -- 172.21.15.138:0/1844953747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04108310 msgr2=0x7f7c04199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04108310 0x7f7c04199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 -- 172.21.15.138:0/1844953747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0419eda0 con 0x7f7c04102900 2024-08-20T01:07:31.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c02a00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9f0 tx=0x7f7bf400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf400cd80 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf40108e0 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf4018980 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.214+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0419f090 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.215+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c041a5750 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.216+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c04102ce0 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.218+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f7bf4010430 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.219+0000 7f7bfb400640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 0x7f7be80b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.219+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f7bf40e4600 con 0x7f7c04102900 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.219+0000 7f7c02000640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 0x7f7be80b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.220+0000 7f7c02000640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 0x7f7be80b62a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005ab0 tx=0x7f7bf0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:31.338 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.223+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7bf40ab620 con 0x7f7c04102900 2024-08-20T01:07:31.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.443+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f7c041a5b10 con 0x7f7c04102900 2024-08-20T01:07:31.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.444+0000 7f7bfb400640 1 -- 172.21.15.138:0/1844953747 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7bf40aaf20 con 0x7f7c04102900 2024-08-20T01:07:31.772 INFO:teuthology.orchestra.run.smithi138.stdout:206158430267 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.446+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 msgr2=0x7f7be80b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.446+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 0x7f7be80b62a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0005ab0 tx=0x7f7bf0005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.446+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 msgr2=0x7f7c04198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.446+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7bf400e9f0 tx=0x7f7bf400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 shutdown_connections 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7be80b3de0 0x7f7be80b62a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7c04108c90 0x7f7c0419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c04108310 0x7f7c04199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 --2- 172.21.15.138:0/1844953747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c04102900 0x7f7c04198cf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 >> 172.21.15.138:0/1844953747 conn(0x7f7c040fe390 msgr2=0x7f7c04101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:31.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 shutdown_connections 2024-08-20T01:07:31.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.447+0000 7f7c03e00640 1 -- 172.21.15.138:0/1844953747 wait complete. 2024-08-20T01:07:31.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 -- 172.21.15.138:0/1998434820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6601084c0 msgr2=0x7fe660108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 --2- 172.21.15.138:0/1998434820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6601084c0 0x7fe660108900 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 -- 172.21.15.138:0/1998434820 shutdown_connections 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 --2- 172.21.15.138:0/1998434820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe660108e40 0x7fe66010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 --2- 172.21.15.138:0/1998434820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe6601084c0 0x7fe660108900 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 --2- 172.21.15.138:0/1998434820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe660102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 -- 172.21.15.138:0/1998434820 >> 172.21.15.138:0/1998434820 conn(0x7fe6600fe500 msgr2=0x7fe660100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 -- 172.21.15.138:0/1998434820 shutdown_connections 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.348+0000 7fe667c00640 1 -- 172.21.15.138:0/1998434820 wait complete. 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.349+0000 7fe667c00640 1 Processor -- start 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.349+0000 7fe667c00640 1 -- start start 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6601084c0 0x7fe66019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe660108e40 0x7fe6601a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe660112b00 con 0x7fe6601084c0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe660112980 con 0x7fe660102ab0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe667c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe660112c80 con 0x7fe660108e40 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe665200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe65fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6601084c0 0x7fe66019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe665200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:59264/0 (socket says 172.21.15.138:59264) 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe665200640 1 -- 172.21.15.138:0/3737082601 learned_addr learned my addr 172.21.15.138:0/3737082601 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.350+0000 7fe665c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe660108e40 0x7fe6601a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 -- 172.21.15.138:0/3737082601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe660108e40 msgr2=0x7fe6601a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe660108e40 0x7fe6601a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 -- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6601084c0 msgr2=0x7fe66019b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6601084c0 0x7fe66019b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 -- 172.21.15.138:0/3737082601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6601a1270 con 0x7fe660102ab0 2024-08-20T01:07:31.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe665200640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9f0 tx=0x7fe65400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65400cd80 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6601a1560 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6540054d0 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe654010690 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.351+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6601a7c20 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.353+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe660102e90 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.356+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fe654004e90 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.358+0000 7fe65d600640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 0x7fe6480b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.359+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe6540e46d0 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.359+0000 7fe65fe00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 0x7fe6480b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.359+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe6540e4b30 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.359+0000 7fe65fe00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 0x7fe6480b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe64c031440 tx=0x7fe64c005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.581+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe66002e340 con 0x7fe660102ab0 2024-08-20T01:07:31.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.583+0000 7fe65d600640 1 -- 172.21.15.138:0/3737082601 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6540ab6f0 con 0x7fe660102ab0 2024-08-20T01:07:31.778 INFO:teuthology.orchestra.run.smithi138.stdout:158913790023 2024-08-20T01:07:31.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 msgr2=0x7fe6480b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 0x7fe6480b61f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe64c031440 tx=0x7fe64c005b50 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 msgr2=0x7fe66019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe65400e9f0 tx=0x7fe65400edf0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 shutdown_connections 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe6480b3d30 0x7fe6480b61f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe660108e40 0x7fe6601a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6601084c0 0x7fe66019b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 --2- 172.21.15.138:0/3737082601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe660102ab0 0x7fe66019b1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.584+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 >> 172.21.15.138:0/3737082601 conn(0x7fe6600fe500 msgr2=0x7fe660104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.585+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 shutdown_connections 2024-08-20T01:07:31.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:31.585+0000 7fe667c00640 1 -- 172.21.15.138:0/3737082601 wait complete. 2024-08-20T01:07:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:31 smithi138 ceph-mon[18076]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:31 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1844953747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T01:07:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:31 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3737082601' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T01:07:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:31 smithi167 ceph-mon[21965]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:31 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1844953747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T01:07:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:31 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3737082601' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T01:07:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:31 smithi142 ceph-mon[21799]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:31 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1844953747' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-08-20T01:07:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:31 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3737082601' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-08-20T01:07:32.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 -- 172.21.15.138:0/12816145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 msgr2=0x7fb79810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:32.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 --2- 172.21.15.138:0/12816145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79810a960 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb784009900 tx=0x7fb78402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 -- 172.21.15.138:0/12816145 shutdown_connections 2024-08-20T01:07:32.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 --2- 172.21.15.138:0/12816145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 --2- 172.21.15.138:0/12816145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79810a960 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 --2- 172.21.15.138:0/12816145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb798104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.110+0000 7fb79d600640 1 -- 172.21.15.138:0/12816145 >> 172.21.15.138:0/12816145 conn(0x7fb798100540 msgr2=0x7fb798102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:32.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.111+0000 7fb79d600640 1 -- 172.21.15.138:0/12816145 shutdown_connections 2024-08-20T01:07:32.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.111+0000 7fb79d600640 1 -- 172.21.15.138:0/12816145 wait complete. 2024-08-20T01:07:32.111 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 Processor -- start 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 -- start start 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb79819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb798114bb0 con 0x7fb79810a520 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb798114a30 con 0x7fb798104b10 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb79d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb798114d30 con 0x7fb79810aea0 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.112+0000 7fb797400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb79819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb79cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:32.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42956/0 (socket says 172.21.15.138:42956) 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797400640 1 -- 172.21.15.138:0/3507176991 learned_addr learned my addr 172.21.15.138:0/3507176991 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 -- 172.21.15.138:0/3507176991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 msgr2=0x7fb7981a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 -- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 msgr2=0x7fb79819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 -- 172.21.15.138:0/3507176991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7981a0fd0 con 0x7fb798104b10 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb79cc00640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:32.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.113+0000 7fb797e00640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb79819af20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e9f0 tx=0x7fb78800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:32.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.114+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78800ccc0 con 0x7fb798104b10 2024-08-20T01:07:32.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.114+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7981a12c0 con 0x7fb798104b10 2024-08-20T01:07:32.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.114+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7880108e0 con 0x7fb798104b10 2024-08-20T01:07:32.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.114+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788018880 con 0x7fb798104b10 2024-08-20T01:07:32.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.114+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981a7980 con 0x7fb798104b10 2024-08-20T01:07:32.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.115+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb798104ef0 con 0x7fb798104b10 2024-08-20T01:07:32.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.117+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7fb788010430 con 0x7fb798104b10 2024-08-20T01:07:32.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.119+0000 7fb794c00640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 0x7fb77c0b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:32.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.119+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fb7880e4760 con 0x7fb798104b10 2024-08-20T01:07:32.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.119+0000 7fb797400640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 0x7fb77c0b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:32.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.120+0000 7fb797400640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 0x7fb77c0b6220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb79810a0b0 tx=0x7fb784005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:32.118 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.120+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb7880ab130 con 0x7fb798104b10 2024-08-20T01:07:32.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.306+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fb7981a7cf0 con 0x7fb798104b10 2024-08-20T01:07:32.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.307+0000 7fb794c00640 1 -- 172.21.15.138:0/3507176991 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb7880af680 con 0x7fb798104b10 2024-08-20T01:07:32.306 INFO:teuthology.orchestra.run.smithi138.stdout:231928234038 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.309+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 msgr2=0x7fb77c0b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.309+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 0x7fb77c0b6220 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb79810a0b0 tx=0x7fb784005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 msgr2=0x7fb79819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb79819af20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb78800e9f0 tx=0x7fb78800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 shutdown_connections 2024-08-20T01:07:32.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb77c0b3d60 0x7fb77c0b6220 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb79810aea0 0x7fb7981a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb79810a520 0x7fb79819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 --2- 172.21.15.138:0/3507176991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb798104b10 0x7fb79819af20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 >> 172.21.15.138:0/3507176991 conn(0x7fb798100540 msgr2=0x7fb798103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 shutdown_connections 2024-08-20T01:07:32.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:32.310+0000 7fb79d600640 1 -- 172.21.15.138:0/3507176991 wait complete. 2024-08-20T01:07:32.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430265 got 206158430267 for osd.7 2024-08-20T01:07:32.481 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:32.781 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:32 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3507176991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T01:07:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:32 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3507176991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T01:07:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:32 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3507176991' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 -- 172.21.15.138:0/2218011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738010b030 msgr2=0x7f7380111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 --2- 172.21.15.138:0/2218011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738010b030 0x7f7380111830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7378009ca0 tx=0x7f737802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 -- 172.21.15.138:0/2218011591 shutdown_connections 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 --2- 172.21.15.138:0/2218011591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f738010b030 0x7f7380111830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 --2- 172.21.15.138:0/2218011591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010a6b0 0x7f738010aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 --2- 172.21.15.138:0/2218011591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7380104ca0 0x7f7380105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 -- 172.21.15.138:0/2218011591 >> 172.21.15.138:0/2218011591 conn(0x7f7380100730 msgr2=0x7f7380102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 -- 172.21.15.138:0/2218011591 shutdown_connections 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.239+0000 7f7387200640 1 -- 172.21.15.138:0/2218011591 wait complete. 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.240+0000 7f7387200640 1 Processor -- start 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.240+0000 7f7387200640 1 -- start start 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738010a6b0 0x7f738019b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010b030 0x7f73801a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114d00 con 0x7f7380104ca0 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114b80 con 0x7f738010a6b0 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7387200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7380114e80 con 0x7f738010b030 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42978/0 (socket says 172.21.15.138:42978) 2024-08-20T01:07:33.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 -- 172.21.15.138:0/727272440 learned_addr learned my addr 172.21.15.138:0/727272440 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f7385200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010b030 0x7f73801a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737f400640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738010a6b0 0x7f738019b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 -- 172.21.15.138:0/727272440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010b030 msgr2=0x7f73801a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010b030 0x7f73801a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 -- 172.21.15.138:0/727272440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738010a6b0 msgr2=0x7f738019b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738010a6b0 0x7f738019b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.241+0000 7f737fe00640 1 -- 172.21.15.138:0/727272440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73801a11e0 con 0x7f7380104ca0 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f737fe00640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f737000e9f0 tx=0x7f737000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f737000cd80 con 0x7f7380104ca0 2024-08-20T01:07:33.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73801a14d0 con 0x7f7380104ca0 2024-08-20T01:07:33.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73700108e0 con 0x7f7380104ca0 2024-08-20T01:07:33.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73801a7d60 con 0x7f7380104ca0 2024-08-20T01:07:33.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.242+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7370018920 con 0x7f7380104ca0 2024-08-20T01:07:33.541 INFO:teuthology.orchestra.run.smithi138.stdout:137438953548 2024-08-20T01:07:33.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.244+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7380105080 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.246+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 21) ==== 151128+0+0 (secure 0 0 0) 0x7f7370010430 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.247+0000 7f737cc00640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 0x7f73640b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.247+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f73700e46e0 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.247+0000 7f737f400640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 0x7f73640b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.248+0000 7f737f400640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 0x7f73640b6170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73801073f0 tx=0x7f736c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.251+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f73700ab0b0 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.444+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f738010ce40 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.444+0000 7f737cc00640 1 -- 172.21.15.138:0/727272440 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f73700af600 con 0x7f7380104ca0 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.446+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 msgr2=0x7f73640b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.446+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 0x7f73640b6170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f73801073f0 tx=0x7f736c005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.446+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 msgr2=0x7f738019b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:33.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.446+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f737000e9f0 tx=0x7f737000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 shutdown_connections 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f73640b3cb0 0x7f73640b6170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f738010b030 0x7f73801a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f738010a6b0 0x7f738019b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 --2- 172.21.15.138:0/727272440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7380104ca0 0x7f738019b130 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 >> 172.21.15.138:0/727272440 conn(0x7f7380100730 msgr2=0x7f7380102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 shutdown_connections 2024-08-20T01:07:33.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:33.447+0000 7f7387200640 1 -- 172.21.15.138:0/727272440 wait complete. 2024-08-20T01:07:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:33 smithi138 ceph-mon[18076]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:33.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:33 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/727272440' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T01:07:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:33 smithi167 ceph-mon[21965]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:33 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/727272440' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T01:07:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:33 smithi142 ceph-mon[21799]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:33 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/727272440' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-08-20T01:07:34.464 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790018 got 158913790023 for osd.5 2024-08-20T01:07:34.464 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:34.867 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953544 got 137438953548 for osd.4 2024-08-20T01:07:34.867 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:35.247 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378707 got 81604378710 for osd.2 2024-08-20T01:07:35.247 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:35.517 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:35.837 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:35 smithi138 ceph-mon[18076]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:35 smithi167 ceph-mon[21965]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:35 smithi142 ceph-mon[21799]: pgmap v346: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:36.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593791 got 184683593793 for osd.6 2024-08-20T01:07:36.109 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:36.663 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234035 got 231928234038 for osd.8 2024-08-20T01:07:36.663 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:37.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037805 got 257698037808 for osd.9 2024-08-20T01:07:37.572 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:37.761 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:37 smithi138 ceph-mon[18076]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:37.761 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:37 smithi138 ceph-mon[18076]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 -- 172.21.15.138:0/187775547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 msgr2=0x7f355c10e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 --2- 172.21.15.138:0/187775547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c10e960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3548009900 tx=0x7f354802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 -- 172.21.15.138:0/187775547 shutdown_connections 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 --2- 172.21.15.138:0/187775547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c10ef30 0x7f355c111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 --2- 172.21.15.138:0/187775547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c10e960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 --2- 172.21.15.138:0/187775547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c075270 0x7f355c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 -- 172.21.15.138:0/187775547 >> 172.21.15.138:0/187775547 conn(0x7f355c0fe3c0 msgr2=0x7f355c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 -- 172.21.15.138:0/187775547 shutdown_connections 2024-08-20T01:07:37.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.865+0000 7f3562200640 1 -- 172.21.15.138:0/187775547 wait complete. 2024-08-20T01:07:37.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.866+0000 7f3562200640 1 Processor -- start 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 -- start start 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 0x7f355c19af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 0x7f355c1a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f355c114b30 con 0x7f355c075c70 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f355c1149b0 con 0x7f355c10ef30 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3562200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f355c114cb0 con 0x7f355c075270 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f355be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:37.866 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f355be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44820/0 (socket says 172.21.15.138:44820) 2024-08-20T01:07:37.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f355be00640 1 -- 172.21.15.138:0/386703739 learned_addr learned my addr 172.21.15.138:0/386703739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:37.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3560e00640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 0x7f355c19af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:37.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.867+0000 7f3561800640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 0x7f355c1a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 -- 172.21.15.138:0/386703739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 msgr2=0x7f355c19af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 0x7f355c19af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 -- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 msgr2=0x7f355c19b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 -- 172.21.15.138:0/386703739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355c1a1010 con 0x7f355c10ef30 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3560e00640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 0x7f355c19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f355be00640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3561800640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 0x7f355c1a0840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f355400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:37.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355400cdf0 con 0x7f355c10ef30 2024-08-20T01:07:37.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3554005020 con 0x7f355c10ef30 2024-08-20T01:07:37.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3554010690 con 0x7f355c10ef30 2024-08-20T01:07:37.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.868+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355c1a1300 con 0x7f355c10ef30 2024-08-20T01:07:37.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.869+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355c1a79c0 con 0x7f355c10ef30 2024-08-20T01:07:37.869 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.870+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355c075650 con 0x7f355c10ef30 2024-08-20T01:07:37.870 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.872+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f3554005460 con 0x7f355c10ef30 2024-08-20T01:07:37.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.873+0000 7f3559600640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 0x7f35440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:37.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.873+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f35540e4660 con 0x7f355c10ef30 2024-08-20T01:07:37.871 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.873+0000 7f3560e00640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 0x7f35440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:37.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.874+0000 7f3560e00640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 0x7f35440b6200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3550005d00 tx=0x7f355000b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:37.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:37.877+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f35540ab7e0 con 0x7f355c10ef30 2024-08-20T01:07:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:37 smithi167 ceph-mon[21965]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:37 smithi167 ceph-mon[21965]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-20T01:07:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:37 smithi142 ceph-mon[21799]: pgmap v347: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:37 smithi142 ceph-mon[21799]: mgrmap e22: x(active, since 9m), standbys: z, y 2024-08-20T01:07:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.066+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f355c1a7db0 con 0x7f355c10ef30 2024-08-20T01:07:38.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.066+0000 7f3559600640 1 -- 172.21.15.138:0/386703739 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35540ab0e0 con 0x7f355c10ef30 2024-08-20T01:07:38.065 INFO:teuthology.orchestra.run.smithi138.stdout:60129542235 2024-08-20T01:07:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 msgr2=0x7f35440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 0x7f35440b6200 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3550005d00 tx=0x7f355000b410 comp rx=0 tx=0).stop 2024-08-20T01:07:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 msgr2=0x7f355c1a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:38.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 0x7f355c1a0840 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3554005fd0 tx=0x7f355400ef10 comp rx=0 tx=0).stop 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 shutdown_connections 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35440b3d40 0x7f35440b6200 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f355c10ef30 0x7f355c1a0840 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f355c075c70 0x7f355c19b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 --2- 172.21.15.138:0/386703739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f355c075270 0x7f355c19af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 >> 172.21.15.138:0/386703739 conn(0x7f355c0fe3c0 msgr2=0x7f355c111080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 shutdown_connections 2024-08-20T01:07:38.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:38.069+0000 7f3562200640 1 -- 172.21.15.138:0/386703739 wait complete. 2024-08-20T01:07:38.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:38 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/386703739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T01:07:38.987 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:38 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/386703739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T01:07:38.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:38 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/386703739' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-08-20T01:07:39.452 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542232 got 60129542235 for osd.1 2024-08-20T01:07:39.452 DEBUG:teuthology.parallel:result is None 2024-08-20T01:07:39.452 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T01:07:39.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:07:39.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:39 smithi138 ceph-mon[18076]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:39.867 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:39 smithi167 ceph-mon[21965]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:39 smithi142 ceph-mon[21799]: pgmap v348: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:40.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 -- 172.21.15.138:0/2438880578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3188075c70 msgr2=0x7f318810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 --2- 172.21.15.138:0/2438880578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3188075c70 0x7f318810eb00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3174009900 tx=0x7f317402a410 comp rx=0 tx=0).stop 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 -- 172.21.15.138:0/2438880578 shutdown_connections 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 --2- 172.21.15.138:0/2438880578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f318810f0d0 0x7f31881114c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 --2- 172.21.15.138:0/2438880578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3188075c70 0x7f318810eb00 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 --2- 172.21.15.138:0/2438880578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 0x7f3188075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 -- 172.21.15.138:0/2438880578 >> 172.21.15.138:0/2438880578 conn(0x7f31880fe500 msgr2=0x7f3188100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 -- 172.21.15.138:0/2438880578 shutdown_connections 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.512+0000 7f318f000640 1 -- 172.21.15.138:0/2438880578 wait complete. 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 Processor -- start 2024-08-20T01:07:40.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 -- start start 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 0x7f318819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3188075c70 0x7f318819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114cd0 con 0x7f318810f0d0 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114b50 con 0x7f3188075c70 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3188114e50 con 0x7f3188075270 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.513+0000 7f318d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44830/0 (socket says 172.21.15.138:44830) 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 -- 172.21.15.138:0/1386547764 learned_addr learned my addr 172.21.15.138:0/1386547764 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f3187e00640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 0x7f318819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:40.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f3187400640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3188075c70 0x7f318819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 -- 172.21.15.138:0/1386547764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 msgr2=0x7f318819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 0x7f318819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 -- 172.21.15.138:0/1386547764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3188075c70 msgr2=0x7f318819b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3188075c70 0x7f318819b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 -- 172.21.15.138:0/1386547764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31881a1250 con 0x7f318810f0d0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.514+0000 7f318d000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f318000dac0 tx=0x7f318000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.515+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180017070 con 0x7f318810f0d0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.515+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f318000be30 con 0x7f318810f0d0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.515+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31881a1540 con 0x7f318810f0d0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.515+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3180010460 con 0x7f318810f0d0 2024-08-20T01:07:40.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.515+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31881a7c00 con 0x7f318810f0d0 2024-08-20T01:07:40.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.516+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3188076780 con 0x7f318810f0d0 2024-08-20T01:07:40.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.518+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f31800040b0 con 0x7f318810f0d0 2024-08-20T01:07:40.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.520+0000 7f3184c00640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 0x7f316c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:40.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.520+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f31800e3c90 con 0x7f318810f0d0 2024-08-20T01:07:40.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.520+0000 7f3187e00640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 0x7f316c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:40.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.521+0000 7f3187e00640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 0x7f316c0b60e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3178005d00 tx=0x7f317800b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:40.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.526+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f31800aa7c0 con 0x7f318810f0d0 2024-08-20T01:07:40.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.742+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3188100110 con 0x7f316c0b3c20 2024-08-20T01:07:40.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.748+0000 7f3184c00640 1 -- 172.21.15.138:0/1386547764 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340841 (secure 0 0 0) 0x7f3188100110 con 0x7f316c0b3c20 2024-08-20T01:07:40.751 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:40.753 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 msgr2=0x7f316c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 0x7f316c0b60e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3178005d00 tx=0x7f317800b410 comp rx=0 tx=0).stop 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 msgr2=0x7f31881a0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f318000dac0 tx=0x7f318000df80 comp rx=0 tx=0).stop 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 shutdown_connections 2024-08-20T01:07:40.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f316c0b3c20 0x7f316c0b60e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f318810f0d0 0x7f31881a0a80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3188075c70 0x7f318819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 --2- 172.21.15.138:0/1386547764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3188075270 0x7f318819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.753+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 >> 172.21.15.138:0/1386547764 conn(0x7f31880fe500 msgr2=0x7f31881112c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.754+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 shutdown_connections 2024-08-20T01:07:40.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:40.754+0000 7f318f000640 1 -- 172.21.15.138:0/1386547764 wait complete. 2024-08-20T01:07:41.570 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-08-20T01:07:40.508113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659611,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":584,"num_read_kb":397,"num_write":302,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352804,"kb_used_data":11876,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589020,"statfs":{"total":1151940427776,"available":1151579156480,"internally_reserved":0,"allocated":12161024,"data_stored":8160923,"data_compressed":427158,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":88,"num_read_kb":48,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003657"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621129+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621129+0000","last_peered":"2024-08-20T01:06:36.621129+0000","last_clean":"2024-08-20T01:06:36.621129+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621129+0000","last_undegraded":"2024-08-20T01:06:36.621129+0000","last_fullsized":"2024-08-20T01:06:36.621129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731638+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:07:34.731638+0000","last_peered":"2024-08-20T01:07:34.731638+0000","last_clean":"2024-08-20T01:07:34.731638+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:07:34.731638+0000","last_undegraded":"2024-08-20T01:07:34.731638+0000","last_fullsized":"2024-08-20T01:07:34.731638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426580+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.426580+0000","last_peered":"2024-08-20T01:06:37.426580+0000","last_clean":"2024-08-20T01:06:37.426580+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.426580+0000","last_undegraded":"2024-08-20T01:06:37.426580+0000","last_fullsized":"2024-08-20T01:06:37.426580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564918+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564918+0000","last_peered":"2024-08-20T01:06:36.564918+0000","last_clean":"2024-08-20T01:06:36.564918+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564918+0000","last_undegraded":"2024-08-20T01:06:36.564918+0000","last_fullsized":"2024-08-20T01:06:36.564918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747371+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:07:34.747371+0000","last_peered":"2024-08-20T01:07:34.747371+0000","last_clean":"2024-08-20T01:07:34.747371+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:07:34.747371+0000","last_undegraded":"2024-08-20T01:07:34.747371+0000","last_fullsized":"2024-08-20T01:07:34.747371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564424+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564424+0000","last_peered":"2024-08-20T01:06:36.564424+0000","last_clean":"2024-08-20T01:06:36.564424+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564424+0000","last_undegraded":"2024-08-20T01:06:36.564424+0000","last_fullsized":"2024-08-20T01:06:36.564424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.744470+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:07:34.744470+0000","last_peered":"2024-08-20T01:07:34.744470+0000","last_clean":"2024-08-20T01:07:34.744470+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:07:34.744470+0000","last_undegraded":"2024-08-20T01:07:34.744470+0000","last_fullsized":"2024-08-20T01:07:34.744470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649109+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649109+0000","last_peered":"2024-08-20T01:06:36.649109+0000","last_clean":"2024-08-20T01:06:36.649109+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649109+0000","last_undegraded":"2024-08-20T01:06:36.649109+0000","last_fullsized":"2024-08-20T01:06:36.649109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730170+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:07:34.730170+0000","last_peered":"2024-08-20T01:07:34.730170+0000","last_clean":"2024-08-20T01:07:34.730170+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:07:34.730170+0000","last_undegraded":"2024-08-20T01:07:34.730170+0000","last_fullsized":"2024-08-20T01:07:34.730170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564564+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564564+0000","last_peered":"2024-08-20T01:06:36.564564+0000","last_clean":"2024-08-20T01:06:36.564564+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564564+0000","last_undegraded":"2024-08-20T01:06:36.564564+0000","last_fullsized":"2024-08-20T01:06:36.564564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733459+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:07:34.733459+0000","last_peered":"2024-08-20T01:07:34.733459+0000","last_clean":"2024-08-20T01:07:34.733459+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:07:34.733459+0000","last_undegraded":"2024-08-20T01:07:34.733459+0000","last_fullsized":"2024-08-20T01:07:34.733459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":57,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567524+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567524+0000","last_peered":"2024-08-20T01:06:36.567524+0000","last_clean":"2024-08-20T01:06:36.567524+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567524+0000","last_undegraded":"2024-08-20T01:06:36.567524+0000","last_fullsized":"2024-08-20T01:06:36.567524+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564813+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564813+0000","last_peered":"2024-08-20T01:06:36.564813+0000","last_clean":"2024-08-20T01:06:36.564813+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564813+0000","last_undegraded":"2024-08-20T01:06:36.564813+0000","last_fullsized":"2024-08-20T01:06:36.564813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739534+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:07:34.739534+0000","last_peered":"2024-08-20T01:07:34.739534+0000","last_clean":"2024-08-20T01:07:34.739534+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:07:34.739534+0000","last_undegraded":"2024-08-20T01:07:34.739534+0000","last_fullsized":"2024-08-20T01:07:34.739534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566999+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566999+0000","last_peered":"2024-08-20T01:06:36.566999+0000","last_clean":"2024-08-20T01:06:36.566999+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566999+0000","last_undegraded":"2024-08-20T01:06:36.566999+0000","last_fullsized":"2024-08-20T01:06:36.566999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564610+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564610+0000","last_peered":"2024-08-20T01:06:36.564610+0000","last_clean":"2024-08-20T01:06:36.564610+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564610+0000","last_undegraded":"2024-08-20T01:06:36.564610+0000","last_fullsized":"2024-08-20T01:06:36.564610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.737254+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:07:34.737254+0000","last_peered":"2024-08-20T01:07:34.737254+0000","last_clean":"2024-08-20T01:07:34.737254+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:07:34.737254+0000","last_undegraded":"2024-08-20T01:07:34.737254+0000","last_fullsized":"2024-08-20T01:07:34.737254+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564578+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564578+0000","last_peered":"2024-08-20T01:06:36.564578+0000","last_clean":"2024-08-20T01:06:36.564578+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564578+0000","last_undegraded":"2024-08-20T01:06:36.564578+0000","last_fullsized":"2024-08-20T01:06:36.564578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727978+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:07:34.727978+0000","last_peered":"2024-08-20T01:07:34.727978+0000","last_clean":"2024-08-20T01:07:34.727978+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:07:34.727978+0000","last_undegraded":"2024-08-20T01:07:34.727978+0000","last_fullsized":"2024-08-20T01:07:34.727978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642277+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642277+0000","last_peered":"2024-08-20T01:06:36.642277+0000","last_clean":"2024-08-20T01:06:36.642277+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642277+0000","last_undegraded":"2024-08-20T01:06:36.642277+0000","last_fullsized":"2024-08-20T01:06:36.642277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564951+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.564951+0000","last_peered":"2024-08-20T01:06:36.564951+0000","last_clean":"2024-08-20T01:06:36.564951+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.564951+0000","last_undegraded":"2024-08-20T01:06:36.564951+0000","last_fullsized":"2024-08-20T01:06:36.564951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738711+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:07:34.738711+0000","last_peered":"2024-08-20T01:07:34.738711+0000","last_clean":"2024-08-20T01:07:34.738711+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:07:34.738711+0000","last_undegraded":"2024-08-20T01:07:34.738711+0000","last_fullsized":"2024-08-20T01:07:34.738711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565514+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565514+0000","last_peered":"2024-08-20T01:06:36.565514+0000","last_clean":"2024-08-20T01:06:36.565514+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565514+0000","last_undegraded":"2024-08-20T01:06:36.565514+0000","last_fullsized":"2024-08-20T01:06:36.565514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566954+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.566954+0000","last_peered":"2024-08-20T01:06:36.566954+0000","last_clean":"2024-08-20T01:06:36.566954+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.566954+0000","last_undegraded":"2024-08-20T01:06:36.566954+0000","last_fullsized":"2024-08-20T01:06:36.566954+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727759+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:07:34.727759+0000","last_peered":"2024-08-20T01:07:34.727759+0000","last_clean":"2024-08-20T01:07:34.727759+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:07:34.727759+0000","last_undegraded":"2024-08-20T01:07:34.727759+0000","last_fullsized":"2024-08-20T01:07:34.727759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.313638+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:07:34.313638+0000","last_peered":"2024-08-20T01:07:34.313638+0000","last_clean":"2024-08-20T01:07:34.313638+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:07:34.313638+0000","last_undegraded":"2024-08-20T01:07:34.313638+0000","last_fullsized":"2024-08-20T01:07:34.313638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564152+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564152+0000","last_peered":"2024-08-20T01:06:36.564152+0000","last_clean":"2024-08-20T01:06:36.564152+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564152+0000","last_undegraded":"2024-08-20T01:06:36.564152+0000","last_fullsized":"2024-08-20T01:06:36.564152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729501+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:07:34.729501+0000","last_peered":"2024-08-20T01:07:34.729501+0000","last_clean":"2024-08-20T01:07:34.729501+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:07:34.729501+0000","last_undegraded":"2024-08-20T01:07:34.729501+0000","last_fullsized":"2024-08-20T01:07:34.729501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566711+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566711+0000","last_peered":"2024-08-20T01:06:36.566711+0000","last_clean":"2024-08-20T01:06:36.566711+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566711+0000","last_undegraded":"2024-08-20T01:06:36.566711+0000","last_fullsized":"2024-08-20T01:06:36.566711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338256+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338256+0000","last_peered":"2024-08-20T01:06:37.338256+0000","last_clean":"2024-08-20T01:06:37.338256+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338256+0000","last_undegraded":"2024-08-20T01:06:37.338256+0000","last_fullsized":"2024-08-20T01:06:37.338256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732734+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:07:34.732734+0000","last_peered":"2024-08-20T01:07:34.732734+0000","last_clean":"2024-08-20T01:07:34.732734+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:07:34.732734+0000","last_undegraded":"2024-08-20T01:07:34.732734+0000","last_fullsized":"2024-08-20T01:07:34.732734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620842+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620842+0000","last_peered":"2024-08-20T01:06:36.620842+0000","last_clean":"2024-08-20T01:06:36.620842+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620842+0000","last_undegraded":"2024-08-20T01:06:36.620842+0000","last_fullsized":"2024-08-20T01:06:36.620842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566713+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566713+0000","last_peered":"2024-08-20T01:06:36.566713+0000","last_clean":"2024-08-20T01:06:36.566713+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566713+0000","last_undegraded":"2024-08-20T01:06:36.566713+0000","last_fullsized":"2024-08-20T01:06:36.566713+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740243+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:07:34.740243+0000","last_peered":"2024-08-20T01:07:34.740243+0000","last_clean":"2024-08-20T01:07:34.740243+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:07:34.740243+0000","last_undegraded":"2024-08-20T01:07:34.740243+0000","last_fullsized":"2024-08-20T01:07:34.740243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900778+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900778+0000","last_peered":"2024-08-20T01:06:36.900778+0000","last_clean":"2024-08-20T01:06:36.900778+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900778+0000","last_undegraded":"2024-08-20T01:06:36.900778+0000","last_fullsized":"2024-08-20T01:06:36.900778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566915+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566915+0000","last_peered":"2024-08-20T01:06:36.566915+0000","last_clean":"2024-08-20T01:06:36.566915+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566915+0000","last_undegraded":"2024-08-20T01:06:36.566915+0000","last_fullsized":"2024-08-20T01:06:36.566915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731513+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:07:34.731513+0000","last_peered":"2024-08-20T01:07:34.731513+0000","last_clean":"2024-08-20T01:07:34.731513+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:07:34.731513+0000","last_undegraded":"2024-08-20T01:07:34.731513+0000","last_fullsized":"2024-08-20T01:07:34.731513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:38.739543+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:07:38.739543+0000","last_peered":"2024-08-20T01:07:38.739543+0000","last_clean":"2024-08-20T01:07:38.739543+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:07:38.739543+0000","last_undegraded":"2024-08-20T01:07:38.739543+0000","last_fullsized":"2024-08-20T01:07:38.739543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727983+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:07:34.727983+0000","last_peered":"2024-08-20T01:07:34.727983+0000","last_clean":"2024-08-20T01:07:34.727983+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:07:34.727983+0000","last_undegraded":"2024-08-20T01:07:34.727983+0000","last_fullsized":"2024-08-20T01:07:34.727983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900807+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900807+0000","last_peered":"2024-08-20T01:06:36.900807+0000","last_clean":"2024-08-20T01:06:36.900807+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900807+0000","last_undegraded":"2024-08-20T01:06:36.900807+0000","last_fullsized":"2024-08-20T01:06:36.900807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747444+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:07:34.747444+0000","last_peered":"2024-08-20T01:07:34.747444+0000","last_clean":"2024-08-20T01:07:34.747444+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:07:34.747444+0000","last_undegraded":"2024-08-20T01:07:34.747444+0000","last_fullsized":"2024-08-20T01:07:34.747444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565335+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565335+0000","last_peered":"2024-08-20T01:06:36.565335+0000","last_clean":"2024-08-20T01:06:36.565335+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565335+0000","last_undegraded":"2024-08-20T01:06:36.565335+0000","last_fullsized":"2024-08-20T01:06:36.565335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731812+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:07:34.731812+0000","last_peered":"2024-08-20T01:07:34.731812+0000","last_clean":"2024-08-20T01:07:34.731812+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:07:34.731812+0000","last_undegraded":"2024-08-20T01:07:34.731812+0000","last_fullsized":"2024-08-20T01:07:34.731812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567767+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567767+0000","last_peered":"2024-08-20T01:06:36.567767+0000","last_clean":"2024-08-20T01:06:36.567767+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567767+0000","last_undegraded":"2024-08-20T01:06:36.567767+0000","last_fullsized":"2024-08-20T01:06:36.567767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728844+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:07:34.728844+0000","last_peered":"2024-08-20T01:07:34.728844+0000","last_clean":"2024-08-20T01:07:34.728844+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:07:34.728844+0000","last_undegraded":"2024-08-20T01:07:34.728844+0000","last_fullsized":"2024-08-20T01:07:34.728844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647722+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647722+0000","last_peered":"2024-08-20T01:06:36.647722+0000","last_clean":"2024-08-20T01:06:36.647722+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647722+0000","last_undegraded":"2024-08-20T01:06:36.647722+0000","last_fullsized":"2024-08-20T01:06:36.647722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731203+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:07:34.731203+0000","last_peered":"2024-08-20T01:07:34.731203+0000","last_clean":"2024-08-20T01:07:34.731203+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:07:34.731203+0000","last_undegraded":"2024-08-20T01:07:34.731203+0000","last_fullsized":"2024-08-20T01:07:34.731203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738945+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:07:34.738945+0000","last_peered":"2024-08-20T01:07:34.738945+0000","last_clean":"2024-08-20T01:07:34.738945+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:07:34.738945+0000","last_undegraded":"2024-08-20T01:07:34.738945+0000","last_fullsized":"2024-08-20T01:07:34.738945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566418+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566418+0000","last_peered":"2024-08-20T01:06:36.566418+0000","last_clean":"2024-08-20T01:06:36.566418+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566418+0000","last_undegraded":"2024-08-20T01:06:36.566418+0000","last_fullsized":"2024-08-20T01:06:36.566418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731672+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:07:34.731672+0000","last_peered":"2024-08-20T01:07:34.731672+0000","last_clean":"2024-08-20T01:07:34.731672+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:07:34.731672+0000","last_undegraded":"2024-08-20T01:07:34.731672+0000","last_fullsized":"2024-08-20T01:07:34.731672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338120+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338120+0000","last_peered":"2024-08-20T01:06:37.338120+0000","last_clean":"2024-08-20T01:06:37.338120+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338120+0000","last_undegraded":"2024-08-20T01:06:37.338120+0000","last_fullsized":"2024-08-20T01:06:37.338120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724032+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:07:34.724032+0000","last_peered":"2024-08-20T01:07:34.724032+0000","last_clean":"2024-08-20T01:07:34.724032+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:07:34.724032+0000","last_undegraded":"2024-08-20T01:07:34.724032+0000","last_fullsized":"2024-08-20T01:07:34.724032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":121,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:33.732639+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:07:33.732639+0000","last_peered":"2024-08-20T01:07:33.732639+0000","last_clean":"2024-08-20T01:07:33.732639+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:07:33.732639+0000","last_undegraded":"2024-08-20T01:07:33.732639+0000","last_fullsized":"2024-08-20T01:07:33.732639+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":78,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739852+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:07:34.739852+0000","last_peered":"2024-08-20T01:07:34.739852+0000","last_clean":"2024-08-20T01:07:34.739852+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:07:34.739852+0000","last_undegraded":"2024-08-20T01:07:34.739852+0000","last_fullsized":"2024-08-20T01:07:34.739852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338281+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338281+0000","last_peered":"2024-08-20T01:06:37.338281+0000","last_clean":"2024-08-20T01:06:37.338281+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338281+0000","last_undegraded":"2024-08-20T01:06:37.338281+0000","last_fullsized":"2024-08-20T01:06:37.338281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724968+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:07:34.724968+0000","last_peered":"2024-08-20T01:07:34.724968+0000","last_clean":"2024-08-20T01:07:34.724968+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:07:34.724968+0000","last_undegraded":"2024-08-20T01:07:34.724968+0000","last_fullsized":"2024-08-20T01:07:34.724968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566684+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566684+0000","last_peered":"2024-08-20T01:06:36.566684+0000","last_clean":"2024-08-20T01:06:36.566684+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566684+0000","last_undegraded":"2024-08-20T01:06:36.566684+0000","last_fullsized":"2024-08-20T01:06:36.566684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.718438+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:07:34.718438+0000","last_peered":"2024-08-20T01:07:34.718438+0000","last_clean":"2024-08-20T01:07:34.718438+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:07:34.718438+0000","last_undegraded":"2024-08-20T01:07:34.718438+0000","last_fullsized":"2024-08-20T01:07:34.718438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'38","reported_seq":249,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:38.740926+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:07:38.740926+0000","last_peered":"2024-08-20T01:07:38.740926+0000","last_clean":"2024-08-20T01:07:38.740926+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:07:38.740926+0000","last_undegraded":"2024-08-20T01:07:38.740926+0000","last_fullsized":"2024-08-20T01:07:38.740926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64867,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":149,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747351+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:07:34.747351+0000","last_peered":"2024-08-20T01:07:34.747351+0000","last_clean":"2024-08-20T01:07:34.747351+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:07:34.747351+0000","last_undegraded":"2024-08-20T01:07:34.747351+0000","last_fullsized":"2024-08-20T01:07:34.747351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564971+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.564971+0000","last_peered":"2024-08-20T01:06:36.564971+0000","last_clean":"2024-08-20T01:06:36.564971+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.564971+0000","last_undegraded":"2024-08-20T01:06:36.564971+0000","last_fullsized":"2024-08-20T01:06:36.564971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739755+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:07:34.739755+0000","last_peered":"2024-08-20T01:07:34.739755+0000","last_clean":"2024-08-20T01:07:34.739755+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:07:34.739755+0000","last_undegraded":"2024-08-20T01:07:34.739755+0000","last_fullsized":"2024-08-20T01:07:34.739755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.745045+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:07:34.745045+0000","last_peered":"2024-08-20T01:07:34.745045+0000","last_clean":"2024-08-20T01:07:34.745045+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:07:34.745045+0000","last_undegraded":"2024-08-20T01:07:34.745045+0000","last_fullsized":"2024-08-20T01:07:34.745045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:37.739306+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:07:37.739306+0000","last_peered":"2024-08-20T01:07:37.739306+0000","last_clean":"2024-08-20T01:07:37.739306+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:07:37.739306+0000","last_undegraded":"2024-08-20T01:07:37.739306+0000","last_fullsized":"2024-08-20T01:07:37.739306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425144+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425144+0000","last_peered":"2024-08-20T01:06:37.425144+0000","last_clean":"2024-08-20T01:06:37.425144+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425144+0000","last_undegraded":"2024-08-20T01:06:37.425144+0000","last_fullsized":"2024-08-20T01:06:37.425144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731533+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:07:34.731533+0000","last_peered":"2024-08-20T01:07:34.731533+0000","last_clean":"2024-08-20T01:07:34.731533+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:07:34.731533+0000","last_undegraded":"2024-08-20T01:07:34.731533+0000","last_fullsized":"2024-08-20T01:07:34.731533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338439+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338439+0000","last_peered":"2024-08-20T01:06:37.338439+0000","last_clean":"2024-08-20T01:06:37.338439+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338439+0000","last_undegraded":"2024-08-20T01:06:37.338439+0000","last_fullsized":"2024-08-20T01:06:37.338439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728608+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:07:34.728608+0000","last_peered":"2024-08-20T01:07:34.728608+0000","last_clean":"2024-08-20T01:07:34.728608+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:07:34.728608+0000","last_undegraded":"2024-08-20T01:07:34.728608+0000","last_fullsized":"2024-08-20T01:07:34.728608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730294+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:07:34.730294+0000","last_peered":"2024-08-20T01:07:34.730294+0000","last_clean":"2024-08-20T01:07:34.730294+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:07:34.730294+0000","last_undegraded":"2024-08-20T01:07:34.730294+0000","last_fullsized":"2024-08-20T01:07:34.730294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566490+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566490+0000","last_peered":"2024-08-20T01:06:36.566490+0000","last_clean":"2024-08-20T01:06:36.566490+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566490+0000","last_undegraded":"2024-08-20T01:06:36.566490+0000","last_fullsized":"2024-08-20T01:06:36.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:37.739916+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:07:37.739916+0000","last_peered":"2024-08-20T01:07:37.739916+0000","last_clean":"2024-08-20T01:07:37.739916+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:07:37.739916+0000","last_undegraded":"2024-08-20T01:07:37.739916+0000","last_fullsized":"2024-08-20T01:07:37.739916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730611+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:07:34.730611+0000","last_peered":"2024-08-20T01:07:34.730611+0000","last_clean":"2024-08-20T01:07:34.730611+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:07:34.730611+0000","last_undegraded":"2024-08-20T01:07:34.730611+0000","last_fullsized":"2024-08-20T01:07:34.730611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567358+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567358+0000","last_peered":"2024-08-20T01:06:36.567358+0000","last_clean":"2024-08-20T01:06:36.567358+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567358+0000","last_undegraded":"2024-08-20T01:06:36.567358+0000","last_fullsized":"2024-08-20T01:06:36.567358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732743+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:07:34.732743+0000","last_peered":"2024-08-20T01:07:34.732743+0000","last_clean":"2024-08-20T01:07:34.732743+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:07:34.732743+0000","last_undegraded":"2024-08-20T01:07:34.732743+0000","last_fullsized":"2024-08-20T01:07:34.732743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733589+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:07:34.733589+0000","last_peered":"2024-08-20T01:07:34.733589+0000","last_clean":"2024-08-20T01:07:34.733589+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:07:34.733589+0000","last_undegraded":"2024-08-20T01:07:34.733589+0000","last_fullsized":"2024-08-20T01:07:34.733589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739063+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:07:34.739063+0000","last_peered":"2024-08-20T01:07:34.739063+0000","last_clean":"2024-08-20T01:07:34.739063+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:07:34.739063+0000","last_undegraded":"2024-08-20T01:07:34.739063+0000","last_fullsized":"2024-08-20T01:07:34.739063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739518+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:07:34.739518+0000","last_peered":"2024-08-20T01:07:34.739518+0000","last_clean":"2024-08-20T01:07:34.739518+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:07:34.739518+0000","last_undegraded":"2024-08-20T01:07:34.739518+0000","last_fullsized":"2024-08-20T01:07:34.739518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740881+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:07:34.740881+0000","last_peered":"2024-08-20T01:07:34.740881+0000","last_clean":"2024-08-20T01:07:34.740881+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:07:34.740881+0000","last_undegraded":"2024-08-20T01:07:34.740881+0000","last_fullsized":"2024-08-20T01:07:34.740881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729279+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:07:34.729279+0000","last_peered":"2024-08-20T01:07:34.729279+0000","last_clean":"2024-08-20T01:07:34.729279+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:07:34.729279+0000","last_undegraded":"2024-08-20T01:07:34.729279+0000","last_fullsized":"2024-08-20T01:07:34.729279+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725362+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:07:34.725362+0000","last_peered":"2024-08-20T01:07:34.725362+0000","last_clean":"2024-08-20T01:07:34.725362+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:07:34.725362+0000","last_undegraded":"2024-08-20T01:07:34.725362+0000","last_fullsized":"2024-08-20T01:07:34.725362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728179+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:07:34.728179+0000","last_peered":"2024-08-20T01:07:34.728179+0000","last_clean":"2024-08-20T01:07:34.728179+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:07:34.728179+0000","last_undegraded":"2024-08-20T01:07:34.728179+0000","last_fullsized":"2024-08-20T01:07:34.728179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732606+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:07:34.732606+0000","last_peered":"2024-08-20T01:07:34.732606+0000","last_clean":"2024-08-20T01:07:34.732606+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:07:34.732606+0000","last_undegraded":"2024-08-20T01:07:34.732606+0000","last_fullsized":"2024-08-20T01:07:34.732606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729178+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:07:34.729178+0000","last_peered":"2024-08-20T01:07:34.729178+0000","last_clean":"2024-08-20T01:07:34.729178+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:07:34.729178+0000","last_undegraded":"2024-08-20T01:07:34.729178+0000","last_fullsized":"2024-08-20T01:07:34.729178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747342+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:07:34.747342+0000","last_peered":"2024-08-20T01:07:34.747342+0000","last_clean":"2024-08-20T01:07:34.747342+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:07:34.747342+0000","last_undegraded":"2024-08-20T01:07:34.747342+0000","last_fullsized":"2024-08-20T01:07:34.747342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741357+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:07:34.741357+0000","last_peered":"2024-08-20T01:07:34.741357+0000","last_clean":"2024-08-20T01:07:34.741357+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:07:34.741357+0000","last_undegraded":"2024-08-20T01:07:34.741357+0000","last_fullsized":"2024-08-20T01:07:34.741357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731379+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:07:34.731379+0000","last_peered":"2024-08-20T01:07:34.731379+0000","last_clean":"2024-08-20T01:07:34.731379+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:07:34.731379+0000","last_undegraded":"2024-08-20T01:07:34.731379+0000","last_fullsized":"2024-08-20T01:07:34.731379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730402+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:07:34.730402+0000","last_peered":"2024-08-20T01:07:34.730402+0000","last_clean":"2024-08-20T01:07:34.730402+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:07:34.730402+0000","last_undegraded":"2024-08-20T01:07:34.730402+0000","last_fullsized":"2024-08-20T01:07:34.730402+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.735358+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:07:34.735358+0000","last_peered":"2024-08-20T01:07:34.735358+0000","last_clean":"2024-08-20T01:07:34.735358+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:07:34.735358+0000","last_undegraded":"2024-08-20T01:07:34.735358+0000","last_fullsized":"2024-08-20T01:07:34.735358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741881+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:07:34.741881+0000","last_peered":"2024-08-20T01:07:34.741881+0000","last_clean":"2024-08-20T01:07:34.741881+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:07:34.741881+0000","last_undegraded":"2024-08-20T01:07:34.741881+0000","last_fullsized":"2024-08-20T01:07:34.741881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746898+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:07:34.746898+0000","last_peered":"2024-08-20T01:07:34.746898+0000","last_clean":"2024-08-20T01:07:34.746898+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:07:34.746898+0000","last_undegraded":"2024-08-20T01:07:34.746898+0000","last_fullsized":"2024-08-20T01:07:34.746898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740158+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:07:34.740158+0000","last_peered":"2024-08-20T01:07:34.740158+0000","last_clean":"2024-08-20T01:07:34.740158+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:07:34.740158+0000","last_undegraded":"2024-08-20T01:07:34.740158+0000","last_fullsized":"2024-08-20T01:07:34.740158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738949+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:07:34.738949+0000","last_peered":"2024-08-20T01:07:34.738949+0000","last_clean":"2024-08-20T01:07:34.738949+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:07:34.738949+0000","last_undegraded":"2024-08-20T01:07:34.738949+0000","last_fullsized":"2024-08-20T01:07:34.738949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.736362+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:07:34.736362+0000","last_peered":"2024-08-20T01:07:34.736362+0000","last_clean":"2024-08-20T01:07:34.736362+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:07:34.736362+0000","last_undegraded":"2024-08-20T01:07:34.736362+0000","last_fullsized":"2024-08-20T01:07:34.736362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730265+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:07:34.730265+0000","last_peered":"2024-08-20T01:07:34.730265+0000","last_clean":"2024-08-20T01:07:34.730265+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:07:34.730265+0000","last_undegraded":"2024-08-20T01:07:34.730265+0000","last_fullsized":"2024-08-20T01:07:34.730265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730892+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:07:34.730892+0000","last_peered":"2024-08-20T01:07:34.730892+0000","last_clean":"2024-08-20T01:07:34.730892+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:07:34.730892+0000","last_undegraded":"2024-08-20T01:07:34.730892+0000","last_fullsized":"2024-08-20T01:07:34.730892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739871+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:07:34.739871+0000","last_peered":"2024-08-20T01:07:34.739871+0000","last_clean":"2024-08-20T01:07:34.739871+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:07:34.739871+0000","last_undegraded":"2024-08-20T01:07:34.739871+0000","last_fullsized":"2024-08-20T01:07:34.739871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.743763+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:07:34.743763+0000","last_peered":"2024-08-20T01:07:34.743763+0000","last_clean":"2024-08-20T01:07:34.743763+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:07:34.743763+0000","last_undegraded":"2024-08-20T01:07:34.743763+0000","last_fullsized":"2024-08-20T01:07:34.743763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734211+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:07:34.734211+0000","last_peered":"2024-08-20T01:07:34.734211+0000","last_clean":"2024-08-20T01:07:34.734211+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:07:34.734211+0000","last_undegraded":"2024-08-20T01:07:34.734211+0000","last_fullsized":"2024-08-20T01:07:34.734211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729816+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:07:34.729816+0000","last_peered":"2024-08-20T01:07:34.729816+0000","last_clean":"2024-08-20T01:07:34.729816+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:07:34.729816+0000","last_undegraded":"2024-08-20T01:07:34.729816+0000","last_fullsized":"2024-08-20T01:07:34.729816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740550+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:07:34.740550+0000","last_peered":"2024-08-20T01:07:34.740550+0000","last_clean":"2024-08-20T01:07:34.740550+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:07:34.740550+0000","last_undegraded":"2024-08-20T01:07:34.740550+0000","last_fullsized":"2024-08-20T01:07:34.740550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729779+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:07:34.729779+0000","last_peered":"2024-08-20T01:07:34.729779+0000","last_clean":"2024-08-20T01:07:34.729779+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:07:34.729779+0000","last_undegraded":"2024-08-20T01:07:34.729779+0000","last_fullsized":"2024-08-20T01:07:34.729779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733118+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:07:34.733118+0000","last_peered":"2024-08-20T01:07:34.733118+0000","last_clean":"2024-08-20T01:07:34.733118+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:07:34.733118+0000","last_undegraded":"2024-08-20T01:07:34.733118+0000","last_fullsized":"2024-08-20T01:07:34.733118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747505+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:07:34.747505+0000","last_peered":"2024-08-20T01:07:34.747505+0000","last_clean":"2024-08-20T01:07:34.747505+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:07:34.747505+0000","last_undegraded":"2024-08-20T01:07:34.747505+0000","last_fullsized":"2024-08-20T01:07:34.747505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739436+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:07:34.739436+0000","last_peered":"2024-08-20T01:07:34.739436+0000","last_clean":"2024-08-20T01:07:34.739436+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:07:34.739436+0000","last_undegraded":"2024-08-20T01:07:34.739436+0000","last_fullsized":"2024-08-20T01:07:34.739436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742422+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:07:34.742422+0000","last_peered":"2024-08-20T01:07:34.742422+0000","last_clean":"2024-08-20T01:07:34.742422+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:07:34.742422+0000","last_undegraded":"2024-08-20T01:07:34.742422+0000","last_fullsized":"2024-08-20T01:07:34.742422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69009,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":503,"num_read_kb":328,"num_write":177,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678056,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874286,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1169315,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037811,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234040,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1113386,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1108974,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953549,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378712,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":579251,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542236,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":577369,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:07:41.576 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:07:41.795 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:41 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.947 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:07:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:41 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: pgmap v349: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:07:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:41 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 -- 172.21.15.138:0/3760032608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 msgr2=0x7f83401094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 --2- 172.21.15.138:0/3760032608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f83401094c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8330009900 tx=0x7f833002a410 comp rx=0 tx=0).stop 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 -- 172.21.15.138:0/3760032608 shutdown_connections 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 --2- 172.21.15.138:0/3760032608 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010f470 0x7f8340115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 --2- 172.21.15.138:0/3760032608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010eaf0 0x7f834010ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 --2- 172.21.15.138:0/3760032608 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f83401094c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 -- 172.21.15.138:0/3760032608 >> 172.21.15.138:0/3760032608 conn(0x7f8340078760 msgr2=0x7f834007ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:42.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 -- 172.21.15.138:0/3760032608 shutdown_connections 2024-08-20T01:07:42.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.521+0000 7f8346800640 1 -- 172.21.15.138:0/3760032608 wait complete. 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.522+0000 7f8346800640 1 Processor -- start 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 -- start start 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010eaf0 0x7f834019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010f470 0x7f83401a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f83401190f0 con 0x7f83401090e0 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8340118f70 con 0x7f834010f470 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f8346800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8340119270 con 0x7f834010eaf0 2024-08-20T01:07:42.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f833f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f833f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44840/0 (socket says 172.21.15.138:44840) 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f833f400640 1 -- 172.21.15.138:0/890052825 learned_addr learned my addr 172.21.15.138:0/890052825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f833ea00640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010eaf0 0x7f834019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.523+0000 7f833fe00640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010f470 0x7f83401a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 -- 172.21.15.138:0/890052825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010eaf0 msgr2=0x7f834019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010eaf0 0x7f834019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 -- 172.21.15.138:0/890052825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010f470 msgr2=0x7f83401a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010f470 0x7f83401a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 -- 172.21.15.138:0/890052825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83401a5670 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f833f400640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8330005ab0 tx=0x7f8330003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833003f070 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8330031d60 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330035030 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.524+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83401a5900 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.525+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83401ac020 con 0x7f83401090e0 2024-08-20T01:07:42.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.526+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83401094c0 con 0x7f83401090e0 2024-08-20T01:07:42.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.528+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f8330042070 con 0x7f83401090e0 2024-08-20T01:07:42.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.529+0000 7f8337e00640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 0x7f83240b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:42.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.529+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8330104f20 con 0x7f83401090e0 2024-08-20T01:07:42.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.529+0000 7f833ea00640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 0x7f83240b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:42.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.530+0000 7f833ea00640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 0x7f83240b60e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f832c0095f0 tx=0x7f832c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:42.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.531+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f83300cba50 con 0x7f83401090e0 2024-08-20T01:07:42.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.722+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83400632c0 con 0x7f83240b3c20 2024-08-20T01:07:42.721 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:42 smithi138 ceph-mon[18076]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:42.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.729+0000 7f8337e00640 1 -- 172.21.15.138:0/890052825 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340805 (secure 0 0 0) 0x7f83400632c0 con 0x7f83240b3c20 2024-08-20T01:07:42.732 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:42.733 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 msgr2=0x7f83240b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 0x7f83240b60e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f832c0095f0 tx=0x7f832c005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 msgr2=0x7f834019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8330005ab0 tx=0x7f8330003c80 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 shutdown_connections 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f83240b3c20 0x7f83240b60e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f834010f470 0x7f83401a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f834010eaf0 0x7f834019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 --2- 172.21.15.138:0/890052825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83401090e0 0x7f834019f5c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 >> 172.21.15.138:0/890052825 conn(0x7f8340078760 msgr2=0x7f8340078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 shutdown_connections 2024-08-20T01:07:42.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:42.734+0000 7f8346800640 1 -- 172.21.15.138:0/890052825 wait complete. 2024-08-20T01:07:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:42 smithi167 ceph-mon[21965]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:42 smithi142 ceph-mon[21799]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:43.471 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-08-20T01:07:42.508679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659611,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":589,"num_read_kb":397,"num_write":302,"num_write_kb":2278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352804,"kb_used_data":11876,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124589020,"statfs":{"total":1151940427776,"available":1151579156480,"internally_reserved":0,"allocated":12161024,"data_stored":8160923,"data_compressed":427158,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":68,"num_read_kb":33,"num_write":7,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003758"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621129+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621129+0000","last_peered":"2024-08-20T01:06:36.621129+0000","last_clean":"2024-08-20T01:06:36.621129+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621129+0000","last_undegraded":"2024-08-20T01:06:36.621129+0000","last_fullsized":"2024-08-20T01:06:36.621129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731638+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:07:34.731638+0000","last_peered":"2024-08-20T01:07:34.731638+0000","last_clean":"2024-08-20T01:07:34.731638+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:07:34.731638+0000","last_undegraded":"2024-08-20T01:07:34.731638+0000","last_fullsized":"2024-08-20T01:07:34.731638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426580+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.426580+0000","last_peered":"2024-08-20T01:06:37.426580+0000","last_clean":"2024-08-20T01:06:37.426580+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.426580+0000","last_undegraded":"2024-08-20T01:06:37.426580+0000","last_fullsized":"2024-08-20T01:06:37.426580+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564918+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564918+0000","last_peered":"2024-08-20T01:06:36.564918+0000","last_clean":"2024-08-20T01:06:36.564918+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564918+0000","last_undegraded":"2024-08-20T01:06:36.564918+0000","last_fullsized":"2024-08-20T01:06:36.564918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747371+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:07:34.747371+0000","last_peered":"2024-08-20T01:07:34.747371+0000","last_clean":"2024-08-20T01:07:34.747371+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:07:34.747371+0000","last_undegraded":"2024-08-20T01:07:34.747371+0000","last_fullsized":"2024-08-20T01:07:34.747371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564424+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564424+0000","last_peered":"2024-08-20T01:06:36.564424+0000","last_clean":"2024-08-20T01:06:36.564424+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564424+0000","last_undegraded":"2024-08-20T01:06:36.564424+0000","last_fullsized":"2024-08-20T01:06:36.564424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566559+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566559+0000","last_peered":"2024-08-20T01:06:36.566559+0000","last_clean":"2024-08-20T01:06:36.566559+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566559+0000","last_undegraded":"2024-08-20T01:06:36.566559+0000","last_fullsized":"2024-08-20T01:06:36.566559+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.744470+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:07:34.744470+0000","last_peered":"2024-08-20T01:07:34.744470+0000","last_clean":"2024-08-20T01:07:34.744470+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:07:34.744470+0000","last_undegraded":"2024-08-20T01:07:34.744470+0000","last_fullsized":"2024-08-20T01:07:34.744470+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649109+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649109+0000","last_peered":"2024-08-20T01:06:36.649109+0000","last_clean":"2024-08-20T01:06:36.649109+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649109+0000","last_undegraded":"2024-08-20T01:06:36.649109+0000","last_fullsized":"2024-08-20T01:06:36.649109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567277+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567277+0000","last_peered":"2024-08-20T01:06:36.567277+0000","last_clean":"2024-08-20T01:06:36.567277+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567277+0000","last_undegraded":"2024-08-20T01:06:36.567277+0000","last_fullsized":"2024-08-20T01:06:36.567277+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730170+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:07:34.730170+0000","last_peered":"2024-08-20T01:07:34.730170+0000","last_clean":"2024-08-20T01:07:34.730170+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:07:34.730170+0000","last_undegraded":"2024-08-20T01:07:34.730170+0000","last_fullsized":"2024-08-20T01:07:34.730170+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564564+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564564+0000","last_peered":"2024-08-20T01:06:36.564564+0000","last_clean":"2024-08-20T01:06:36.564564+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564564+0000","last_undegraded":"2024-08-20T01:06:36.564564+0000","last_fullsized":"2024-08-20T01:06:36.564564+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733459+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:07:34.733459+0000","last_peered":"2024-08-20T01:07:34.733459+0000","last_clean":"2024-08-20T01:07:34.733459+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:07:34.733459+0000","last_undegraded":"2024-08-20T01:07:34.733459+0000","last_fullsized":"2024-08-20T01:07:34.733459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564813+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564813+0000","last_peered":"2024-08-20T01:06:36.564813+0000","last_clean":"2024-08-20T01:06:36.564813+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564813+0000","last_undegraded":"2024-08-20T01:06:36.564813+0000","last_fullsized":"2024-08-20T01:06:36.564813+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739534+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:07:34.739534+0000","last_peered":"2024-08-20T01:07:34.739534+0000","last_clean":"2024-08-20T01:07:34.739534+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:07:34.739534+0000","last_undegraded":"2024-08-20T01:07:34.739534+0000","last_fullsized":"2024-08-20T01:07:34.739534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566999+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566999+0000","last_peered":"2024-08-20T01:06:36.566999+0000","last_clean":"2024-08-20T01:06:36.566999+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566999+0000","last_undegraded":"2024-08-20T01:06:36.566999+0000","last_fullsized":"2024-08-20T01:06:36.566999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564610+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564610+0000","last_peered":"2024-08-20T01:06:36.564610+0000","last_clean":"2024-08-20T01:06:36.564610+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564610+0000","last_undegraded":"2024-08-20T01:06:36.564610+0000","last_fullsized":"2024-08-20T01:06:36.564610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741181+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:07:34.741181+0000","last_peered":"2024-08-20T01:07:34.741181+0000","last_clean":"2024-08-20T01:07:34.741181+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:07:34.741181+0000","last_undegraded":"2024-08-20T01:07:34.741181+0000","last_fullsized":"2024-08-20T01:07:34.741181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564578+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564578+0000","last_peered":"2024-08-20T01:06:36.564578+0000","last_clean":"2024-08-20T01:06:36.564578+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564578+0000","last_undegraded":"2024-08-20T01:06:36.564578+0000","last_fullsized":"2024-08-20T01:06:36.564578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620901+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620901+0000","last_peered":"2024-08-20T01:06:36.620901+0000","last_clean":"2024-08-20T01:06:36.620901+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620901+0000","last_undegraded":"2024-08-20T01:06:36.620901+0000","last_fullsized":"2024-08-20T01:06:36.620901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727978+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:07:34.727978+0000","last_peered":"2024-08-20T01:07:34.727978+0000","last_clean":"2024-08-20T01:07:34.727978+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:07:34.727978+0000","last_undegraded":"2024-08-20T01:07:34.727978+0000","last_fullsized":"2024-08-20T01:07:34.727978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642277+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642277+0000","last_peered":"2024-08-20T01:06:36.642277+0000","last_clean":"2024-08-20T01:06:36.642277+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642277+0000","last_undegraded":"2024-08-20T01:06:36.642277+0000","last_fullsized":"2024-08-20T01:06:36.642277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564951+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.564951+0000","last_peered":"2024-08-20T01:06:36.564951+0000","last_clean":"2024-08-20T01:06:36.564951+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.564951+0000","last_undegraded":"2024-08-20T01:06:36.564951+0000","last_fullsized":"2024-08-20T01:06:36.564951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738711+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:07:34.738711+0000","last_peered":"2024-08-20T01:07:34.738711+0000","last_clean":"2024-08-20T01:07:34.738711+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:07:34.738711+0000","last_undegraded":"2024-08-20T01:07:34.738711+0000","last_fullsized":"2024-08-20T01:07:34.738711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565514+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565514+0000","last_peered":"2024-08-20T01:06:36.565514+0000","last_clean":"2024-08-20T01:06:36.565514+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565514+0000","last_undegraded":"2024-08-20T01:06:36.565514+0000","last_fullsized":"2024-08-20T01:06:36.565514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567004+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.567004+0000","last_peered":"2024-08-20T01:06:36.567004+0000","last_clean":"2024-08-20T01:06:36.567004+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.567004+0000","last_undegraded":"2024-08-20T01:06:36.567004+0000","last_fullsized":"2024-08-20T01:06:36.567004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727759+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:07:34.727759+0000","last_peered":"2024-08-20T01:07:34.727759+0000","last_clean":"2024-08-20T01:07:34.727759+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:07:34.727759+0000","last_undegraded":"2024-08-20T01:07:34.727759+0000","last_fullsized":"2024-08-20T01:07:34.727759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.313638+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:07:34.313638+0000","last_peered":"2024-08-20T01:07:34.313638+0000","last_clean":"2024-08-20T01:07:34.313638+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:07:34.313638+0000","last_undegraded":"2024-08-20T01:07:34.313638+0000","last_fullsized":"2024-08-20T01:07:34.313638+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564152+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564152+0000","last_peered":"2024-08-20T01:06:36.564152+0000","last_clean":"2024-08-20T01:06:36.564152+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564152+0000","last_undegraded":"2024-08-20T01:06:36.564152+0000","last_fullsized":"2024-08-20T01:06:36.564152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729501+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:07:34.729501+0000","last_peered":"2024-08-20T01:07:34.729501+0000","last_clean":"2024-08-20T01:07:34.729501+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:07:34.729501+0000","last_undegraded":"2024-08-20T01:07:34.729501+0000","last_fullsized":"2024-08-20T01:07:34.729501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566711+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566711+0000","last_peered":"2024-08-20T01:06:36.566711+0000","last_clean":"2024-08-20T01:06:36.566711+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566711+0000","last_undegraded":"2024-08-20T01:06:36.566711+0000","last_fullsized":"2024-08-20T01:06:36.566711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338363+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338363+0000","last_peered":"2024-08-20T01:06:37.338363+0000","last_clean":"2024-08-20T01:06:37.338363+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338363+0000","last_undegraded":"2024-08-20T01:06:37.338363+0000","last_fullsized":"2024-08-20T01:06:37.338363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732734+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:07:34.732734+0000","last_peered":"2024-08-20T01:07:34.732734+0000","last_clean":"2024-08-20T01:07:34.732734+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:07:34.732734+0000","last_undegraded":"2024-08-20T01:07:34.732734+0000","last_fullsized":"2024-08-20T01:07:34.732734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620842+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620842+0000","last_peered":"2024-08-20T01:06:36.620842+0000","last_clean":"2024-08-20T01:06:36.620842+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620842+0000","last_undegraded":"2024-08-20T01:06:36.620842+0000","last_fullsized":"2024-08-20T01:06:36.620842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566765+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566765+0000","last_peered":"2024-08-20T01:06:36.566765+0000","last_clean":"2024-08-20T01:06:36.566765+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566765+0000","last_undegraded":"2024-08-20T01:06:36.566765+0000","last_fullsized":"2024-08-20T01:06:36.566765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740243+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:07:34.740243+0000","last_peered":"2024-08-20T01:07:34.740243+0000","last_clean":"2024-08-20T01:07:34.740243+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:07:34.740243+0000","last_undegraded":"2024-08-20T01:07:34.740243+0000","last_fullsized":"2024-08-20T01:07:34.740243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900778+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900778+0000","last_peered":"2024-08-20T01:06:36.900778+0000","last_clean":"2024-08-20T01:06:36.900778+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900778+0000","last_undegraded":"2024-08-20T01:06:36.900778+0000","last_fullsized":"2024-08-20T01:06:36.900778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566915+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566915+0000","last_peered":"2024-08-20T01:06:36.566915+0000","last_clean":"2024-08-20T01:06:36.566915+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566915+0000","last_undegraded":"2024-08-20T01:06:36.566915+0000","last_fullsized":"2024-08-20T01:06:36.566915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731513+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:07:34.731513+0000","last_peered":"2024-08-20T01:07:34.731513+0000","last_clean":"2024-08-20T01:07:34.731513+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:07:34.731513+0000","last_undegraded":"2024-08-20T01:07:34.731513+0000","last_fullsized":"2024-08-20T01:07:34.731513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:38.739543+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:07:38.739543+0000","last_peered":"2024-08-20T01:07:38.739543+0000","last_clean":"2024-08-20T01:07:38.739543+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:07:38.739543+0000","last_undegraded":"2024-08-20T01:07:38.739543+0000","last_fullsized":"2024-08-20T01:07:38.739543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":80,"num_read_kb":80,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567468+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567468+0000","last_peered":"2024-08-20T01:06:36.567468+0000","last_clean":"2024-08-20T01:06:36.567468+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567468+0000","last_undegraded":"2024-08-20T01:06:36.567468+0000","last_fullsized":"2024-08-20T01:06:36.567468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727983+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:07:34.727983+0000","last_peered":"2024-08-20T01:07:34.727983+0000","last_clean":"2024-08-20T01:07:34.727983+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:07:34.727983+0000","last_undegraded":"2024-08-20T01:07:34.727983+0000","last_fullsized":"2024-08-20T01:07:34.727983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900807+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900807+0000","last_peered":"2024-08-20T01:06:36.900807+0000","last_clean":"2024-08-20T01:06:36.900807+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900807+0000","last_undegraded":"2024-08-20T01:06:36.900807+0000","last_fullsized":"2024-08-20T01:06:36.900807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647618+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647618+0000","last_peered":"2024-08-20T01:06:36.647618+0000","last_clean":"2024-08-20T01:06:36.647618+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647618+0000","last_undegraded":"2024-08-20T01:06:36.647618+0000","last_fullsized":"2024-08-20T01:06:36.647618+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747444+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:07:34.747444+0000","last_peered":"2024-08-20T01:07:34.747444+0000","last_clean":"2024-08-20T01:07:34.747444+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:07:34.747444+0000","last_undegraded":"2024-08-20T01:07:34.747444+0000","last_fullsized":"2024-08-20T01:07:34.747444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565335+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565335+0000","last_peered":"2024-08-20T01:06:36.565335+0000","last_clean":"2024-08-20T01:06:36.565335+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565335+0000","last_undegraded":"2024-08-20T01:06:36.565335+0000","last_fullsized":"2024-08-20T01:06:36.565335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900775+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900775+0000","last_peered":"2024-08-20T01:06:36.900775+0000","last_clean":"2024-08-20T01:06:36.900775+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900775+0000","last_undegraded":"2024-08-20T01:06:36.900775+0000","last_fullsized":"2024-08-20T01:06:36.900775+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731812+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:07:34.731812+0000","last_peered":"2024-08-20T01:07:34.731812+0000","last_clean":"2024-08-20T01:07:34.731812+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:07:34.731812+0000","last_undegraded":"2024-08-20T01:07:34.731812+0000","last_fullsized":"2024-08-20T01:07:34.731812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567812+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567812+0000","last_peered":"2024-08-20T01:06:36.567812+0000","last_clean":"2024-08-20T01:06:36.567812+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567812+0000","last_undegraded":"2024-08-20T01:06:36.567812+0000","last_fullsized":"2024-08-20T01:06:36.567812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642176+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642176+0000","last_peered":"2024-08-20T01:06:36.642176+0000","last_clean":"2024-08-20T01:06:36.642176+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642176+0000","last_undegraded":"2024-08-20T01:06:36.642176+0000","last_fullsized":"2024-08-20T01:06:36.642176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728844+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:07:34.728844+0000","last_peered":"2024-08-20T01:07:34.728844+0000","last_clean":"2024-08-20T01:07:34.728844+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:07:34.728844+0000","last_undegraded":"2024-08-20T01:07:34.728844+0000","last_fullsized":"2024-08-20T01:07:34.728844+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647722+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647722+0000","last_peered":"2024-08-20T01:06:36.647722+0000","last_clean":"2024-08-20T01:06:36.647722+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647722+0000","last_undegraded":"2024-08-20T01:06:36.647722+0000","last_fullsized":"2024-08-20T01:06:36.647722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425060+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425060+0000","last_peered":"2024-08-20T01:06:37.425060+0000","last_clean":"2024-08-20T01:06:37.425060+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425060+0000","last_undegraded":"2024-08-20T01:06:37.425060+0000","last_fullsized":"2024-08-20T01:06:37.425060+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731203+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:07:34.731203+0000","last_peered":"2024-08-20T01:07:34.731203+0000","last_clean":"2024-08-20T01:07:34.731203+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:07:34.731203+0000","last_undegraded":"2024-08-20T01:07:34.731203+0000","last_fullsized":"2024-08-20T01:07:34.731203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646880+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646880+0000","last_peered":"2024-08-20T01:06:36.646880+0000","last_clean":"2024-08-20T01:06:36.646880+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646880+0000","last_undegraded":"2024-08-20T01:06:36.646880+0000","last_fullsized":"2024-08-20T01:06:36.646880+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738945+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:07:34.738945+0000","last_peered":"2024-08-20T01:07:34.738945+0000","last_clean":"2024-08-20T01:07:34.738945+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:07:34.738945+0000","last_undegraded":"2024-08-20T01:07:34.738945+0000","last_fullsized":"2024-08-20T01:07:34.738945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566920+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566920+0000","last_peered":"2024-08-20T01:06:36.566920+0000","last_clean":"2024-08-20T01:06:36.566920+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566920+0000","last_undegraded":"2024-08-20T01:06:36.566920+0000","last_fullsized":"2024-08-20T01:06:36.566920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731672+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:07:34.731672+0000","last_peered":"2024-08-20T01:07:34.731672+0000","last_clean":"2024-08-20T01:07:34.731672+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:07:34.731672+0000","last_undegraded":"2024-08-20T01:07:34.731672+0000","last_fullsized":"2024-08-20T01:07:34.731672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338120+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338120+0000","last_peered":"2024-08-20T01:06:37.338120+0000","last_clean":"2024-08-20T01:06:37.338120+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338120+0000","last_undegraded":"2024-08-20T01:06:37.338120+0000","last_fullsized":"2024-08-20T01:06:37.338120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340111+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340111+0000","last_peered":"2024-08-20T01:06:37.340111+0000","last_clean":"2024-08-20T01:06:37.340111+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340111+0000","last_undegraded":"2024-08-20T01:06:37.340111+0000","last_fullsized":"2024-08-20T01:06:37.340111+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724032+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:07:34.724032+0000","last_peered":"2024-08-20T01:07:34.724032+0000","last_clean":"2024-08-20T01:07:34.724032+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:07:34.724032+0000","last_undegraded":"2024-08-20T01:07:34.724032+0000","last_fullsized":"2024-08-20T01:07:34.724032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":126,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:38.740135+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:07:38.740135+0000","last_peered":"2024-08-20T01:07:38.740135+0000","last_clean":"2024-08-20T01:07:38.740135+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:07:38.740135+0000","last_undegraded":"2024-08-20T01:07:38.740135+0000","last_fullsized":"2024-08-20T01:07:38.740135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620964+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.620964+0000","last_peered":"2024-08-20T01:06:36.620964+0000","last_clean":"2024-08-20T01:06:36.620964+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.620964+0000","last_undegraded":"2024-08-20T01:06:36.620964+0000","last_fullsized":"2024-08-20T01:06:36.620964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739852+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:07:34.739852+0000","last_peered":"2024-08-20T01:07:34.739852+0000","last_clean":"2024-08-20T01:07:34.739852+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:07:34.739852+0000","last_undegraded":"2024-08-20T01:07:34.739852+0000","last_fullsized":"2024-08-20T01:07:34.739852+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647397+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647397+0000","last_peered":"2024-08-20T01:06:36.647397+0000","last_clean":"2024-08-20T01:06:36.647397+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647397+0000","last_undegraded":"2024-08-20T01:06:36.647397+0000","last_fullsized":"2024-08-20T01:06:36.647397+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338281+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338281+0000","last_peered":"2024-08-20T01:06:37.338281+0000","last_clean":"2024-08-20T01:06:37.338281+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338281+0000","last_undegraded":"2024-08-20T01:06:37.338281+0000","last_fullsized":"2024-08-20T01:06:37.338281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724968+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:07:34.724968+0000","last_peered":"2024-08-20T01:07:34.724968+0000","last_clean":"2024-08-20T01:07:34.724968+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:07:34.724968+0000","last_undegraded":"2024-08-20T01:07:34.724968+0000","last_fullsized":"2024-08-20T01:07:34.724968+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566684+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566684+0000","last_peered":"2024-08-20T01:06:36.566684+0000","last_clean":"2024-08-20T01:06:36.566684+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566684+0000","last_undegraded":"2024-08-20T01:06:36.566684+0000","last_fullsized":"2024-08-20T01:06:36.566684+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647354+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647354+0000","last_peered":"2024-08-20T01:06:36.647354+0000","last_clean":"2024-08-20T01:06:36.647354+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647354+0000","last_undegraded":"2024-08-20T01:06:36.647354+0000","last_fullsized":"2024-08-20T01:06:36.647354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.718438+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:07:34.718438+0000","last_peered":"2024-08-20T01:07:34.718438+0000","last_clean":"2024-08-20T01:07:34.718438+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:07:34.718438+0000","last_undegraded":"2024-08-20T01:07:34.718438+0000","last_fullsized":"2024-08-20T01:07:34.718438+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'38","reported_seq":249,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:38.740926+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:07:38.740926+0000","last_peered":"2024-08-20T01:07:38.740926+0000","last_clean":"2024-08-20T01:07:38.740926+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:07:38.740926+0000","last_undegraded":"2024-08-20T01:07:38.740926+0000","last_fullsized":"2024-08-20T01:07:38.740926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64867,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":149,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620220+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620220+0000","last_peered":"2024-08-20T01:06:36.620220+0000","last_clean":"2024-08-20T01:06:36.620220+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620220+0000","last_undegraded":"2024-08-20T01:06:36.620220+0000","last_fullsized":"2024-08-20T01:06:36.620220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747351+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:07:34.747351+0000","last_peered":"2024-08-20T01:07:34.747351+0000","last_clean":"2024-08-20T01:07:34.747351+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:07:34.747351+0000","last_undegraded":"2024-08-20T01:07:34.747351+0000","last_fullsized":"2024-08-20T01:07:34.747351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647649+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647649+0000","last_peered":"2024-08-20T01:06:36.647649+0000","last_clean":"2024-08-20T01:06:36.647649+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647649+0000","last_undegraded":"2024-08-20T01:06:36.647649+0000","last_fullsized":"2024-08-20T01:06:36.647649+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564971+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.564971+0000","last_peered":"2024-08-20T01:06:36.564971+0000","last_clean":"2024-08-20T01:06:36.564971+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.564971+0000","last_undegraded":"2024-08-20T01:06:36.564971+0000","last_fullsized":"2024-08-20T01:06:36.564971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739755+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:07:34.739755+0000","last_peered":"2024-08-20T01:07:34.739755+0000","last_clean":"2024-08-20T01:07:34.739755+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:07:34.739755+0000","last_undegraded":"2024-08-20T01:07:34.739755+0000","last_fullsized":"2024-08-20T01:07:34.739755+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900942+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.900942+0000","last_peered":"2024-08-20T01:06:36.900942+0000","last_clean":"2024-08-20T01:06:36.900942+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.900942+0000","last_undegraded":"2024-08-20T01:06:36.900942+0000","last_fullsized":"2024-08-20T01:06:36.900942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620103+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620103+0000","last_peered":"2024-08-20T01:06:36.620103+0000","last_clean":"2024-08-20T01:06:36.620103+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620103+0000","last_undegraded":"2024-08-20T01:06:36.620103+0000","last_fullsized":"2024-08-20T01:06:36.620103+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.745045+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:07:34.745045+0000","last_peered":"2024-08-20T01:07:34.745045+0000","last_clean":"2024-08-20T01:07:34.745045+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:07:34.745045+0000","last_undegraded":"2024-08-20T01:07:34.745045+0000","last_fullsized":"2024-08-20T01:07:34.745045+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":124,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:37.739306+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:07:37.739306+0000","last_peered":"2024-08-20T01:07:37.739306+0000","last_clean":"2024-08-20T01:07:37.739306+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:07:37.739306+0000","last_undegraded":"2024-08-20T01:07:37.739306+0000","last_fullsized":"2024-08-20T01:07:37.739306+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564729+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564729+0000","last_peered":"2024-08-20T01:06:36.564729+0000","last_clean":"2024-08-20T01:06:36.564729+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564729+0000","last_undegraded":"2024-08-20T01:06:36.564729+0000","last_fullsized":"2024-08-20T01:06:36.564729+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425144+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425144+0000","last_peered":"2024-08-20T01:06:37.425144+0000","last_clean":"2024-08-20T01:06:37.425144+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425144+0000","last_undegraded":"2024-08-20T01:06:37.425144+0000","last_fullsized":"2024-08-20T01:06:37.425144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731533+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:07:34.731533+0000","last_peered":"2024-08-20T01:07:34.731533+0000","last_clean":"2024-08-20T01:07:34.731533+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:07:34.731533+0000","last_undegraded":"2024-08-20T01:07:34.731533+0000","last_fullsized":"2024-08-20T01:07:34.731533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647281+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647281+0000","last_peered":"2024-08-20T01:06:36.647281+0000","last_clean":"2024-08-20T01:06:36.647281+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647281+0000","last_undegraded":"2024-08-20T01:06:36.647281+0000","last_fullsized":"2024-08-20T01:06:36.647281+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338530+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338530+0000","last_peered":"2024-08-20T01:06:37.338530+0000","last_clean":"2024-08-20T01:06:37.338530+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338530+0000","last_undegraded":"2024-08-20T01:06:37.338530+0000","last_fullsized":"2024-08-20T01:06:37.338530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728608+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:07:34.728608+0000","last_peered":"2024-08-20T01:07:34.728608+0000","last_clean":"2024-08-20T01:07:34.728608+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:07:34.728608+0000","last_undegraded":"2024-08-20T01:07:34.728608+0000","last_fullsized":"2024-08-20T01:07:34.728608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566458+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566458+0000","last_peered":"2024-08-20T01:06:36.566458+0000","last_clean":"2024-08-20T01:06:36.566458+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566458+0000","last_undegraded":"2024-08-20T01:06:36.566458+0000","last_fullsized":"2024-08-20T01:06:36.566458+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644313+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644313+0000","last_peered":"2024-08-20T01:06:36.644313+0000","last_clean":"2024-08-20T01:06:36.644313+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644313+0000","last_undegraded":"2024-08-20T01:06:36.644313+0000","last_fullsized":"2024-08-20T01:06:36.644313+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730294+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:07:34.730294+0000","last_peered":"2024-08-20T01:07:34.730294+0000","last_clean":"2024-08-20T01:07:34.730294+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:07:34.730294+0000","last_undegraded":"2024-08-20T01:07:34.730294+0000","last_fullsized":"2024-08-20T01:07:34.730294+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566490+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566490+0000","last_peered":"2024-08-20T01:06:36.566490+0000","last_clean":"2024-08-20T01:06:36.566490+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566490+0000","last_undegraded":"2024-08-20T01:06:36.566490+0000","last_fullsized":"2024-08-20T01:06:36.566490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:37.739916+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:07:37.739916+0000","last_peered":"2024-08-20T01:07:37.739916+0000","last_clean":"2024-08-20T01:07:37.739916+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:07:37.739916+0000","last_undegraded":"2024-08-20T01:07:37.739916+0000","last_fullsized":"2024-08-20T01:07:37.739916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730611+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:07:34.730611+0000","last_peered":"2024-08-20T01:07:34.730611+0000","last_clean":"2024-08-20T01:07:34.730611+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:07:34.730611+0000","last_undegraded":"2024-08-20T01:07:34.730611+0000","last_fullsized":"2024-08-20T01:07:34.730611+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567358+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567358+0000","last_peered":"2024-08-20T01:06:36.567358+0000","last_clean":"2024-08-20T01:06:36.567358+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567358+0000","last_undegraded":"2024-08-20T01:06:36.567358+0000","last_fullsized":"2024-08-20T01:06:36.567358+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732743+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:07:34.732743+0000","last_peered":"2024-08-20T01:07:34.732743+0000","last_clean":"2024-08-20T01:07:34.732743+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:07:34.732743+0000","last_undegraded":"2024-08-20T01:07:34.732743+0000","last_fullsized":"2024-08-20T01:07:34.732743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733589+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:07:34.733589+0000","last_peered":"2024-08-20T01:07:34.733589+0000","last_clean":"2024-08-20T01:07:34.733589+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:07:34.733589+0000","last_undegraded":"2024-08-20T01:07:34.733589+0000","last_fullsized":"2024-08-20T01:07:34.733589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739063+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:07:34.739063+0000","last_peered":"2024-08-20T01:07:34.739063+0000","last_clean":"2024-08-20T01:07:34.739063+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:07:34.739063+0000","last_undegraded":"2024-08-20T01:07:34.739063+0000","last_fullsized":"2024-08-20T01:07:34.739063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739518+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:07:34.739518+0000","last_peered":"2024-08-20T01:07:34.739518+0000","last_clean":"2024-08-20T01:07:34.739518+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:07:34.739518+0000","last_undegraded":"2024-08-20T01:07:34.739518+0000","last_fullsized":"2024-08-20T01:07:34.739518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740881+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:07:34.740881+0000","last_peered":"2024-08-20T01:07:34.740881+0000","last_clean":"2024-08-20T01:07:34.740881+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:07:34.740881+0000","last_undegraded":"2024-08-20T01:07:34.740881+0000","last_fullsized":"2024-08-20T01:07:34.740881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729504+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:07:34.729504+0000","last_peered":"2024-08-20T01:07:34.729504+0000","last_clean":"2024-08-20T01:07:34.729504+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:07:34.729504+0000","last_undegraded":"2024-08-20T01:07:34.729504+0000","last_fullsized":"2024-08-20T01:07:34.729504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725362+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:07:34.725362+0000","last_peered":"2024-08-20T01:07:34.725362+0000","last_clean":"2024-08-20T01:07:34.725362+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:07:34.725362+0000","last_undegraded":"2024-08-20T01:07:34.725362+0000","last_fullsized":"2024-08-20T01:07:34.725362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728179+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:07:34.728179+0000","last_peered":"2024-08-20T01:07:34.728179+0000","last_clean":"2024-08-20T01:07:34.728179+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:07:34.728179+0000","last_undegraded":"2024-08-20T01:07:34.728179+0000","last_fullsized":"2024-08-20T01:07:34.728179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732606+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:07:34.732606+0000","last_peered":"2024-08-20T01:07:34.732606+0000","last_clean":"2024-08-20T01:07:34.732606+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:07:34.732606+0000","last_undegraded":"2024-08-20T01:07:34.732606+0000","last_fullsized":"2024-08-20T01:07:34.732606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729178+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:07:34.729178+0000","last_peered":"2024-08-20T01:07:34.729178+0000","last_clean":"2024-08-20T01:07:34.729178+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:07:34.729178+0000","last_undegraded":"2024-08-20T01:07:34.729178+0000","last_fullsized":"2024-08-20T01:07:34.729178+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747342+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:07:34.747342+0000","last_peered":"2024-08-20T01:07:34.747342+0000","last_clean":"2024-08-20T01:07:34.747342+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:07:34.747342+0000","last_undegraded":"2024-08-20T01:07:34.747342+0000","last_fullsized":"2024-08-20T01:07:34.747342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741357+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:07:34.741357+0000","last_peered":"2024-08-20T01:07:34.741357+0000","last_clean":"2024-08-20T01:07:34.741357+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:07:34.741357+0000","last_undegraded":"2024-08-20T01:07:34.741357+0000","last_fullsized":"2024-08-20T01:07:34.741357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731379+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:07:34.731379+0000","last_peered":"2024-08-20T01:07:34.731379+0000","last_clean":"2024-08-20T01:07:34.731379+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:07:34.731379+0000","last_undegraded":"2024-08-20T01:07:34.731379+0000","last_fullsized":"2024-08-20T01:07:34.731379+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732616+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:07:34.732616+0000","last_peered":"2024-08-20T01:07:34.732616+0000","last_clean":"2024-08-20T01:07:34.732616+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:07:34.732616+0000","last_undegraded":"2024-08-20T01:07:34.732616+0000","last_fullsized":"2024-08-20T01:07:34.732616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.735358+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:07:34.735358+0000","last_peered":"2024-08-20T01:07:34.735358+0000","last_clean":"2024-08-20T01:07:34.735358+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:07:34.735358+0000","last_undegraded":"2024-08-20T01:07:34.735358+0000","last_fullsized":"2024-08-20T01:07:34.735358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741881+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:07:34.741881+0000","last_peered":"2024-08-20T01:07:34.741881+0000","last_clean":"2024-08-20T01:07:34.741881+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:07:34.741881+0000","last_undegraded":"2024-08-20T01:07:34.741881+0000","last_fullsized":"2024-08-20T01:07:34.741881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746898+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:07:34.746898+0000","last_peered":"2024-08-20T01:07:34.746898+0000","last_clean":"2024-08-20T01:07:34.746898+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:07:34.746898+0000","last_undegraded":"2024-08-20T01:07:34.746898+0000","last_fullsized":"2024-08-20T01:07:34.746898+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740158+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:07:34.740158+0000","last_peered":"2024-08-20T01:07:34.740158+0000","last_clean":"2024-08-20T01:07:34.740158+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:07:34.740158+0000","last_undegraded":"2024-08-20T01:07:34.740158+0000","last_fullsized":"2024-08-20T01:07:34.740158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738949+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:07:34.738949+0000","last_peered":"2024-08-20T01:07:34.738949+0000","last_clean":"2024-08-20T01:07:34.738949+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:07:34.738949+0000","last_undegraded":"2024-08-20T01:07:34.738949+0000","last_fullsized":"2024-08-20T01:07:34.738949+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.736362+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:07:34.736362+0000","last_peered":"2024-08-20T01:07:34.736362+0000","last_clean":"2024-08-20T01:07:34.736362+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:07:34.736362+0000","last_undegraded":"2024-08-20T01:07:34.736362+0000","last_fullsized":"2024-08-20T01:07:34.736362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730265+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:07:34.730265+0000","last_peered":"2024-08-20T01:07:34.730265+0000","last_clean":"2024-08-20T01:07:34.730265+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:07:34.730265+0000","last_undegraded":"2024-08-20T01:07:34.730265+0000","last_fullsized":"2024-08-20T01:07:34.730265+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730892+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:07:34.730892+0000","last_peered":"2024-08-20T01:07:34.730892+0000","last_clean":"2024-08-20T01:07:34.730892+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:07:34.730892+0000","last_undegraded":"2024-08-20T01:07:34.730892+0000","last_fullsized":"2024-08-20T01:07:34.730892+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739871+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:07:34.739871+0000","last_peered":"2024-08-20T01:07:34.739871+0000","last_clean":"2024-08-20T01:07:34.739871+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:07:34.739871+0000","last_undegraded":"2024-08-20T01:07:34.739871+0000","last_fullsized":"2024-08-20T01:07:34.739871+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.743763+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:07:34.743763+0000","last_peered":"2024-08-20T01:07:34.743763+0000","last_clean":"2024-08-20T01:07:34.743763+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:07:34.743763+0000","last_undegraded":"2024-08-20T01:07:34.743763+0000","last_fullsized":"2024-08-20T01:07:34.743763+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734211+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:07:34.734211+0000","last_peered":"2024-08-20T01:07:34.734211+0000","last_clean":"2024-08-20T01:07:34.734211+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:07:34.734211+0000","last_undegraded":"2024-08-20T01:07:34.734211+0000","last_fullsized":"2024-08-20T01:07:34.734211+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729816+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:07:34.729816+0000","last_peered":"2024-08-20T01:07:34.729816+0000","last_clean":"2024-08-20T01:07:34.729816+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:07:34.729816+0000","last_undegraded":"2024-08-20T01:07:34.729816+0000","last_fullsized":"2024-08-20T01:07:34.729816+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740550+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:07:34.740550+0000","last_peered":"2024-08-20T01:07:34.740550+0000","last_clean":"2024-08-20T01:07:34.740550+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:07:34.740550+0000","last_undegraded":"2024-08-20T01:07:34.740550+0000","last_fullsized":"2024-08-20T01:07:34.740550+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729779+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:07:34.729779+0000","last_peered":"2024-08-20T01:07:34.729779+0000","last_clean":"2024-08-20T01:07:34.729779+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:07:34.729779+0000","last_undegraded":"2024-08-20T01:07:34.729779+0000","last_fullsized":"2024-08-20T01:07:34.729779+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733118+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:07:34.733118+0000","last_peered":"2024-08-20T01:07:34.733118+0000","last_clean":"2024-08-20T01:07:34.733118+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:07:34.733118+0000","last_undegraded":"2024-08-20T01:07:34.733118+0000","last_fullsized":"2024-08-20T01:07:34.733118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747505+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:07:34.747505+0000","last_peered":"2024-08-20T01:07:34.747505+0000","last_clean":"2024-08-20T01:07:34.747505+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:07:34.747505+0000","last_undegraded":"2024-08-20T01:07:34.747505+0000","last_fullsized":"2024-08-20T01:07:34.747505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739436+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:07:34.739436+0000","last_peered":"2024-08-20T01:07:34.739436+0000","last_clean":"2024-08-20T01:07:34.739436+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:07:34.739436+0000","last_undegraded":"2024-08-20T01:07:34.739436+0000","last_fullsized":"2024-08-20T01:07:34.739436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742422+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:07:34.742422+0000","last_peered":"2024-08-20T01:07:34.742422+0000","last_clean":"2024-08-20T01:07:34.742422+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:07:34.742422+0000","last_undegraded":"2024-08-20T01:07:34.742422+0000","last_fullsized":"2024-08-20T01:07:34.742422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69009,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":508,"num_read_kb":328,"num_write":177,"num_write_kb":126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":207027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151,"ondisk_log_size":151,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678057,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874286,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1169315,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037811,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":991232,"data_stored":521512,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234041,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430269,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1113386,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593796,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1108974,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":512786,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378713,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":579251,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542236,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":577369,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705762,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":64867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:07:43.476 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T01:07:43.477 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-08-20T01:07:43.477 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-08-20T01:07:43.477 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph health --format=json 2024-08-20T01:07:43.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:43 smithi138 ceph-mon[18076]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-08-20T01:07:43.848 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:43.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:43 smithi167 ceph-mon[21965]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-08-20T01:07:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:43 smithi142 ceph-mon[21799]: pgmap v350: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-08-20T01:07:44.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 -- 172.21.15.138:0/2304302345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2010a520 msgr2=0x7f5d2010a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 --2- 172.21.15.138:0/2304302345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2010a520 0x7f5d2010a960 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009900 tx=0x7f5d0c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 -- 172.21.15.138:0/2304302345 shutdown_connections 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 --2- 172.21.15.138:0/2304302345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 0x7f5d201116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 --2- 172.21.15.138:0/2304302345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d2010a520 0x7f5d2010a960 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 --2- 172.21.15.138:0/2304302345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d20104b10 0x7f5d20104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 -- 172.21.15.138:0/2304302345 >> 172.21.15.138:0/2304302345 conn(0x7f5d20100540 msgr2=0x7f5d20102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 -- 172.21.15.138:0/2304302345 shutdown_connections 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.471+0000 7f5d1f200640 1 -- 172.21.15.138:0/2304302345 wait complete. 2024-08-20T01:07:44.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.472+0000 7f5d1f200640 1 Processor -- start 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 -- start start 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2010a520 0x7f5d2019b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 0x7f5d201a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20114b70 con 0x7f5d20104b10 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5d201149f0 con 0x7f5d2010a520 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5d20114cf0 con 0x7f5d2010aea0 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44848/0 (socket says 172.21.15.138:44848) 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1de00640 1 -- 172.21.15.138:0/1061749812 learned_addr learned my addr 172.21.15.138:0/1061749812 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1e800640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 0x7f5d201a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:44.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.473+0000 7f5d1d400640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2010a520 0x7f5d2019b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 -- 172.21.15.138:0/1061749812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 msgr2=0x7f5d201a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 0x7f5d201a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 -- 172.21.15.138:0/1061749812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2010a520 msgr2=0x7f5d2019b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2010a520 0x7f5d2019b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 -- 172.21.15.138:0/1061749812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d201a0ff0 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1de00640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ea60 tx=0x7f5d1000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000cde0 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100035f0 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d201a12e0 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.474+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10010690 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.475+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d201a7b70 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.476+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d20104ef0 con 0x7f5d20104b10 2024-08-20T01:07:44.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.478+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f5d100037a0 con 0x7f5d20104b10 2024-08-20T01:07:44.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.480+0000 7f5d16a00640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 0x7f5d040b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:44.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.480+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5d100e4470 con 0x7f5d20104b10 2024-08-20T01:07:44.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.480+0000 7f5d1d400640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 0x7f5d040b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:44.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.481+0000 7f5d1d400640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 0x7f5d040b6200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:44.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.482+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5d100ab5f0 con 0x7f5d20104b10 2024-08-20T01:07:44.702 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.704+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5d2019c070 con 0x7f5d20104b10 2024-08-20T01:07:44.702 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:44 smithi138 ceph-mon[18076]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:44.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.704+0000 7f5d16a00640 1 -- 172.21.15.138:0/1061749812 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5d2019c070 con 0x7f5d20104b10 2024-08-20T01:07:44.703 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:44.703 INFO:teuthology.orchestra.run.smithi138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-08-20T01:07:44.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 msgr2=0x7f5d040b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:44.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 0x7f5d040b6200 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c005ab0 tx=0x7f5d0c005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:44.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 msgr2=0x7f5d2019af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ea60 tx=0x7f5d1000ef20 comp rx=0 tx=0).stop 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 shutdown_connections 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d040b3d40 0x7f5d040b6200 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5d2010aea0 0x7f5d201a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d2010a520 0x7f5d2019b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 --2- 172.21.15.138:0/1061749812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d20104b10 0x7f5d2019af40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 >> 172.21.15.138:0/1061749812 conn(0x7f5d20100540 msgr2=0x7f5d20102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 shutdown_connections 2024-08-20T01:07:44.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:44.707+0000 7f5d1f200640 1 -- 172.21.15.138:0/1061749812 wait complete. 2024-08-20T01:07:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:44 smithi167 ceph-mon[21965]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:44 smithi142 ceph-mon[21799]: from='client.15141 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:07:45.536 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-08-20T01:07:45.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:07:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:45 smithi138 ceph-mon[18076]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1061749812' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T01:07:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:45 smithi167 ceph-mon[21965]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:45.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:45 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1061749812' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T01:07:45.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:45 smithi142 ceph-mon[21799]: pgmap v351: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:45.892 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:45 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1061749812' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-08-20T01:07:45.904 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 -- 172.21.15.138:0/319465378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010ae90 msgr2=0x7f5aa0111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 --2- 172.21.15.138:0/319465378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010ae90 0x7f5aa0111690 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c009ca0 tx=0x7f5a9c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 -- 172.21.15.138:0/319465378 shutdown_connections 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 --2- 172.21.15.138:0/319465378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010ae90 0x7f5aa0111690 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 --2- 172.21.15.138:0/319465378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa010a510 0x7f5aa010a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 --2- 172.21.15.138:0/319465378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa0104b00 0x7f5aa0104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.488+0000 7f5aa7200640 1 -- 172.21.15.138:0/319465378 >> 172.21.15.138:0/319465378 conn(0x7f5aa0100550 msgr2=0x7f5aa0102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.489+0000 7f5aa7200640 1 -- 172.21.15.138:0/319465378 shutdown_connections 2024-08-20T01:07:46.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.489+0000 7f5aa7200640 1 -- 172.21.15.138:0/319465378 wait complete. 2024-08-20T01:07:46.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.490+0000 7f5aa7200640 1 Processor -- start 2024-08-20T01:07:46.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.490+0000 7f5aa7200640 1 -- start start 2024-08-20T01:07:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.490+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.490+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010a510 0x7f5aa019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa010ae90 0x7f5aa01a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114ba0 con 0x7f5aa010a510 2024-08-20T01:07:46.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114a20 con 0x7f5aa010ae90 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0114d20 con 0x7f5aa0104b00 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43092/0 (socket says 172.21.15.138:43092) 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 -- 172.21.15.138:0/2401408037 learned_addr learned my addr 172.21.15.138:0/2401408037 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5400640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010a510 0x7f5aa019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa6800640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa010ae90 0x7f5aa01a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 -- 172.21.15.138:0/2401408037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa010ae90 msgr2=0x7f5aa01a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa010ae90 0x7f5aa01a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 -- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010a510 msgr2=0x7f5aa019b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010a510 0x7f5aa019b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.491+0000 7f5aa5e00640 1 -- 172.21.15.138:0/2401408037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa01a0fe0 con 0x7f5aa0104b00 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5aa5e00640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9f0 tx=0x7f5a9400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400cd80 con 0x7f5aa0104b00 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a940108e0 con 0x7f5aa0104b00 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa01a12d0 con 0x7f5aa0104b00 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94018980 con 0x7f5aa0104b00 2024-08-20T01:07:46.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.492+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa01a7990 con 0x7f5aa0104b00 2024-08-20T01:07:46.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.493+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa0104ee0 con 0x7f5aa0104b00 2024-08-20T01:07:46.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.495+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f5a94010430 con 0x7f5aa0104b00 2024-08-20T01:07:46.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.497+0000 7f5a9aa00640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 0x7f5a880b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:46.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.497+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5a940e4900 con 0x7f5aa0104b00 2024-08-20T01:07:46.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.497+0000 7f5aa5400640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 0x7f5a880b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:46.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.498+0000 7f5aa5400640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 0x7f5a880b6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0102dc0 tx=0x7f5a90005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:46.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.498+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5a940ab430 con 0x7f5aa0104b00 2024-08-20T01:07:46.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.720+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5aa01a7d50 con 0x7f5aa0104b00 2024-08-20T01:07:46.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.721+0000 7f5a9aa00640 1 -- 172.21.15.138:0/2401408037 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f5a940af980 con 0x7f5aa0104b00 2024-08-20T01:07:46.720 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:46.721 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:07:46.721 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:07:46.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 msgr2=0x7f5a880b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:46.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 0x7f5a880b6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0102dc0 tx=0x7f5a90005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 msgr2=0x7f5aa019af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400e9f0 tx=0x7f5a9400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 shutdown_connections 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5a880b3cc0 0x7f5a880b6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa010ae90 0x7f5aa01a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5aa010a510 0x7f5aa019b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 --2- 172.21.15.138:0/2401408037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5aa0104b00 0x7f5aa019af30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:46.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 >> 172.21.15.138:0/2401408037 conn(0x7f5aa0100550 msgr2=0x7f5aa0106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:46.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 shutdown_connections 2024-08-20T01:07:46.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:46.724+0000 7f5aa7200640 1 -- 172.21.15.138:0/2401408037 wait complete. 2024-08-20T01:07:47.540 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:07:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:47 smithi138 ceph-mon[18076]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-08-20T01:07:47.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2401408037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:47.912 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:47 smithi167 ceph-mon[21965]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-08-20T01:07:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2401408037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:47 smithi142 ceph-mon[21799]: pgmap v352: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 8 op/s 2024-08-20T01:07:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2401408037' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 -- 172.21.15.138:0/2003576252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 msgr2=0x7fbecc102ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 --2- 172.21.15.138:0/2003576252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc102ce0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbec0009900 tx=0x7fbec002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 -- 172.21.15.138:0/2003576252 shutdown_connections 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 --2- 172.21.15.138:0/2003576252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108c90 0x7fbecc10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 --2- 172.21.15.138:0/2003576252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108310 0x7fbecc108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 --2- 172.21.15.138:0/2003576252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc102ce0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 -- 172.21.15.138:0/2003576252 >> 172.21.15.138:0/2003576252 conn(0x7fbecc0fe390 msgr2=0x7fbecc1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 -- 172.21.15.138:0/2003576252 shutdown_connections 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.500+0000 7fbed2e00640 1 -- 172.21.15.138:0/2003576252 wait complete. 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.501+0000 7fbed2e00640 1 Processor -- start 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.501+0000 7fbed2e00640 1 -- start start 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc19afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 0x7fbecc19b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:48.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc1129a0 con 0x7fbecc108c90 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc112820 con 0x7fbecc108310 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc112b20 con 0x7fbecc102900 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 0x7fbecc19b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc19afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 0x7fbecc19b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37126/0 (socket says 172.21.15.138:37126) 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed2400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54916/0 (socket says 172.21.15.138:54916) 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1000640 1 -- 172.21.15.138:0/1838895779 learned_addr learned my addr 172.21.15.138:0/1838895779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 -- 172.21.15.138:0/1838895779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 msgr2=0x7fbecc19b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 0x7fbecc19b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 -- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 msgr2=0x7fbecc1a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.502+0000 7fbed1a00640 1 -- 172.21.15.138:0/1838895779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc1a1060 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbed2400640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbed1a00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc19afb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec002a9e0 tx=0x7fbec0003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec003f070 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec0031b50 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0035030 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc1a12f0 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.503+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1a7a10 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.504+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc103bb0 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.507+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fbec0042070 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.508+0000 7fbeca000640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 0x7fbeb40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.508+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fbec01047a0 con 0x7fbecc102900 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.508+0000 7fbed1000640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 0x7fbeb40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:48.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.509+0000 7fbed1000640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 0x7fbeb40b6200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbecc107b80 tx=0x7fbeb8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:48.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.510+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fbec0031d00 con 0x7fbecc102900 2024-08-20T01:07:48.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.736+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbecc10eee0 con 0x7fbecc102900 2024-08-20T01:07:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.737+0000 7fbeca000640 1 -- 172.21.15.138:0/1838895779 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7fbec00cb2d0 con 0x7fbecc102900 2024-08-20T01:07:48.735 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:48.735 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:07:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 msgr2=0x7fbeb40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 0x7fbeb40b6200 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbecc107b80 tx=0x7fbeb8005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 msgr2=0x7fbecc19afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc19afb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbec002a9e0 tx=0x7fbec0003b90 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 shutdown_connections 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbeb40b3d40 0x7fbeb40b6200 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbecc108c90 0x7fbecc1a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbecc108310 0x7fbecc19b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 --2- 172.21.15.138:0/1838895779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbecc102900 0x7fbecc19afb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 >> 172.21.15.138:0/1838895779 conn(0x7fbecc0fe390 msgr2=0x7fbecc0758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.739+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 shutdown_connections 2024-08-20T01:07:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:48.740+0000 7fbed2e00640 1 -- 172.21.15.138:0/1838895779 wait complete. 2024-08-20T01:07:49.517 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:07:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:49 smithi138 ceph-mon[18076]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:49.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:49 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1838895779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:49.884 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:49 smithi167 ceph-mon[21965]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:49 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1838895779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:49 smithi142 ceph-mon[21799]: pgmap v353: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-08-20T01:07:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:49 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1838895779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 -- 172.21.15.138:0/3443734892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 msgr2=0x7f51b810f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 --2- 172.21.15.138:0/3443734892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b810f640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009be0 tx=0x7f51b002a6c0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 -- 172.21.15.138:0/3443734892 shutdown_connections 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 --2- 172.21.15.138:0/3443734892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b810f640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 --2- 172.21.15.138:0/3443734892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 0x7f51b8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 --2- 172.21.15.138:0/3443734892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 -- 172.21.15.138:0/3443734892 >> 172.21.15.138:0/3443734892 conn(0x7f51b80fe500 msgr2=0x7f51b8100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 -- 172.21.15.138:0/3443734892 shutdown_connections 2024-08-20T01:07:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.493+0000 7f51bf800640 1 -- 172.21.15.138:0/3443734892 wait complete. 2024-08-20T01:07:50.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.494+0000 7f51bf800640 1 Processor -- start 2024-08-20T01:07:50.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.494+0000 7f51bf800640 1 -- start start 2024-08-20T01:07:50.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:50.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 0x7f51b8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:50.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8112b00 con 0x7f51b8108e40 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8112980 con 0x7f51b81084c0 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bf800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f51b8112c80 con 0x7f51b8102ab0 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54934/0 (socket says 172.21.15.138:54934) 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bd800640 1 -- 172.21.15.138:0/1147299163 learned_addr learned my addr 172.21.15.138:0/1147299163 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bce00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:50.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51b7e00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 0x7f51b8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bce00640 1 -- 172.21.15.138:0/1147299163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 msgr2=0x7f51b8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.495+0000 7f51bce00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 0x7f51b8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bce00640 1 -- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 msgr2=0x7f51b819e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bce00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bce00640 1 -- 172.21.15.138:0/1147299163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b819efc0 con 0x7f51b8102ab0 2024-08-20T01:07:50.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bd800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bce00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8198f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00e790 tx=0x7f51ac00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac00eea0 con 0x7f51b8102ab0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b819f2b0 con 0x7f51b8102ab0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51ac00ccc0 con 0x7f51b8102ab0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51ac018660 con 0x7f51b8102ab0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.496+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b81a5b40 con 0x7f51b8102ab0 2024-08-20T01:07:50.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.497+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b8102e90 con 0x7f51b8102ab0 2024-08-20T01:07:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.500+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f51ac010430 con 0x7f51b8102ab0 2024-08-20T01:07:50.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.502+0000 7f51b5600640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 0x7f51a00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:50.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.502+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f51ac0e4770 con 0x7f51b8102ab0 2024-08-20T01:07:50.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.502+0000 7f51b7e00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 0x7f51a00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.503+0000 7f51b7e00640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 0x7f51a00b60e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51b8107b90 tx=0x7f51a4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.503+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f51ac0ab2a0 con 0x7f51b8102ab0 2024-08-20T01:07:50.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.694+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b81a5d70 con 0x7f51b8102ab0 2024-08-20T01:07:50.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.696+0000 7f51b5600640 1 -- 172.21.15.138:0/1147299163 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7f51ac0af7f0 con 0x7f51b8102ab0 2024-08-20T01:07:50.694 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:50.694 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:07:50.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 msgr2=0x7f51a00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:50.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 0x7f51a00b60e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f51b8107b90 tx=0x7f51a4005f90 comp rx=0 tx=0).stop 2024-08-20T01:07:50.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 msgr2=0x7f51b8198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8198f10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00e790 tx=0x7f51ac00ec50 comp rx=0 tx=0).stop 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 shutdown_connections 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f51a00b3c20 0x7f51a00b60e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51b8108e40 0x7f51b819e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b81084c0 0x7f51b8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 --2- 172.21.15.138:0/1147299163 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f51b8102ab0 0x7f51b8198f10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 >> 172.21.15.138:0/1147299163 conn(0x7f51b80fe500 msgr2=0x7f51b81008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 shutdown_connections 2024-08-20T01:07:50.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:50.699+0000 7f51bf800640 1 -- 172.21.15.138:0/1147299163 wait complete. 2024-08-20T01:07:51.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:07:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:51 smithi138 ceph-mon[18076]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:51 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1147299163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:51.839 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:51 smithi167 ceph-mon[21965]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:51 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1147299163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:51 smithi142 ceph-mon[21799]: pgmap v354: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 7 op/s 2024-08-20T01:07:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:51 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1147299163' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:07:52.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 -- 172.21.15.138:0/4047143057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b0104ca0 msgr2=0x7f04b0105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 --2- 172.21.15.138:0/4047143057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b0104ca0 0x7f04b0105080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04a0009900 tx=0x7f04a002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 -- 172.21.15.138:0/4047143057 shutdown_connections 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 --2- 172.21.15.138:0/4047143057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b010b030 0x7f04b0111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 --2- 172.21.15.138:0/4047143057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b010aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 --2- 172.21.15.138:0/4047143057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b0104ca0 0x7f04b0105080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 -- 172.21.15.138:0/4047143057 >> 172.21.15.138:0/4047143057 conn(0x7f04b0100730 msgr2=0x7f04b0102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 -- 172.21.15.138:0/4047143057 shutdown_connections 2024-08-20T01:07:52.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.529+0000 7f04b6400640 1 -- 172.21.15.138:0/4047143057 wait complete. 2024-08-20T01:07:52.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.530+0000 7f04b6400640 1 Processor -- start 2024-08-20T01:07:52.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.530+0000 7f04b6400640 1 -- start start 2024-08-20T01:07:52.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:52.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b010b030 0x7f04b01a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0114d40 con 0x7f04b010b030 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0114bc0 con 0x7f04b010a6b0 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04b6400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f04b0114ec0 con 0x7f04b0104ca0 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04af400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44622/0 (socket says 172.21.15.138:44622) 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04a6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04af400640 1 -- 172.21.15.138:0/2715792591 learned_addr learned my addr 172.21.15.138:0/2715792591 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:52.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.531+0000 7f04afe00640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b010b030 0x7f04b01a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:52.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 -- 172.21.15.138:0/2715792591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 msgr2=0x7f04b019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:52.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 -- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b010b030 msgr2=0x7f04b01a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b010b030 0x7f04b01a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 -- 172.21.15.138:0/2715792591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04b01a12e0 con 0x7f04b010a6b0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04af400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04a6a00640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b019b770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f049800d6d0 tx=0x7f049800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498017070 con 0x7f04b010a6b0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04b01a15d0 con 0x7f04b010a6b0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0498009820 con 0x7f04b010a6b0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0498004c40 con 0x7f04b010a6b0 2024-08-20T01:07:52.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.532+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04b01a7e60 con 0x7f04b010a6b0 2024-08-20T01:07:52.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.533+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04b0105f50 con 0x7f04b010a6b0 2024-08-20T01:07:52.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.536+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f0498005220 con 0x7f04b010a6b0 2024-08-20T01:07:52.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.537+0000 7f04acc00640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 0x7f04940b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:52.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.537+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f04980e3f70 con 0x7f04b010a6b0 2024-08-20T01:07:52.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.538+0000 7f04af400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 0x7f04940b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:52.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.538+0000 7f04af400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 0x7f04940b6160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04a00096c0 tx=0x7f04a0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:52.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.539+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f04980aab20 con 0x7f04b010a6b0 2024-08-20T01:07:52.770 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.772+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04b0105080 con 0x7f04b010a6b0 2024-08-20T01:07:52.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.773+0000 7f04acc00640 1 -- 172.21.15.138:0/2715792591 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f04b0105080 con 0x7f04b010a6b0 2024-08-20T01:07:52.771 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:07:52.771 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:07:52.772 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:07:52.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.775+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 msgr2=0x7f04940b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:52.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 0x7f04940b6160 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04a00096c0 tx=0x7f04a0005a20 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 msgr2=0x7f04b019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b019b770 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f049800d6d0 tx=0x7f049800db90 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 shutdown_connections 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f04940b3ca0 0x7f04940b6160 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04b010b030 0x7f04b01a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f04b010a6b0 0x7f04b019b770 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 --2- 172.21.15.138:0/2715792591 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04b0104ca0 0x7f04b019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 >> 172.21.15.138:0/2715792591 conn(0x7f04b0100730 msgr2=0x7f04b0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 shutdown_connections 2024-08-20T01:07:52.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:52.776+0000 7f04b6400640 1 -- 172.21.15.138:0/2715792591 wait complete. 2024-08-20T01:07:53.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T01:05:27.648119+0000', 'modified': '2024-08-20T01:06:23.367255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24743, 'mds_1': 24706, 'mds_2': 24704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2224495590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2224495590}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2224495590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2270224611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270224611}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270224611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/2113916191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 2113916191}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 2113916191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.142:6837/1103910039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1103910039}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1103910039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.138:6839/1714719372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1714719372}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1714719372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.142:6839/80285616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 80285616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 80285616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24743, 'qdb_cluster': [24704, 24743, 24706]} 2024-08-20T01:07:53.528 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-20T01:07:53.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T01:07:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:53 smithi138 ceph-mon[18076]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:53 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2715792591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:53.901 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:53 smithi167 ceph-mon[21965]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:53 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2715792591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:53 smithi142 ceph-mon[21799]: pgmap v355: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:07:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:53 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2715792591' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:07:54.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 -- 172.21.15.138:0/826191329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb650108e40 msgr2=0x7fb65010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 --2- 172.21.15.138:0/826191329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb650108e40 0x7fb65010f640 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb64802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 -- 172.21.15.138:0/826191329 shutdown_connections 2024-08-20T01:07:54.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 --2- 172.21.15.138:0/826191329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb650108e40 0x7fb65010f640 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 --2- 172.21.15.138:0/826191329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6501084c0 0x7fb650108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 --2- 172.21.15.138:0/826191329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 -- 172.21.15.138:0/826191329 >> 172.21.15.138:0/826191329 conn(0x7fb6500fe500 msgr2=0x7fb650100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 -- 172.21.15.138:0/826191329 shutdown_connections 2024-08-20T01:07:54.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.497+0000 7fb656a00640 1 -- 172.21.15.138:0/826191329 wait complete. 2024-08-20T01:07:54.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 Processor -- start 2024-08-20T01:07:54.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 -- start start 2024-08-20T01:07:54.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb650108e40 0x7fb65019e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112ac0 con 0x7fb6501084c0 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112940 con 0x7fb650102ab0 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb656a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112c40 con 0x7fb650108e40 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.498+0000 7fb64f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54952/0 (socket says 172.21.15.138:54952) 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 -- 172.21.15.138:0/1263112042 learned_addr learned my addr 172.21.15.138:0/1263112042 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb654a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb650108e40 0x7fb65019e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 -- 172.21.15.138:0/1263112042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb650108e40 msgr2=0x7fb65019e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb650108e40 0x7fb65019e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 -- 172.21.15.138:0/1263112042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 msgr2=0x7fb650198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 -- 172.21.15.138:0/1263112042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65019f080 con 0x7fb6501084c0 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.499+0000 7fb64f400640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9f0 tx=0x7fb63c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.500+0000 7fb64cc00640 1 -- 172.21.15.138:0/1263112042 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c00cd80 con 0x7fb6501084c0 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.500+0000 7fb64cc00640 1 -- 172.21.15.138:0/1263112042 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb63c0054d0 con 0x7fb6501084c0 2024-08-20T01:07:54.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.500+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65019f370 con 0x7fb6501084c0 2024-08-20T01:07:54.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.500+0000 7fb64cc00640 1 -- 172.21.15.138:0/1263112042 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c010690 con 0x7fb6501084c0 2024-08-20T01:07:54.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.500+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501a5a30 con 0x7fb6501084c0 2024-08-20T01:07:54.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.504+0000 7fb659674740 1 Processor -- start 2024-08-20T01:07:54.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.504+0000 7fb659674740 1 -- start start 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.504+0000 7fb659674740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.504+0000 7fb659674740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d90a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb659674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d90cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb659674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56278d870840 con 0x56278d90abd0 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb659674740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56278d85fd40 con 0x56278d8fa830 2024-08-20T01:07:54.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb659674740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278d863d80 con 0x56278d8f9f10 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb64cc00640 1 -- 172.21.15.138:0/1263112042 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fb63c004e90 con 0x7fb6501084c0 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44662/0 (socket says 172.21.15.138:44662) 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 -- 172.21.15.138:0/2110644677 learned_addr learned my addr 172.21.15.138:0/2110644677 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb64fe00640 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d90a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb64f400640 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d90cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 -- 172.21.15.138:0/2110644677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 msgr2=0x56278d90a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d90a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 -- 172.21.15.138:0/2110644677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 msgr2=0x56278d90cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d90cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.505+0000 7fb654a00640 1 -- 172.21.15.138:0/2110644677 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278d8061c0 con 0x56278d8f9f10 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.506+0000 7fb654a00640 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb648032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb0f2dc376d1643 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.506+0000 7fb645200640 1 -- 172.21.15.138:0/2110644677 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648035040 con 0x56278d8f9f10 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.506+0000 7fb645200640 1 -- 172.21.15.138:0/2110644677 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb648004030 con 0x56278d8f9f10 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.506+0000 7fb645200640 1 -- 172.21.15.138:0/2110644677 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6480054d0 con 0x56278d8f9f10 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb64cc00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 0x7fb6340b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb64cc00640 1 -- 172.21.15.138:0/1263112042 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fb63c0e4430 con 0x7fb6501084c0 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 -- 172.21.15.138:0/2110644677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 msgr2=0x56278d8fa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb648009900 tx=0x7fb648032b20 comp rx=0 tx=0).stop 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 0x7fb6340b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 -- 172.21.15.138:0/2110644677 shutdown_connections 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d90cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d90a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 --2- 172.21.15.138:0/2110644677 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fa2f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 -- 172.21.15.138:0/2110644677 >> 172.21.15.138:0/2110644677 conn(0x56278d8f7d40 msgr2=0x56278d8f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 -- 172.21.15.138:0/2110644677 shutdown_connections 2024-08-20T01:07:54.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.507+0000 7fb659674740 1 -- 172.21.15.138:0/2110644677 wait complete. 2024-08-20T01:07:54.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 0x7fb6340b6160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x56278d8f99f0 tx=0x7fb640009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 -- start start 2024-08-20T01:07:54.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 Processor -- start 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 0x56278d90a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8fa830 0x56278d907810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56278d6c25f0 con 0x56278d8fa830 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56278d7fb170 con 0x56278d8f9f10 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278d90d820 con 0x56278d90abd0 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.508+0000 7fb659674740 10 client.0 _ll_register_callbacks cb 0x56278d85a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 0x56278d90a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44676/0 (socket says 172.21.15.138:44676) 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 -- 172.21.15.138:0/1342912429 learned_addr learned my addr 172.21.15.138:0/1342912429 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 0x56278d90a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37220/0 (socket says 172.21.15.138:37220) 2024-08-20T01:07:54.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb64f400640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8fa830 0x56278d907810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 -- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 msgr2=0x56278d90a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 0x56278d90a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 -- 172.21.15.138:0/1342912429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8fa830 msgr2=0x56278d907810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8fa830 0x56278d907810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.509+0000 7fb654a00640 1 -- 172.21.15.138:0/1342912429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278d5bf540 con 0x56278d90abd0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb654a00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb64802a910 tx=0x7fb648005050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2affe45f5fd59183 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb639600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb648035070 con 0x56278d90abd0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb648004030 con 0x56278d90abd0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fb648005270 con 0x56278d90abd0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb648005470 con 0x56278d90abd0 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb659674740 10 client.25192 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:54.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56278d919080 con 0x56278d90abd0 2024-08-20T01:07:54.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.510+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb648004030 con 0x56278d90abd0 2024-08-20T01:07:54.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb648004c30 con 0x56278d90abd0 2024-08-20T01:07:54.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 10 client.25192 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:54.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 10 client.25192 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 20 client.25192 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d91b1e0 0x56278d91c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 4 client.25192 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278d872790 con 0x56278d91b1e0 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.511+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d91b1e0 0x56278d91c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.512+0000 7fb639600640 10 client.25192 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.512+0000 7fb64fe00640 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d91b1e0 0x56278d91c650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27d4fd9101532b12 server_cookie=1b1ab2a8ff247d4e in_seq=0 out_seq=0 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb639600640 1 -- 172.21.15.138:0/1342912429 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56278d872790 con 0x56278d91b1e0 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb639600640 10 client.25192 handle_command_reply: tid=0 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 client.25192 shutdown 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 msgr2=0x56278d907d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb64802a910 tx=0x7fb648005050 comp rx=0 tx=0).stop 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 shutdown_connections 2024-08-20T01:07:54.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d91b1e0 0x56278d91c650 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d90abd0 0x56278d907d50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8fa830 0x56278d907810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 --2- 172.21.15.138:0/1342912429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8f9f10 0x56278d90a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb639600640 0 client.25192 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 >> 172.21.15.138:0/1342912429 conn(0x56278d8f7d40 msgr2=0x56278d8fd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 shutdown_connections 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 1 -- 172.21.15.138:0/1342912429 wait complete. 2024-08-20T01:07:54.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.513+0000 7fb659674740 20 client.25192 trim_cache size 0 max 16384 2024-08-20T01:07:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 Processor -- start 2024-08-20T01:07:54.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 -- start start 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 0x56278d907cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d90abd0 0x56278d8ff060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56278d91a990 con 0x56278d8f9f10 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56278d91b060 con 0x56278d90abd0 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb659674740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278d91b730 con 0x56278d8fa830 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54980/0 (socket says 172.21.15.138:54980) 2024-08-20T01:07:54.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.532+0000 7fb64fe00640 1 -- 172.21.15.138:0/2923846641 learned_addr learned my addr 172.21.15.138:0/2923846641 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64f400640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 0x56278d907cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb654a00640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d90abd0 0x56278d8ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 -- 172.21.15.138:0/2923846641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 msgr2=0x56278d907cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 0x56278d907cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 -- 172.21.15.138:0/2923846641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d90abd0 msgr2=0x56278d8ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d90abd0 0x56278d8ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 -- 172.21.15.138:0/2923846641 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278d905d50 con 0x56278d8f9f10 2024-08-20T01:07:54.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64f400640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 0x56278d907cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb64fe00640 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb640012440 tx=0x7fb64001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07927938248ad58 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.533+0000 7fb63be00640 1 -- 172.21.15.138:0/2923846641 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640003970 con 0x56278d8f9f10 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb63be00640 1 -- 172.21.15.138:0/2923846641 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64002b530 con 0x56278d8f9f10 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb63be00640 1 -- 172.21.15.138:0/2923846641 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640004300 con 0x56278d8f9f10 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 -- 172.21.15.138:0/2923846641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 msgr2=0x56278d916af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb640012440 tx=0x7fb64001a000 comp rx=0 tx=0).stop 2024-08-20T01:07:54.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 -- 172.21.15.138:0/2923846641 shutdown_connections 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d90abd0 0x56278d8ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8fa830 0x56278d907cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 --2- 172.21.15.138:0/2923846641 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d8f9f10 0x56278d916af0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 -- 172.21.15.138:0/2923846641 >> 172.21.15.138:0/2923846641 conn(0x56278d8f7d40 msgr2=0x56278d9075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 -- 172.21.15.138:0/2923846641 shutdown_connections 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.534+0000 7fb659674740 1 -- 172.21.15.138:0/2923846641 wait complete. 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 -- start start 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 Processor -- start 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d8f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56278d91c4d0 con 0x56278d90abd0 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56278d91be00 con 0x56278d8fa830 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56278d91cba0 con 0x56278d8f9f10 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.535+0000 7fb659674740 10 client.0 _ll_register_callbacks cb 0x7fb640009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44688/0 (socket says 172.21.15.138:44688) 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37244/0 (socket says 172.21.15.138:37244) 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64fe00640 1 -- 172.21.15.138:0/4101719773 learned_addr learned my addr 172.21.15.138:0/4101719773 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb654a00640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d8f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 -- 172.21.15.138:0/4101719773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 msgr2=0x56278d8fc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 -- 172.21.15.138:0/4101719773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 msgr2=0x56278d8f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:54.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d8f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 -- 172.21.15.138:0/4101719773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278d8f77e0 con 0x56278d8fa830 2024-08-20T01:07:54.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb64f400640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x56278d8f9ec0 tx=0x7fb63c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a226f5824ee73118 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:54.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb639600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:07:54.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.536+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63c0aa9c0 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64002b530 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fb63c0a7140 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb63c005680 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 10 client.25220 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x56278d985a00 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64002b530 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fb63c0b3ad0 con 0x56278d8fa830 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 10 client.25220 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 10 client.25220 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 20 client.25220 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d98a4d0 0x56278d9891e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:54.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 4 client.25220 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.537+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "status"}) -- 0x56278d86e230 con 0x56278d98a4d0 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.538+0000 7fb64fe00640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d98a4d0 0x56278d9891e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.538+0000 7fb64fe00640 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d98a4d0 0x56278d9891e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a7900a852507d283 server_cookie=550be71dacb6869 in_seq=0 out_seq=0 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.538+0000 7fb639600640 10 client.25220 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb639600640 1 -- 172.21.15.138:0/4101719773 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56278d86e230 con 0x56278d98a4d0 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb639600640 10 client.25220 handle_command_reply: tid=0 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 client.25220 shutdown 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 msgr2=0x56278d8f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x56278d8f9ec0 tx=0x7fb63c0048f0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 shutdown_connections 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x56278d98a4d0 0x56278d9891e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56278d90abd0 0x56278d8f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb639600640 0 client.25220 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56278d8fa830 0x56278d8f6a10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 --2- 172.21.15.138:0/4101719773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56278d8f9f10 0x56278d8fc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 >> 172.21.15.138:0/4101719773 conn(0x56278d8f7d40 msgr2=0x56278d908e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 shutdown_connections 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 1 -- 172.21.15.138:0/4101719773 wait complete. 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.539+0000 7fb659674740 20 client.25220 trim_cache size 0 max 16384 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:07:54.542 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 2, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24704, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 94.937474281999997, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 173.40895153400001 2024-08-20T01:07:54.543 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 msgr2=0x7fb6340b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 0x7fb6340b6160 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x56278d8f99f0 tx=0x7fb640009500 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 msgr2=0x7fb650199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00e9f0 tx=0x7fb63c00edf0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 shutdown_connections 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6340b3ca0 0x7fb6340b6160 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb650108e40 0x7fb65019e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6501084c0 0x7fb650199510 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 --2- 172.21.15.138:0/1263112042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb650102ab0 0x7fb650198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 >> 172.21.15.138:0/1263112042 conn(0x7fb6500fe500 msgr2=0x7fb6501008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 shutdown_connections 2024-08-20T01:07:54.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:54.540+0000 7fb656a00640 1 -- 172.21.15.138:0/1263112042 wait complete. 2024-08-20T01:07:54.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:54 smithi138 ceph-mon[18076]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:54.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:54 smithi167 ceph-mon[21965]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:54.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:54 smithi142 ceph-mon[21799]: pgmap v356: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:55.307 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.g status 2024-08-20T01:07:55.676 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.309+0000 7f381d400640 1 -- 172.21.15.138:0/1582855706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810b030 msgr2=0x7f3818111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.309+0000 7f381d400640 1 --2- 172.21.15.138:0/1582855706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810b030 0x7f3818111830 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3810009ca0 tx=0x7f381002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 -- 172.21.15.138:0/1582855706 shutdown_connections 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 --2- 172.21.15.138:0/1582855706 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810b030 0x7f3818111830 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 --2- 172.21.15.138:0/1582855706 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f381810a6b0 0x7f381810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.340 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 --2- 172.21.15.138:0/1582855706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3818104ca0 0x7f3818105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 -- 172.21.15.138:0/1582855706 >> 172.21.15.138:0/1582855706 conn(0x7f38181006f0 msgr2=0x7f3818102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 -- 172.21.15.138:0/1582855706 shutdown_connections 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.310+0000 7f381d400640 1 -- 172.21.15.138:0/1582855706 wait complete. 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 Processor -- start 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 -- start start 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3818104ca0 0x7f381819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381810b030 0x7f38181a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3818114cb0 con 0x7f381810a6b0 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3818114b30 con 0x7f381810b030 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f381d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3818114e30 con 0x7f3818104ca0 2024-08-20T01:07:56.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f380fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f380fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55004/0 (socket says 172.21.15.138:55004) 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f380fe00640 1 -- 172.21.15.138:0/2977050048 learned_addr learned my addr 172.21.15.138:0/2977050048 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.311+0000 7f3816800640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3818104ca0 0x7f381819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f3817200640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381810b030 0x7f38181a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 -- 172.21.15.138:0/2977050048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3818104ca0 msgr2=0x7f381819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3818104ca0 0x7f381819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 -- 172.21.15.138:0/2977050048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381810b030 msgr2=0x7f38181a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381810b030 0x7f38181a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 -- 172.21.15.138:0/2977050048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38181a1210 con 0x7f381810a6b0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380fe00640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9f0 tx=0x7f380000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380f400640 1 -- 172.21.15.138:0/2977050048 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380000cd80 con 0x7f381810a6b0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380f400640 1 -- 172.21.15.138:0/2977050048 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38000108e0 con 0x7f381810a6b0 2024-08-20T01:07:56.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38181a1500 con 0x7f381810a6b0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.312+0000 7f380f400640 1 -- 172.21.15.138:0/2977050048 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800018980 con 0x7f381810a6b0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.313+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38181a7d90 con 0x7f381810a6b0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 Processor -- start 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 -- start start 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f380f400640 1 -- 172.21.15.138:0/2977050048 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f3800010430 con 0x7f381810a6b0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x56332657b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633265877c0 0x563326589bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633264f1840 con 0x56332657b830 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5633264e0d40 con 0x5633265877c0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f381fec8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5633264e4d80 con 0x56332657af10 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f3816800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f3816800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55014/0 (socket says 172.21.15.138:55014) 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.316+0000 7f3816800640 1 -- 172.21.15.138:0/3383909440 learned_addr learned my addr 172.21.15.138:0/3383909440 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3817200640 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x56332657b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 -- 172.21.15.138:0/3383909440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 msgr2=0x56332657b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x56332657b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 -- 172.21.15.138:0/3383909440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633265877c0 msgr2=0x563326589bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633265877c0 0x563326589bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 -- 172.21.15.138:0/3383909440 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633264871c0 con 0x56332657b830 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3816800640 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3808009d20 tx=0x7f3808011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec943de03e87478e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3807400640 1 -- 172.21.15.138:0/3383909440 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808004230 con 0x56332657b830 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3807400640 1 -- 172.21.15.138:0/3383909440 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080043d0 con 0x56332657b830 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.317+0000 7f3807400640 1 -- 172.21.15.138:0/3383909440 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080197c0 con 0x56332657b830 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f380f400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 0x7f37fc0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 -- 172.21.15.138:0/3383909440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 msgr2=0x563326587280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3808009d20 tx=0x7f3808011810 comp rx=0 tx=0).stop 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f380f400640 1 -- 172.21.15.138:0/2977050048 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f38000e4320 con 0x7f381810a6b0 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f3817200640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 0x7f37fc0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 -- 172.21.15.138:0/3383909440 shutdown_connections 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633265877c0 0x563326589bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657b830 0x563326587280 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 --2- 172.21.15.138:0/3383909440 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x56332657b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 -- 172.21.15.138:0/3383909440 >> 172.21.15.138:0/3383909440 conn(0x563326578d40 msgr2=0x563326579ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 -- 172.21.15.138:0/3383909440 shutdown_connections 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.318+0000 7f381fec8740 1 -- 172.21.15.138:0/3383909440 wait complete. 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.319+0000 7f381fec8740 1 -- start start 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.319+0000 7f3817200640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 0x7f37fc0b6160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3810006010 tx=0x7f38100098e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.319+0000 7f381fec8740 1 Processor -- start 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x5633265843a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633265877c0 0x5633265848e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633263435f0 con 0x56332657af10 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56332647c170 con 0x56332657b830 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56332658a470 con 0x5633265877c0 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f381fec8740 10 client.0 _ll_register_callbacks cb 0x5633264db8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55018/0 (socket says 172.21.15.138:55018) 2024-08-20T01:07:56.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 -- 172.21.15.138:0/908811851 learned_addr learned my addr 172.21.15.138:0/908811851 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f380fe00640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x5633265843a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3817200640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633265877c0 0x5633265848e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 -- 172.21.15.138:0/908811851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633265877c0 msgr2=0x5633265848e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633265877c0 0x5633265848e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 -- 172.21.15.138:0/908811851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 msgr2=0x5633265843a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x5633265843a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f3816800640 1 -- 172.21.15.138:0/908811851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563326240540 con 0x56332657af10 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.320+0000 7f380fe00640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x5633265843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f3816800640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fa0 tx=0x7f38080052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b3e3212b3cf221 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f37fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808027070 con 0x56332657af10 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f38080037f0 con 0x56332657af10 2024-08-20T01:07:56.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38080043d0 con 0x56332657af10 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3808033050 con 0x56332657af10 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f381fec8740 10 client.15201 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563326595c70 con 0x56332657af10 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.321+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38080043d0 con 0x56332657af10 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3808004ce0 con 0x56332657af10 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 10 client.15201 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 10 client.15201 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 20 client.15201 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5633265980e0 0x563326599550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 4 client.15201 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633264f3790 con 0x5633265980e0 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.322+0000 7f380fe00640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5633265980e0 0x563326599550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.323+0000 7f380fe00640 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5633265980e0 0x563326599550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e07a800692f00dc6 server_cookie=7125160baf01fe20 in_seq=0 out_seq=0 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.323+0000 7f37fbe00640 10 client.15201 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.323+0000 7f37fbe00640 1 -- 172.21.15.138:0/908811851 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633264f3790 con 0x5633265980e0 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.323+0000 7f37fbe00640 10 client.15201 handle_command_reply: tid=0 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.323+0000 7f381fec8740 1 client.15201 shutdown 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 msgr2=0x563326586f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3808009fa0 tx=0x7f38080052f0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 shutdown_connections 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5633265980e0 0x563326599550 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5633265877c0 0x5633265848e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x5633265843a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 --2- 172.21.15.138:0/908811851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56332657af10 0x563326586f50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f37fbe00640 0 client.15201 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 >> 172.21.15.138:0/908811851 conn(0x563326578d40 msgr2=0x56332657e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 shutdown_connections 2024-08-20T01:07:56.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 1 -- 172.21.15.138:0/908811851 wait complete. 2024-08-20T01:07:56.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.324+0000 7f381fec8740 20 client.15201 trim_cache size 0 max 16384 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.342+0000 7f381fec8740 1 Processor -- start 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 -- start start 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x563326582c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x563326583160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563326597890 con 0x5633265877c0 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563326597f60 con 0x56332657b830 2024-08-20T01:07:56.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f381fec8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563326598630 con 0x56332657af10 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f3817200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f3817200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55030/0 (socket says 172.21.15.138:55030) 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f3817200640 1 -- 172.21.15.138:0/4152087290 learned_addr learned my addr 172.21.15.138:0/4152087290 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f380fe00640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x563326583160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.343+0000 7f3816800640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x563326582c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 -- 172.21.15.138:0/4152087290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 msgr2=0x563326582c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x563326582c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 -- 172.21.15.138:0/4152087290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 msgr2=0x563326583160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x563326583160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 -- 172.21.15.138:0/4152087290 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633265797c0 con 0x5633265877c0 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f380fe00640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x563326583160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3817200640 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3810031e30 tx=0x7f3810041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca2acbce5a29a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3806a00640 1 -- 172.21.15.138:0/4152087290 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810056070 con 0x5633265877c0 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3806a00640 1 -- 172.21.15.138:0/4152087290 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810036050 con 0x5633265877c0 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.344+0000 7f3806a00640 1 -- 172.21.15.138:0/4152087290 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38100046b0 con 0x5633265877c0 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- 172.21.15.138:0/4152087290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 msgr2=0x563326584150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3810031e30 tx=0x7f3810041000 comp rx=0 tx=0).stop 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- 172.21.15.138:0/4152087290 shutdown_connections 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x563326584150 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657b830 0x563326583160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 --2- 172.21.15.138:0/4152087290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657af10 0x563326582c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- 172.21.15.138:0/4152087290 >> 172.21.15.138:0/4152087290 conn(0x563326578d40 msgr2=0x5633265970e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- 172.21.15.138:0/4152087290 shutdown_connections 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- 172.21.15.138:0/4152087290 wait complete. 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.345+0000 7f381fec8740 1 -- start start 2024-08-20T01:07:56.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 Processor -- start 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657af10 0x56332659ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x5633266007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5633265993d0 con 0x5633265877c0 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563326598d00 con 0x56332657af10 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563326599aa0 con 0x56332657b830 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f381fec8740 10 client.0 _ll_register_callbacks cb 0x7f3800005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f380fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.346+0000 7f3817200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x5633266007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44744/0 (socket says 172.21.15.138:44744) 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 -- 172.21.15.138:0/495411211 learned_addr learned my addr 172.21.15.138:0/495411211 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f3816800640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657af10 0x56332659ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 -- 172.21.15.138:0/495411211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657af10 msgr2=0x56332659ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657af10 0x56332659ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 -- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 msgr2=0x5633266007e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x5633266007e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 -- 172.21.15.138:0/495411211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563326600ea0 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f380fe00640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38000aa730 tx=0x7f38000aaf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e38a208d75ee9e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f37fbe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.347+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38000ae070 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3810036050 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f38000e40c0 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f38000c00d0 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 10 client.25216 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563326602180 con 0x56332657b830 2024-08-20T01:07:56.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3810036050 con 0x56332657b830 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f38000a67c0 con 0x56332657b830 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 10 client.25216 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 10 client.25216 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 20 client.25216 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x563326606d10 0x563326605a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 4 client.25216 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.348+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "status"}) -- 0x5633264ef230 con 0x563326606d10 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f3816800640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x563326606d10 0x563326605a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f37fbe00640 10 client.25216 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f3816800640 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x563326606d10 0x563326605a20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d84db278c8ccf057 server_cookie=2cc14636d6cd221b in_seq=0 out_seq=0 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f37fbe00640 1 -- 172.21.15.138:0/495411211 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x5633264ef230 con 0x563326606d10 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f37fbe00640 10 client.25216 handle_command_reply: tid=0 2024-08-20T01:07:56.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.349+0000 7f381fec8740 1 client.25216 shutdown 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 msgr2=0x56332659d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f38000aa730 tx=0x7f38000aaf10 comp rx=0 tx=0).stop 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 shutdown_connections 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5633265877c0 0x5633266007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x563326606d10 0x563326605a20 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56332657b830 0x56332659d410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 --2- 172.21.15.138:0/495411211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56332657af10 0x56332659ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 >> 172.21.15.138:0/495411211 conn(0x563326578d40 msgr2=0x563326584150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f37fbe00640 0 client.25216 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 shutdown_connections 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 1 -- 172.21.15.138:0/495411211 wait complete. 2024-08-20T01:07:56.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.350+0000 7f381fec8740 20 client.25216 trim_cache size 0 max 16384 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 1, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24706, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 97.757608116, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 168.820202679 2024-08-20T01:07:56.361 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 msgr2=0x7f37fc0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 0x7f37fc0b6160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3810006010 tx=0x7f38100098e0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 msgr2=0x7f381819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f380000e9f0 tx=0x7f380000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 shutdown_connections 2024-08-20T01:07:56.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37fc0b3ca0 0x7f37fc0b6160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f381810b030 0x7f38181a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f381810a6b0 0x7f381819b6a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 --2- 172.21.15.138:0/2977050048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3818104ca0 0x7f381819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 >> 172.21.15.138:0/2977050048 conn(0x7f38181006f0 msgr2=0x7f3818100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 shutdown_connections 2024-08-20T01:07:56.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:56.351+0000 7f381d400640 1 -- 172.21.15.138:0/2977050048 wait complete. 2024-08-20T01:07:57.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T01:07:57.432 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:57.820 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:57 smithi138 ceph-mon[18076]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:57.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:57 smithi167 ceph-mon[21965]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:57 smithi142 ceph-mon[21799]: pgmap v357: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 -- 172.21.15.138:0/1074651745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 msgr2=0x7f3d60108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 --2- 172.21.15.138:0/1074651745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d60108760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009860 tx=0x7f3d4c02a340 comp rx=0 tx=0).stop 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 -- 172.21.15.138:0/1074651745 shutdown_connections 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 --2- 172.21.15.138:0/1074651745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 0x7f3d6010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 --2- 172.21.15.138:0/1074651745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d60108760 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 --2- 172.21.15.138:0/1074651745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d60102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 -- 172.21.15.138:0/1074651745 >> 172.21.15.138:0/1074651745 conn(0x7f3d600fe3c0 msgr2=0x7f3d601007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 -- 172.21.15.138:0/1074651745 shutdown_connections 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.095+0000 7f3d67000640 1 -- 172.21.15.138:0/1074651745 wait complete. 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.096+0000 7f3d67000640 1 Processor -- start 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.096+0000 7f3d67000640 1 -- start start 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.096+0000 7f3d67000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d6019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d67000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d67000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 0x7f3d601a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d67000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112920 con 0x7f3d60108320 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d67000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3d601127a0 con 0x7f3d60102910 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d67000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3d60112aa0 con 0x7f3d60108ca0 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37380/0 (socket says 172.21.15.138:37380) 2024-08-20T01:07:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 -- 172.21.15.138:0/965734516 learned_addr learned my addr 172.21.15.138:0/965734516 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d66600640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 0x7f3d601a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65c00640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d6019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 -- 172.21.15.138:0/965734516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 msgr2=0x7f3d601a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 0x7f3d601a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 -- 172.21.15.138:0/965734516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 msgr2=0x7f3d6019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d6019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65200640 1 -- 172.21.15.138:0/965734516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d601a1070 con 0x7f3d60108320 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.097+0000 7f3d65c00640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d6019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d65200640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c035040 tx=0x7f3d4c032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d5e000640 1 -- 172.21.15.138:0/965734516 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c037040 con 0x7f3d60108320 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d5e000640 1 -- 172.21.15.138:0/965734516 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d4c041070 con 0x7f3d60108320 2024-08-20T01:07:58.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d5e000640 1 -- 172.21.15.138:0/965734516 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c003850 con 0x7f3d60108320 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d601a1300 con 0x7f3d60108320 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.098+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d601a7a20 con 0x7f3d60108320 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.101+0000 7f3d6b0bf740 1 Processor -- start 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.101+0000 7f3d5e000640 1 -- 172.21.15.138:0/965734516 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f3d4c0039f0 con 0x7f3d60108320 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.101+0000 7f3d6b0bf740 1 -- start start 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c2965c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c2965ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c295c6840 con 0x563c2965ca20 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c295b5d40 con 0x563c29650170 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c295b9d80 con 0x563c29650a90 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38286/0 (socket says 172.21.15.138:38286) 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 -- 172.21.15.138:0/1193217634 learned_addr learned my addr 172.21.15.138:0/1193217634 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d65c00640 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c2965c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d65200640 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c2965ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 -- 172.21.15.138:0/1193217634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 msgr2=0x563c2965c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c2965c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 -- 172.21.15.138:0/1193217634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 msgr2=0x563c2965ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.102+0000 7f3d66600640 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c2965ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d66600640 1 -- 172.21.15.138:0/1193217634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2955c1c0 con 0x563c29650170 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d66600640 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009f50 tx=0x7f3d580118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a0eee459564 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d52a00640 1 -- 172.21.15.138:0/1193217634 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58004d20 con 0x563c29650170 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d52a00640 1 -- 172.21.15.138:0/1193217634 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58004050 con 0x563c29650170 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d52a00640 1 -- 172.21.15.138:0/1193217634 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58005280 con 0x563c29650170 2024-08-20T01:07:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d5e000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 0x7f3d480b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d65c00640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 0x7f3d480b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.103+0000 7f3d5e000640 1 -- 172.21.15.138:0/965734516 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3d4c103c70 con 0x7f3d60108320 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1193217634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 msgr2=0x563c29650550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3d58009f50 tx=0x7f3d580118d0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1193217634 shutdown_connections 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c2965ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c2965c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1193217634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29650550 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1193217634 >> 172.21.15.138:0/1193217634 conn(0x563c2964dfa0 msgr2=0x563c2964f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d65c00640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 0x7f3d480b6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x563c296532f0 tx=0x7f3d54009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1193217634 shutdown_connections 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.104+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1193217634 wait complete. 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 -- start start 2024-08-20T01:07:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 Processor -- start 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c2965c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c29659610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c29659b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c294185f0 con 0x563c2965ca20 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c29551170 con 0x563c29650170 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c2965f6d0 con 0x563c29650a90 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.105+0000 7f3d6b0bf740 10 client.0 _ll_register_callbacks cb 0x563c295b08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d66600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c29659b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c2965c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d66600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c29659b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37412/0 (socket says 172.21.15.138:37412) 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d66600640 1 -- 172.21.15.138:0/3265354700 learned_addr learned my addr 172.21.15.138:0/3265354700 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65200640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c29659610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 -- 172.21.15.138:0/3265354700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 msgr2=0x563c29659610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c29659610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.130 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 -- 172.21.15.138:0/3265354700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 msgr2=0x563c29659b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c29659b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 -- 172.21.15.138:0/3265354700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c29315540 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65200640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c29659610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d65c00640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c2965c1c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009530 tx=0x7f3d54004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f812c8d35359ed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.106+0000 7f3d47400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5402d060 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d58004050 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3d54018030 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3d5401d550 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d6b0bf740 10 client.25247 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c2966af90 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.107+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d58004050 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3d5401c850 con 0x563c29650170 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 10 client.25247 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:58.131 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 10 client.25247 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 20 client.25247 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c2966d5d0 0x563c2966ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 4 client.25247 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c295c8790 con 0x563c2966d5d0 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.108+0000 7f3d65200640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c2966d5d0 0x563c2966ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.109+0000 7f3d65200640 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c2966d5d0 0x563c2966ea40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc300d8c7cd59e56 server_cookie=da4336c2fc770dbd in_seq=0 out_seq=0 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.109+0000 7f3d47400640 10 client.25247 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d47400640 1 -- 172.21.15.138:0/3265354700 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c295c8790 con 0x563c2966d5d0 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d47400640 10 client.25247 handle_command_reply: tid=0 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 client.25247 shutdown 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 msgr2=0x563c2965c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c2965c1c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009530 tx=0x7f3d54004730 comp rx=0 tx=0).stop 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 shutdown_connections 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c2966d5d0 0x563c2966ea40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.132 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c2965ca20 0x563c29659b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d47400640 0 client.25247 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c29650a90 0x563c29659610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/3265354700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c2965c1c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 >> 172.21.15.138:0/3265354700 conn(0x563c2964dfa0 msgr2=0x563c29653b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 shutdown_connections 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/3265354700 wait complete. 2024-08-20T01:07:58.133 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.110+0000 7f3d6b0bf740 20 client.25247 trim_cache size 0 max 16384 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 Processor -- start 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 -- start start 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 0x563c29659580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29659ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c2966cd80 con 0x563c29650170 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c2966d450 con 0x563c29650a90 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c2966db20 con 0x563c2965ca20 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d65c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.137 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d65c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37424/0 (socket says 172.21.15.138:37424) 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d65c00640 1 -- 172.21.15.138:0/1280096879 learned_addr learned my addr 172.21.15.138:0/1280096879 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d65200640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 0x563c29659580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.129+0000 7f3d66600640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29659ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 -- 172.21.15.138:0/1280096879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 msgr2=0x563c29659ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29659ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 -- 172.21.15.138:0/1280096879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 msgr2=0x563c29659580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 0x563c29659580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 -- 172.21.15.138:0/1280096879 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c2964ea20 con 0x563c29650170 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65200640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 0x563c29659580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d65c00640 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d5401cc70 tx=0x7f3d540092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5274b9e59106c44 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d52000640 1 -- 172.21.15.138:0/1280096879 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5403f070 con 0x563c29650170 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d52000640 1 -- 172.21.15.138:0/1280096879 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5401c3a0 con 0x563c29650170 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.130+0000 7f3d52000640 1 -- 172.21.15.138:0/1280096879 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5401d780 con 0x563c29650170 2024-08-20T01:07:58.138 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1280096879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 msgr2=0x563c2964f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d5401cc70 tx=0x7f3d540092d0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1280096879 shutdown_connections 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29659ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650a90 0x563c29659580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/1280096879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650170 0x563c2964f700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1280096879 >> 172.21.15.138:0/1280096879 conn(0x563c2964dfa0 msgr2=0x563c2964eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1280096879 shutdown_connections 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.131+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/1280096879 wait complete. 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 -- start start 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 Processor -- start 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29677550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29673160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563c2966e8c0 con 0x563c29650a90 2024-08-20T01:07:58.139 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563c2966e1f0 con 0x563c29650170 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563c2966ef90 con 0x563c2965ca20 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.132+0000 7f3d6b0bf740 10 client.0 _ll_register_callbacks cb 0x7f3d54009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37436/0 (socket says 172.21.15.138:37436) 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 -- 172.21.15.138:0/255930571 learned_addr learned my addr 172.21.15.138:0/255930571 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65c00640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29677550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d66600640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29673160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 -- 172.21.15.138:0/255930571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 msgr2=0x563c29673160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29673160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 -- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 msgr2=0x563c29677550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29677550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 -- 172.21.15.138:0/255930571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c296d6ce0 con 0x563c29650a90 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d65200640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0c6770 tx=0x7f3d4c02a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e106621d502b562 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:07:58.140 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d47400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c030450 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3d4c003ce0 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.133+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5401c3a0 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f3d4c030e70 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 10 client.15225 fetch_fsmap learned FSMap version 24 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563c296d7ef0 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d5401c3a0 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f3d4c10a030 con 0x563c29650a90 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 10 client.15225 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 20 client.15225 populate_metadata read hostname 'smithi138' 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c296dca80 0x563c296db790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:07:58.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 4 client.15225 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.134+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "status"}) -- 0x563c295c4230 con 0x563c296dca80 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d65c00640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c296dca80 0x563c296db790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d65c00640 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c296dca80 0x563c296db790 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f107b3969d3733b server_cookie=b9ab6b6e63896515 in_seq=0 out_seq=0 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d47400640 10 client.15225 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d47400640 1 -- 172.21.15.138:0/255930571 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563c295c4230 con 0x563c296dca80 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d47400640 10 client.15225 handle_command_reply: tid=0 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d6b0bf740 1 client.15225 shutdown 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 msgr2=0x563c29672a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.135+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0c6770 tx=0x7f3d4c02a340 comp rx=0 tx=0).stop 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 shutdown_connections 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563c296dca80 0x563c296db790 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563c2965ca20 0x563c29673160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d47400640 0 client.15225 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:07:58.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563c29650a90 0x563c29672a70 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 --2- 172.21.15.138:0/255930571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563c29650170 0x563c29677550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 >> 172.21.15.138:0/255930571 conn(0x563c2964dfa0 msgr2=0x563c29659580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 shutdown_connections 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 1 -- 172.21.15.138:0/255930571 wait complete. 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d6b0bf740 20 client.15225 trim_cache size 0 max 16384 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24743, 2024-08-20T01:07:58.143 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 149.53003348300001, 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 163.92592416900001 2024-08-20T01:07:58.144 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 msgr2=0x7f3d480b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 0x7f3d480b6180 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x563c296532f0 tx=0x7f3d54009260 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 msgr2=0x7f3d6019b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.136+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c035040 tx=0x7f3d4c032d50 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 shutdown_connections 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3d480b3cc0 0x7f3d480b6180 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3d60108ca0 0x7f3d601a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d60108320 0x7f3d6019b500 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 --2- 172.21.15.138:0/965734516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d60102910 0x7f3d6019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 >> 172.21.15.138:0/965734516 conn(0x7f3d600fe3c0 msgr2=0x7f3d60105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 shutdown_connections 2024-08-20T01:07:58.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:07:58.137+0000 7f3d67000640 1 -- 172.21.15.138:0/965734516 wait complete. 2024-08-20T01:07:58.986 INFO:tasks.cephadm:Setup complete, yielding 2024-08-20T01:07:58.986 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T01:07:58.997 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:07:58.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph orch status' 2024-08-20T01:07:59.366 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:07:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:07:59 smithi138 ceph-mon[18076]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:07:59 smithi167 ceph-mon[21965]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:07:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:07:59 smithi142 ceph-mon[21799]: pgmap v358: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:00.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 -- 172.21.15.138:0/888961861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc1084c0 msgr2=0x7f8dbc108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:00.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/888961861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc108900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8da8009900 tx=0x7f8da802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 -- 172.21.15.138:0/888961861 shutdown_connections 2024-08-20T01:08:00.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/888961861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 0x7f8dbc10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/888961861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc108900 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/888961861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 -- 172.21.15.138:0/888961861 >> 172.21.15.138:0/888961861 conn(0x7f8dbc0fe500 msgr2=0x7f8dbc100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 -- 172.21.15.138:0/888961861 shutdown_connections 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.008+0000 7f8dc1e00640 1 -- 172.21.15.138:0/888961861 wait complete. 2024-08-20T01:08:00.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 Processor -- start 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 -- start start 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 0x7f8dbc1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112b50 con 0x7f8dbc102ab0 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc1129d0 con 0x7f8dbc108e40 2024-08-20T01:08:00.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.009+0000 7f8dc1e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8dbc112cd0 con 0x7f8dbc1084c0 2024-08-20T01:08:00.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:00.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:00.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dbaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37466/0 (socket says 172.21.15.138:37466) 2024-08-20T01:08:00.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dbaa00640 1 -- 172.21.15.138:0/1764215439 learned_addr learned my addr 172.21.15.138:0/1764215439 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dbb400640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 0x7f8dbc1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 -- 172.21.15.138:0/1764215439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 msgr2=0x7f8dbc1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 0x7f8dbc1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 -- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 msgr2=0x7f8dbc19b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc19b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.010+0000 7f8dba000640 1 -- 172.21.15.138:0/1764215439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dbc1a1240 con 0x7f8dbc1084c0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dba000640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc19b6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8da802a9e0 tx=0x7f8da8003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da803f070 con 0x7f8dbc1084c0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da8031b50 con 0x7f8dbc1084c0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8da8035030 con 0x7f8dbc1084c0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dbc1a14d0 con 0x7f8dbc1084c0 2024-08-20T01:08:00.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.011+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dbc1a7bf0 con 0x7f8dbc1084c0 2024-08-20T01:08:00.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.012+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dbc102e90 con 0x7f8dbc1084c0 2024-08-20T01:08:00.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.015+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f8da8042070 con 0x7f8dbc1084c0 2024-08-20T01:08:00.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.017+0000 7f8dc0a00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 0x7f8da00b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:00.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.017+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8da8104820 con 0x7f8dbc1084c0 2024-08-20T01:08:00.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.017+0000 7f8dbaa00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 0x7f8da00b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:00.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.018+0000 7f8dbaa00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 0x7f8da00b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0049d0 tx=0x7f8dac009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:00.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.018+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8da80cb9a0 con 0x7f8dbc1084c0 2024-08-20T01:08:00.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.217+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f8dbc0632c0 con 0x7f8da00b3ca0 2024-08-20T01:08:00.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.219+0000 7f8dc0a00640 1 -- 172.21.15.138:0/1764215439 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f8dbc0632c0 con 0x7f8da00b3ca0 2024-08-20T01:08:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:Backend: cephadm 2024-08-20T01:08:00.218 INFO:teuthology.orchestra.run.smithi138.stdout:Available: Yes 2024-08-20T01:08:00.219 INFO:teuthology.orchestra.run.smithi138.stdout:Paused: No 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 msgr2=0x7f8da00b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 0x7f8da00b6160 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0049d0 tx=0x7f8dac009500 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 msgr2=0x7f8dbc19b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc19b6d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8da802a9e0 tx=0x7f8da8003b90 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 shutdown_connections 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8da00b3ca0 0x7f8da00b6160 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8dbc108e40 0x7f8dbc1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8dbc1084c0 0x7f8dbc19b6d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 --2- 172.21.15.138:0/1764215439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8dbc102ab0 0x7f8dbc19b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.221+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 >> 172.21.15.138:0/1764215439 conn(0x7f8dbc0fe500 msgr2=0x7f8dbc104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.222+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 shutdown_connections 2024-08-20T01:08:00.221 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:00.222+0000 7f8dc1e00640 1 -- 172.21.15.138:0/1764215439 wait complete. 2024-08-20T01:08:01.034 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ps' 2024-08-20T01:08:01.403 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:01.800 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:01 smithi138 ceph-mon[18076]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:01.800 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:01 smithi138 ceph-mon[18076]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:01 smithi167 ceph-mon[21965]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:01 smithi167 ceph-mon[21965]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:01 smithi142 ceph-mon[21799]: from='client.25240 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:01 smithi142 ceph-mon[21799]: pgmap v359: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 -- 172.21.15.138:0/458942896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c102ab0 msgr2=0x7f083c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 --2- 172.21.15.138:0/458942896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c102ab0 0x7f083c102e90 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f082c009900 tx=0x7f082c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 -- 172.21.15.138:0/458942896 shutdown_connections 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 --2- 172.21.15.138:0/458942896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c108e40 0x7f083c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 --2- 172.21.15.138:0/458942896 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c1084c0 0x7f083c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 --2- 172.21.15.138:0/458942896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c102ab0 0x7f083c102e90 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 -- 172.21.15.138:0/458942896 >> 172.21.15.138:0/458942896 conn(0x7f083c0fe500 msgr2=0x7f083c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.030+0000 7f0841600640 1 -- 172.21.15.138:0/458942896 shutdown_connections 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.031+0000 7f0841600640 1 -- 172.21.15.138:0/458942896 wait complete. 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.031+0000 7f0841600640 1 Processor -- start 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 -- start start 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102ab0 0x7f083c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c108e40 0x7f083c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:02.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f083c112b10 con 0x7f083c1084c0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f083c112990 con 0x7f083c102ab0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f0841600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f083c112c90 con 0x7f083c108e40 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f083a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f083a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37496/0 (socket says 172.21.15.138:37496) 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f083a000640 1 -- 172.21.15.138:0/3198769417 learned_addr learned my addr 172.21.15.138:0/3198769417 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f083b400640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c108e40 0x7f083c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.032+0000 7f083aa00640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102ab0 0x7f083c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 -- 172.21.15.138:0/3198769417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c108e40 msgr2=0x7f083c19e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c108e40 0x7f083c19e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 -- 172.21.15.138:0/3198769417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102ab0 msgr2=0x7f083c198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102ab0 0x7f083c198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 -- 172.21.15.138:0/3198769417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c19f030 con 0x7f083c1084c0 2024-08-20T01:08:02.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f083a000640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f082800e9f0 tx=0x7f082800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082800cd80 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08280108e0 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c19f320 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0828018980 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.033+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c1a59e0 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.034+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c102e90 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.036+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f0828010430 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.038+0000 7f0833400640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 0x7f08200b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.039+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f08280e4430 con 0x7f083c1084c0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.039+0000 7f083aa00640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 0x7f08200b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.039+0000 7f083aa00640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 0x7f08200b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f082c005ab0 tx=0x7f082c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:02.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.043+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f08280aaf60 con 0x7f083c1084c0 2024-08-20T01:08:02.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.234+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f083c0632c0 con 0x7f08200b3c20 2024-08-20T01:08:02.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.248+0000 7f0833400640 1 -- 172.21.15.138:0/3198769417 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f083c0632c0 con 0x7f08200b3c20 2024-08-20T01:08:02.295 INFO:teuthology.orchestra.run.smithi138.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.a smithi138 running (2m) 21s ago 2m 13.2M - 19.0.0-5872-gef427541 9f5c88d94bb6 c27bb9affd98 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.b smithi142 running (2m) 36s ago 2m 25.7M - 19.0.0-5872-gef427541 9f5c88d94bb6 1e605f686df8 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.c smithi167 running (3m) 36s ago 3m 20.5M - 19.0.0-5872-gef427541 9f5c88d94bb6 bfc525c668aa 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.d smithi138 running (3m) 21s ago 3m 13.3M - 19.0.0-5872-gef427541 9f5c88d94bb6 3667677061f0 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.e smithi142 running (3m) 36s ago 3m 20.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 ba3e383ad1a1 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.f smithi167 running (3m) 36s ago 3m 14.1M - 19.0.0-5872-gef427541 9f5c88d94bb6 faf0ca144375 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.g smithi138 running (2m) 21s ago 2m 17.8M - 19.0.0-5872-gef427541 9f5c88d94bb6 29665e90289f 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.h smithi142 running (3m) 36s ago 3m 19.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 32c560fc5f41 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.i smithi167 running (2m) 36s ago 2m 19.5M - 19.0.0-5872-gef427541 9f5c88d94bb6 b07db5da6ce8 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.j smithi138 running (3m) 21s ago 3m 13.3M - 19.0.0-5872-gef427541 9f5c88d94bb6 44f7f81bd3ce 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.k smithi142 running (3m) 36s ago 3m 15.0M - 19.0.0-5872-gef427541 9f5c88d94bb6 ce44b9a59e15 2024-08-20T01:08:02.296 INFO:teuthology.orchestra.run.smithi138.stdout:mds.l smithi167 running (3m) 36s ago 3m 14.8M - 19.0.0-5872-gef427541 9f5c88d94bb6 21b5e2d66fae 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mgr.x smithi138 *:9283,8765 running (11m) 21s ago 11m 519M - 19.0.0-5872-gef427541 9f5c88d94bb6 1b4a09a97cbd 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mgr.y smithi142 *:8443,8765 running (9m) 36s ago 9m 456M - 19.0.0-5872-gef427541 9f5c88d94bb6 56c9043d2154 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mgr.z smithi167 *:8443,8765 running (9m) 36s ago 9m 456M - 19.0.0-5872-gef427541 9f5c88d94bb6 e8812edead27 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mon.a smithi138 running (11m) 21s ago 11m 49.7M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 41ad98b5aaea 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mon.b smithi142 running (9m) 36s ago 9m 44.0M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 df50ac551d24 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:mon.c smithi167 running (9m) 36s ago 9m 42.4M 2048M 19.0.0-5872-gef427541 9f5c88d94bb6 f61e619d275c 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 smithi138 running (8m) 21s ago 8m 47.2M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 368db4e02822 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 smithi142 running (8m) 36s ago 8m 47.0M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 b972917eb59d 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2 smithi167 running (7m) 36s ago 7m 49.7M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 f724e22e2393 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.3 smithi138 running (7m) 21s ago 7m 42.6M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 7c1d6b704a3d 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.4 smithi142 running (6m) 36s ago 6m 45.9M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 258bb83c72d3 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.5 smithi167 running (6m) 36s ago 6m 44.6M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 38d2f5814b37 2024-08-20T01:08:02.297 INFO:teuthology.orchestra.run.smithi138.stdout:osd.6 smithi138 running (5m) 21s ago 5m 42.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 844633c85995 2024-08-20T01:08:02.298 INFO:teuthology.orchestra.run.smithi138.stdout:osd.7 smithi142 running (5m) 36s ago 5m 56.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 e138f425d54a 2024-08-20T01:08:02.298 INFO:teuthology.orchestra.run.smithi138.stdout:osd.8 smithi167 running (5m) 36s ago 5m 44.3M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 68806bf13213 2024-08-20T01:08:02.298 INFO:teuthology.orchestra.run.smithi138.stdout:osd.9 smithi138 running (4m) 21s ago 4m 53.9M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 4e434b3c058f 2024-08-20T01:08:02.298 INFO:teuthology.orchestra.run.smithi138.stdout:osd.10 smithi142 running (4m) 36s ago 4m 52.5M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 d31acba36eb6 2024-08-20T01:08:02.298 INFO:teuthology.orchestra.run.smithi138.stdout:osd.11 smithi167 running (3m) 36s ago 3m 55.4M 4272M 19.0.0-5872-gef427541 9f5c88d94bb6 464de6d2d014 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.250+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 msgr2=0x7f08200b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.250+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 0x7f08200b60e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f082c005ab0 tx=0x7f082c005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 msgr2=0x7f083c1994c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f082800e9f0 tx=0x7f082800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 shutdown_connections 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08200b3c20 0x7f08200b60e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f083c108e40 0x7f083c19e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f083c1084c0 0x7f083c1994c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 --2- 172.21.15.138:0/3198769417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f083c102ab0 0x7f083c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 >> 172.21.15.138:0/3198769417 conn(0x7f083c0fe500 msgr2=0x7f083c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 shutdown_connections 2024-08-20T01:08:02.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:02.251+0000 7f0841600640 1 -- 172.21.15.138:0/3198769417 wait complete. 2024-08-20T01:08:03.122 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph orch ls' 2024-08-20T01:08:03.489 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:03.828 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:03 smithi138 ceph-mon[18076]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:03.828 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:03 smithi138 ceph-mon[18076]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:03.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:03 smithi167 ceph-mon[21965]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:03.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:03 smithi167 ceph-mon[21965]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:03 smithi142 ceph-mon[21799]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:03 smithi142 ceph-mon[21799]: pgmap v360: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 -- 172.21.15.138:0/3326552720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 msgr2=0x7f2e7c105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 --2- 172.21.15.138:0/3326552720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c105080 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009900 tx=0x7f2e6c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 -- 172.21.15.138:0/3326552720 shutdown_connections 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 --2- 172.21.15.138:0/3326552720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10b030 0x7f2e7c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 --2- 172.21.15.138:0/3326552720 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10a6b0 0x7f2e7c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 --2- 172.21.15.138:0/3326552720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c105080 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 -- 172.21.15.138:0/3326552720 >> 172.21.15.138:0/3326552720 conn(0x7f2e7c100730 msgr2=0x7f2e7c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:04.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 -- 172.21.15.138:0/3326552720 shutdown_connections 2024-08-20T01:08:04.081 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.080+0000 7f2e81e00640 1 -- 172.21.15.138:0/3326552720 wait complete. 2024-08-20T01:08:04.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.081+0000 7f2e81e00640 1 Processor -- start 2024-08-20T01:08:04.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.081+0000 7f2e81e00640 1 -- start start 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10a6b0 0x7f2e7c10e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10b030 0x7f2e7c10edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c114cf0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c114b70 con 0x7f2e7c10a6b0 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e81e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2e7c114e70 con 0x7f2e7c10b030 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e7aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e7aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37516/0 (socket says 172.21.15.138:37516) 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e7aa00640 1 -- 172.21.15.138:0/463619489 learned_addr learned my addr 172.21.15.138:0/463619489 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e7a000640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10a6b0 0x7f2e7c10e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.082+0000 7f2e7b400640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10b030 0x7f2e7c10edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 -- 172.21.15.138:0/463619489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10b030 msgr2=0x7f2e7c10edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:04.083 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10b030 0x7f2e7c10edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 -- 172.21.15.138:0/463619489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10a6b0 msgr2=0x7f2e7c10e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10a6b0 0x7f2e7c10e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 -- 172.21.15.138:0/463619489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e7c1ab9d0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e7aa00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005940 tx=0x7f2e6c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c03f030 con 0x7f2e7c104ca0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e6c0044c0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e7c1abba0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c035070 con 0x7f2e7c104ca0 2024-08-20T01:08:04.084 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.083+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e7c1abff0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.085+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e7c105080 con 0x7f2e7c104ca0 2024-08-20T01:08:04.085 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.086+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f2e6c042030 con 0x7f2e7c104ca0 2024-08-20T01:08:04.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.089+0000 7f2e80a00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 0x7f2e600b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:04.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.089+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2e6c1043c0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.089+0000 7f2e7a000640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 0x7f2e600b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:04.088 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.090+0000 7f2e7a000640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 0x7f2e600b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c111350 tx=0x7f2e68005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:04.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.093+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2e6c0caef0 con 0x7f2e7c104ca0 2024-08-20T01:08:04.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.284+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2e7c0632c0 con 0x7f2e600b3ca0 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.289+0000 7f2e80a00640 1 -- 172.21.15.138:0/463619489 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2e7c0632c0 con 0x7f2e600b3ca0 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stdout:mds.all 12/12 38s ago 3m smithi138=a;smithi138=d;smithi138=g;smithi138=j;smithi142=b;smithi142=e;smithi142=h;smithi142=k;smithi167=c;smithi167=f;smithi167=i;smithi167=l;count:12 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stdout:mgr 3/3 38s ago 9m smithi138=x;smithi142=y;smithi167=z;count:3 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stdout:mon 3/3 38s ago 9m smithi138:172.21.15.138=a;smithi142:172.21.15.142=b;smithi167:172.21.15.167=c;count:3 2024-08-20T01:08:04.289 INFO:teuthology.orchestra.run.smithi138.stdout:osd 12 38s ago - 2024-08-20T01:08:04.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 msgr2=0x7f2e600b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 0x7f2e600b6160 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c111350 tx=0x7f2e68005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 msgr2=0x7f2e7c111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c005940 tx=0x7f2e6c0040c0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 shutdown_connections 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2e600b3ca0 0x7f2e600b6160 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e7c10b030 0x7f2e7c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e7c10a6b0 0x7f2e7c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 --2- 172.21.15.138:0/463619489 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e7c104ca0 0x7f2e7c111600 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 >> 172.21.15.138:0/463619489 conn(0x7f2e7c100730 msgr2=0x7f2e7c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 shutdown_connections 2024-08-20T01:08:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:04.292+0000 7f2e81e00640 1 -- 172.21.15.138:0/463619489 wait complete. 2024-08-20T01:08:05.299 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph orch host ls' 2024-08-20T01:08:05.669 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:05.834 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:05 smithi138 ceph-mon[18076]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:05.835 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:05 smithi138 ceph-mon[18076]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:05 smithi167 ceph-mon[21965]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:05 smithi167 ceph-mon[21965]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:05 smithi142 ceph-mon[21799]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:05 smithi142 ceph-mon[21799]: pgmap v361: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 -- 172.21.15.138:0/3832506425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 msgr2=0x7f849410c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 --2- 172.21.15.138:0/3832506425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f849410c620 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8480009900 tx=0x7f848002a410 comp rx=0 tx=0).stop 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 -- 172.21.15.138:0/3832506425 shutdown_connections 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 --2- 172.21.15.138:0/3832506425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f849410cc70 0x7f849410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 --2- 172.21.15.138:0/3832506425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f849410c620 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 --2- 172.21.15.138:0/3832506425 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8494103810 0x7f8494103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.266 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 -- 172.21.15.138:0/3832506425 >> 172.21.15.138:0/3832506425 conn(0x7f84940fd2d0 msgr2=0x7f84940ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:06.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 -- 172.21.15.138:0/3832506425 shutdown_connections 2024-08-20T01:08:06.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.266+0000 7f849aa00640 1 -- 172.21.15.138:0/3832506425 wait complete. 2024-08-20T01:08:06.267 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.267+0000 7f849aa00640 1 Processor -- start 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 -- start start 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8494103810 0x7f84941014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 0x7f8494101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84941128e0 con 0x7f8494104130 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8494112760 con 0x7f8494103810 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f849aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8494112a60 con 0x7f849410cc70 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f8498c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f8498c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37546/0 (socket says 172.21.15.138:37546) 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f8498c00640 1 -- 172.21.15.138:0/20177941 learned_addr learned my addr 172.21.15.138:0/20177941 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.268+0000 7f8499600640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8494103810 0x7f84941014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:06.268 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f849a000640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 0x7f8494101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:06.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 -- 172.21.15.138:0/20177941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 msgr2=0x7f8494101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:06.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 0x7f8494101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 -- 172.21.15.138:0/20177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8494103810 msgr2=0x7f84941014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:06.269 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8494103810 0x7f84941014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 -- 172.21.15.138:0/20177941 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84941a7190 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f849a000640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 0x7f8494101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.269+0000 7f8498c00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8480032fd0 tx=0x7f8480031970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.270+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848003f030 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.270+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8480004410 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.270+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84941a7420 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.270+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8480035070 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.270+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84941a7950 con 0x7f8494104130 2024-08-20T01:08:06.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.271+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8494103bf0 con 0x7f8494104130 2024-08-20T01:08:06.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.274+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f8480042030 con 0x7f8494104130 2024-08-20T01:08:06.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.275+0000 7f8492000640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 0x7f847c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:06.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.275+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f84801046e0 con 0x7f8494104130 2024-08-20T01:08:06.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.275+0000 7f8499600640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 0x7f847c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:06.275 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.276+0000 7f8499600640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 0x7f847c0b6200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8488009bd0 tx=0x7f8488005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:06.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.277+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f84800cb210 con 0x7f8494104130 2024-08-20T01:08:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.476+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f84940632c0 con 0x7f847c0b3d40 2024-08-20T01:08:06.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.478+0000 7f8492000640 1 -- 172.21.15.138:0/20177941 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f84940632c0 con 0x7f847c0b3d40 2024-08-20T01:08:06.477 INFO:teuthology.orchestra.run.smithi138.stdout:HOST ADDR LABELS STATUS 2024-08-20T01:08:06.478 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138 172.21.15.138 2024-08-20T01:08:06.478 INFO:teuthology.orchestra.run.smithi138.stdout:smithi142 172.21.15.142 2024-08-20T01:08:06.478 INFO:teuthology.orchestra.run.smithi138.stdout:smithi167 172.21.15.167 2024-08-20T01:08:06.478 INFO:teuthology.orchestra.run.smithi138.stdout:3 hosts in cluster 2024-08-20T01:08:06.480 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.481+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 msgr2=0x7f847c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.481+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 0x7f847c0b6200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8488009bd0 tx=0x7f8488005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.481+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 msgr2=0x7f8494101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.481+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8480032fd0 tx=0x7f8480031970 comp rx=0 tx=0).stop 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 shutdown_connections 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f847c0b3d40 0x7f847c0b6200 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f849410cc70 0x7f8494101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8494104130 0x7f8494101a00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 --2- 172.21.15.138:0/20177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8494103810 0x7f84941014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 >> 172.21.15.138:0/20177941 conn(0x7f84940fd2d0 msgr2=0x7f84940fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 shutdown_connections 2024-08-20T01:08:06.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:06.482+0000 7f849aa00640 1 -- 172.21.15.138:0/20177941 wait complete. 2024-08-20T01:08:07.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph orch device ls' 2024-08-20T01:08:07.740 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:07.879 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:07 smithi138 ceph-mon[18076]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:07.879 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:07 smithi138 ceph-mon[18076]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:07.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:07 smithi167 ceph-mon[21965]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:07 smithi167 ceph-mon[21965]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:07.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:07 smithi142 ceph-mon[21799]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:07 smithi142 ceph-mon[21799]: pgmap v362: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 -- 172.21.15.138:0/3835840492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 msgr2=0x7f29f8113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 --2- 172.21.15.138:0/3835840492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f8113170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009900 tx=0x7f29e402a410 comp rx=0 tx=0).stop 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 -- 172.21.15.138:0/3835840492 shutdown_connections 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 --2- 172.21.15.138:0/3835840492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 0x7f29f8115aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 --2- 172.21.15.138:0/3835840492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f8113170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 --2- 172.21.15.138:0/3835840492 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f80777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 -- 172.21.15.138:0/3835840492 >> 172.21.15.138:0/3835840492 conn(0x7f29f81006b0 msgr2=0x7f29f8102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 -- 172.21.15.138:0/3835840492 shutdown_connections 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.361+0000 7f29ffe00640 1 -- 172.21.15.138:0/3835840492 wait complete. 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 Processor -- start 2024-08-20T01:08:08.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 -- start start 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 0x7f29f81a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8119100 con 0x7f29f8077d20 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8118f80 con 0x7f29f81136b0 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.362+0000 7f29ffe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29f8119280 con 0x7f29f8077400 2024-08-20T01:08:08.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f819fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34700/0 (socket says 172.21.15.138:34700) 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fca00640 1 -- 172.21.15.138:0/1196529616 learned_addr learned my addr 172.21.15.138:0/1196529616 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fde00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 0x7f29f81a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 -- 172.21.15.138:0/1196529616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 msgr2=0x7f29f81a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 0x7f29f81a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 -- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 msgr2=0x7f29f819fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f819fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 -- 172.21.15.138:0/1196529616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f81a56c0 con 0x7f29f8077400 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.363+0000 7f29fd400640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f819f610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00e9f0 tx=0x7f29ec00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.364+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00ccc0 con 0x7f29f8077400 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.364+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29ec005020 con 0x7f29f8077400 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.364+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f81a59b0 con 0x7f29f8077400 2024-08-20T01:08:08.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.364+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec004a00 con 0x7f29f8077400 2024-08-20T01:08:08.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.364+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f81ac070 con 0x7f29f8077400 2024-08-20T01:08:08.365 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.365+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f80777e0 con 0x7f29f8077400 2024-08-20T01:08:08.366 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.367+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f29ec004ba0 con 0x7f29f8077400 2024-08-20T01:08:08.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.369+0000 7f29f6000640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 0x7f29e00b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:08.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.369+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f29ec0e4140 con 0x7f29f8077400 2024-08-20T01:08:08.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.369+0000 7f29fca00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 0x7f29e00b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:08.368 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.370+0000 7f29fca00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 0x7f29e00b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f29f8115bf0 tx=0x7f29e4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:08.370 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.371+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f29ec0ab350 con 0x7f29f8077400 2024-08-20T01:08:08.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.563+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f29f81152c0 con 0x7f29e00b3c20 2024-08-20T01:08:08.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.565+0000 7f29f6000640 1 -- 172.21.15.138:0/1196529616 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f29f81152c0 con 0x7f29e00b3c20 2024-08-20T01:08:08.564 INFO:teuthology.orchestra.run.smithi138.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-08-20T01:08:08.564 INFO:teuthology.orchestra.run.smithi138.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-08-20T01:08:08.564 INFO:teuthology.orchestra.run.smithi138.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-08-20T01:08:08.564 INFO:teuthology.orchestra.run.smithi138.stdout:smithi167 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330080400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-08-20T01:08:08.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 msgr2=0x7f29e00b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 0x7f29e00b60e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f29f8115bf0 tx=0x7f29e4005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 msgr2=0x7f29f819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f819f610 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00e9f0 tx=0x7f29ec00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 shutdown_connections 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f29e00b3c20 0x7f29e00b60e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f29f81136b0 0x7f29f81a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f29f8077d20 0x7f29f819fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 --2- 172.21.15.138:0/1196529616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f29f8077400 0x7f29f819f610 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 >> 172.21.15.138:0/1196529616 conn(0x7f29f81006b0 msgr2=0x7f29f8102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 shutdown_connections 2024-08-20T01:08:08.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:08.567+0000 7f29ffe00640 1 -- 172.21.15.138:0/1196529616 wait complete. 2024-08-20T01:08:09.506 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-08-20T01:08:09.518 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:08:09.518 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph fs dump' 2024-08-20T01:08:09.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:09 smithi138 ceph-mon[18076]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:09 smithi138 ceph-mon[18076]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:09.890 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:09 smithi167 ceph-mon[21965]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:09 smithi167 ceph-mon[21965]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:09 smithi142 ceph-mon[21799]: pgmap v363: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:09 smithi142 ceph-mon[21799]: from='client.25264 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 -- 172.21.15.138:0/1238182609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a810b030 msgr2=0x7f16a8111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 --2- 172.21.15.138:0/1238182609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a810b030 0x7f16a8111830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009be0 tx=0x7f16a002a6c0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 -- 172.21.15.138:0/1238182609 shutdown_connections 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 --2- 172.21.15.138:0/1238182609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a810b030 0x7f16a8111830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 --2- 172.21.15.138:0/1238182609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 0x7f16a810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 --2- 172.21.15.138:0/1238182609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a8104ca0 0x7f16a8105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 -- 172.21.15.138:0/1238182609 >> 172.21.15.138:0/1238182609 conn(0x7f16a8100730 msgr2=0x7f16a8102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 -- 172.21.15.138:0/1238182609 shutdown_connections 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.478+0000 7f16ad400640 1 -- 172.21.15.138:0/1238182609 wait complete. 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 Processor -- start 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 -- start start 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a8104ca0 0x7f16a819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 0x7f16a819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8114cb0 con 0x7f16a810b030 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8114b30 con 0x7f16a8104ca0 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.479+0000 7f16ad400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8114e30 con 0x7f16a810a6b0 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f169fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 0x7f16a819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:10.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34724/0 (socket says 172.21.15.138:34724) 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 -- 172.21.15.138:0/3647546520 learned_addr learned my addr 172.21.15.138:0/3647546520 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a6800640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a8104ca0 0x7f16a819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 -- 172.21.15.138:0/3647546520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 msgr2=0x7f16a819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 0x7f16a819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 -- 172.21.15.138:0/3647546520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a8104ca0 msgr2=0x7f16a819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a8104ca0 0x7f16a819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 -- 172.21.15.138:0/3647546520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a81a1210 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.480+0000 7f16a7200640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009b30 tx=0x7f16a0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.481+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0040020 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.481+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a0030970 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.481+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a81a14a0 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.481+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0036040 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.481+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a81a7bc0 con 0x7f16a810b030 2024-08-20T01:08:10.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.482+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a8105080 con 0x7f16a810b030 2024-08-20T01:08:10.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.484+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f16a0043020 con 0x7f16a810b030 2024-08-20T01:08:10.488 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.486+0000 7f169f400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 0x7f168c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:10.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.486+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f16a01050b0 con 0x7f16a810b030 2024-08-20T01:08:10.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.487+0000 7f16a6800640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 0x7f168c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:10.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.487+0000 7f16a6800640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 0x7f168c0b60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1698009530 tx=0x7f1698009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:10.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.488+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f16a004a020 con 0x7f16a810b030 2024-08-20T01:08:10.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.710+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f16a819c290 con 0x7f16a810b030 2024-08-20T01:08:10.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.711+0000 7f169f400640 1 -- 172.21.15.138:0/3647546520 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3102 (secure 0 0 0) 0x7f16a819c290 con 0x7f16a810b030 2024-08-20T01:08:10.710 INFO:teuthology.orchestra.run.smithi138.stdout:e24 2024-08-20T01:08:10.710 INFO:teuthology.orchestra.run.smithi138.stdout:btime 2024-08-20T01:06:24:166182+0000 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:legacy client fscid: 1 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:Filesystem 'cephfs' (1) 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:fs_name cephfs 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 24 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:created 2024-08-20T01:05:27.648119+0000 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:modified 2024-08-20T01:06:23.367255+0000 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:tableserver 0 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:root 0 2024-08-20T01:08:10.711 INFO:teuthology.orchestra.run.smithi138.stdout:session_timeout 300 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:session_autoclose 300 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:max_file_size 1099511627776 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:max_xattr_size 65536 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:required_client_features {} 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:last_failure 0 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:last_failure_osd_epoch 0 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:max_mds 3 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:in 0,1,2 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:up {0=24743,1=24706,2=24704} 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:failed 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:damaged 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:stopped 2024-08-20T01:08:10.712 INFO:teuthology.orchestra.run.smithi138.stdout:data_pools [3,4] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:metadata_pool 2 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:inline_data disabled 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:balancer 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:bal_rank_mask -1 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:standby_count_wanted 0 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:qdb_cluster leader: 24743 members: 24704,24743,24706 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.b{0:24743} state up:active seq 5 addr [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.i{0:24694} state up:standby-replay seq 1 addr [v2:172.21.15.167:6838/2113916191,v1:172.21.15.167:6839/2113916191] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.g{1:24706} state up:active seq 19 addr [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.c{1:24679} state up:standby-replay seq 1 addr [v2:172.21.15.167:6836/2270224611,v1:172.21.15.167:6837/2270224611] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.h{2:24704} state up:active seq 21 addr [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.e{2:24674} state up:standby-replay seq 1 addr [v2:172.21.15.142:6834/2224495590,v1:172.21.15.142:6835/2224495590] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:10.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:Standby daemons: 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.j{-1:14634} state up:standby seq 1 addr [v2:172.21.15.138:6834/1131228617,v1:172.21.15.138:6835/1131228617] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.d{-1:14646} state up:standby seq 1 addr [v2:172.21.15.138:6836/3862920840,v1:172.21.15.138:6837/3862920840] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.a{-1:14703} state up:standby seq 1 addr [v2:172.21.15.138:6840/3860533151,v1:172.21.15.138:6841/3860533151] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.l{-1:24625} state up:standby seq 1 addr [v2:172.21.15.167:6832/2791099826,v1:172.21.15.167:6833/2791099826] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.k{-1:24638} state up:standby seq 1 addr [v2:172.21.15.142:6832/3285613354,v1:172.21.15.142:6833/3285613354] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.714 INFO:teuthology.orchestra.run.smithi138.stdout:[mds.f{-1:24673} state up:standby seq 1 addr [v2:172.21.15.167:6834/902364258,v1:172.21.15.167:6835/902364258] compat {c=[1],r=[1],i=[1fff]}] 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.713+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 msgr2=0x7f168c0b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.713+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 0x7f168c0b60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1698009530 tx=0x7f1698009260 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.713+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 msgr2=0x7f16a81a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.713+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009b30 tx=0x7f16a0003c10 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 shutdown_connections 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f168c0b3c20 0x7f168c0b60e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16a810b030 0x7f16a81a0a40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f16a810a6b0 0x7f16a819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 --2- 172.21.15.138:0/3647546520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f16a8104ca0 0x7f16a819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:10.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 >> 172.21.15.138:0/3647546520 conn(0x7f16a8100730 msgr2=0x7f16a8102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:10.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 shutdown_connections 2024-08-20T01:08:10.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:10.714+0000 7f16ad400640 1 -- 172.21.15.138:0/3647546520 wait complete. 2024-08-20T01:08:11.615 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- bash -c 'ceph osd dump' 2024-08-20T01:08:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:11 smithi167 ceph-mon[21965]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:11.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:11 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3647546520' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T01:08:11.985 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:11 smithi142 ceph-mon[21799]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:11 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3647546520' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T01:08:12.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:11 smithi138 ceph-mon[18076]: pgmap v364: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:12.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:11 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3647546520' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 -- 172.21.15.138:0/493384760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770113690 msgr2=0x7f8770115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 --2- 172.21.15.138:0/493384760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770113690 0x7f8770115a80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8768009ca0 tx=0x7f876802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 -- 172.21.15.138:0/493384760 shutdown_connections 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 --2- 172.21.15.138:0/493384760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770113690 0x7f8770115a80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 --2- 172.21.15.138:0/493384760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077d20 0x7f8770113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 --2- 172.21.15.138:0/493384760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770077400 0x7f87700777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 -- 172.21.15.138:0/493384760 >> 172.21.15.138:0/493384760 conn(0x7f8770100730 msgr2=0x7f8770102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 -- 172.21.15.138:0/493384760 shutdown_connections 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.593+0000 7f8776600640 1 -- 172.21.15.138:0/493384760 wait complete. 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 Processor -- start 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 -- start start 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 0x7f877019fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770113690 0x7f87701a4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87701190c0 con 0x7f8770077400 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8770118f40 con 0x7f8770113690 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f8776600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8770119240 con 0x7f8770077d20 2024-08-20T01:08:12.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.594+0000 7f876f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34746/0 (socket says 172.21.15.138:34746) 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 -- 172.21.15.138:0/3844669658 learned_addr learned my addr 172.21.15.138:0/3844669658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876fe00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770113690 0x7f87701a4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f8766a00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 0x7f877019fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 -- 172.21.15.138:0/3844669658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 msgr2=0x7f877019fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 0x7f877019fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 -- 172.21.15.138:0/3844669658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770113690 msgr2=0x7f87701a4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770113690 0x7f87701a4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 -- 172.21.15.138:0/3844669658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87701a5640 con 0x7f8770077400 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f8766a00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 0x7f877019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.595+0000 7f876f400640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f876000e9f0 tx=0x7f876000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:12.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.596+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876000cd80 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.596+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87600054d0 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.596+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8760010690 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.596+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87701a5930 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.596+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87701abff0 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.597+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87700777e0 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.600+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f8760004e90 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.601+0000 7f876cc00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 0x7f87540b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.601+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f87600e45b0 con 0x7f8770077400 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.601+0000 7f8766a00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 0x7f87540b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.602+0000 7f8766a00640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 0x7f87540b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8770103420 tx=0x7f8758005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:12.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.604+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f87600ab730 con 0x7f8770077400 2024-08-20T01:08:12.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.789+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f87701ac380 con 0x7f8770077400 2024-08-20T01:08:12.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.790+0000 7f876cc00640 1 -- 172.21.15.138:0/3844669658 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+6072 (secure 0 0 0) 0x7f87600ab030 con 0x7f8770077400 2024-08-20T01:08:12.790 INFO:teuthology.orchestra.run.smithi138.stdout:epoch 88 2024-08-20T01:08:12.790 INFO:teuthology.orchestra.run.smithi138.stdout:fsid d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:created 2024-08-20T00:56:38.483080+0000 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:modified 2024-08-20T01:06:36.560026+0000 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:crush_version 27 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:full_ratio 0.95 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:backfillfull_ratio 0.9 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:nearfull_ratio 0.85 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:require_min_compat_client luminous 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:min_compat_client luminous 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:require_osd_release squid 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:stretch_mode_enabled false 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-08-20T01:08:12.791 INFO:teuthology.orchestra.run.smithi138.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:max_osd 12 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.0 up in weight 1 up_from 9 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6802/2240185671,v1:172.21.15.138:6803/2240185671] [v2:172.21.15.138:6804/2240185671,v1:172.21.15.138:6805/2240185671] exists,up 27269bf1-da9b-45c9-930c-b9db2c82afb0 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.1 up in weight 1 up_from 14 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/1595380391,v1:172.21.15.142:6801/1595380391] [v2:172.21.15.142:6802/1595380391,v1:172.21.15.142:6803/1595380391] exists,up 1a799e88-8e7f-48d0-a9bf-0c7fef85efd0 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.2 up in weight 1 up_from 19 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6800/3518437214,v1:172.21.15.167:6801/3518437214] [v2:172.21.15.167:6802/3518437214,v1:172.21.15.167:6803/3518437214] exists,up 56932528-00db-4491-b933-3db45acdb6e1 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.3 up in weight 1 up_from 26 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6810/876217371,v1:172.21.15.138:6811/876217371] [v2:172.21.15.138:6812/876217371,v1:172.21.15.138:6813/876217371] exists,up 73bed35d-9793-4b09-ba0b-41150f576f8a 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.4 up in weight 1 up_from 32 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/2294964237,v1:172.21.15.142:6809/2294964237] [v2:172.21.15.142:6810/2294964237,v1:172.21.15.142:6811/2294964237] exists,up 495167fb-1b60-4d94-ba5b-ff307d382779 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6808/808331745,v1:172.21.15.167:6809/808331745] [v2:172.21.15.167:6810/808331745,v1:172.21.15.167:6811/808331745] exists,up 520885a9-d0be-4c97-9744-fd7f32d98544 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.6 up in weight 1 up_from 43 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6818/2049480484,v1:172.21.15.138:6819/2049480484] [v2:172.21.15.138:6820/2049480484,v1:172.21.15.138:6821/2049480484] exists,up b985af05-d307-4762-a760-8ab5fca6da56 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.7 up in weight 1 up_from 48 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/2538307187,v1:172.21.15.142:6817/2538307187] [v2:172.21.15.142:6818/2538307187,v1:172.21.15.142:6819/2538307187] exists,up 03d4be9b-678d-41bc-813e-1e1d5bb2a29f 2024-08-20T01:08:12.792 INFO:teuthology.orchestra.run.smithi138.stdout:osd.8 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6816/3511497686,v1:172.21.15.167:6817/3511497686] [v2:172.21.15.167:6818/3511497686,v1:172.21.15.167:6819/3511497686] exists,up 42122c07-2099-4411-8845-faa6f97fd38d 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:osd.9 up in weight 1 up_from 60 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.138:6826/1914094697,v1:172.21.15.138:6827/1914094697] [v2:172.21.15.138:6828/1914094697,v1:172.21.15.138:6829/1914094697] exists,up a1167294-30ac-465b-a5ff-e50abac4436f 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:osd.10 up in weight 1 up_from 65 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/1588947901,v1:172.21.15.142:6825/1588947901] [v2:172.21.15.142:6826/1588947901,v1:172.21.15.142:6827/1588947901] exists,up 7ee92625-f415-4027-952c-7eb2099d607c 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:osd.11 up in weight 1 up_from 71 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.167:6824/3808767407,v1:172.21.15.167:6825/3808767407] [v2:172.21.15.167:6826/3808767407,v1:172.21.15.167:6827/3808767407] exists,up b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 2.8 [2,11] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 2.b [2,4] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 2.10 [8,11] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 3.11 [4,8] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 3.1a [5,6] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 3.2c [4,9] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 3.38 [3,9] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 3.3c [4,9] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 4.9 [5,8] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 4.16 [9,8] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 4.19 [9,8] 2024-08-20T01:08:12.793 INFO:teuthology.orchestra.run.smithi138.stdout:pg_upmap_items 4.1b [9,10] 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6801/256603896 expires 2024-08-21T00:58:34.377803+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6800/200546042 expires 2024-08-21T00:57:02.889250+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/2534070123 expires 2024-08-21T00:57:02.889250+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6800/256603896 expires 2024-08-21T00:58:34.377803+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/3607751007 expires 2024-08-21T00:58:34.377803+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/3948920804 expires 2024-08-21T00:58:34.377803+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6801/200546042 expires 2024-08-21T00:57:02.889250+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6800/3853414921 expires 2024-08-21T00:57:30.262737+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:6801/3853414921 expires 2024-08-21T00:57:30.262737+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/64005733 expires 2024-08-21T00:57:30.262737+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/1354849701 expires 2024-08-21T00:57:02.889250+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/2736003984 expires 2024-08-21T00:57:30.262737+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/3346890941 expires 2024-08-21T00:58:34.377803+0000 2024-08-20T01:08:12.794 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/4262320942 expires 2024-08-21T00:57:02.889250+0000 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stdout:blocklist 172.21.15.138:0/3120862368 expires 2024-08-21T00:57:30.262737+0000 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 msgr2=0x7f87540b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 0x7f87540b60e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8770103420 tx=0x7f8758005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 msgr2=0x7f877019f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f876000e9f0 tx=0x7f876000edf0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 shutdown_connections 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87540b3c20 0x7f87540b60e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8770113690 0x7f87701a4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8770077d20 0x7f877019fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 --2- 172.21.15.138:0/3844669658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8770077400 0x7f877019f590 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 >> 172.21.15.138:0/3844669658 conn(0x7f8770100730 msgr2=0x7f8770102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.793+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 shutdown_connections 2024-08-20T01:08:12.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:12.794+0000 7f8776600640 1 -- 172.21.15.138:0/3844669658 wait complete. 2024-08-20T01:08:13.626 INFO:teuthology.run_tasks:Running task fs.ready... 2024-08-20T01:08:13.638 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:08:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:13 smithi167 ceph-mon[21965]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:13 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3844669658' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T01:08:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:13 smithi142 ceph-mon[21799]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:13 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3844669658' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T01:08:14.007 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:13 smithi138 ceph-mon[18076]: pgmap v365: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:13 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3844669658' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-08-20T01:08:14.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 -- 172.21.15.138:0/3427215122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d010a6b0 msgr2=0x7f82d010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 --2- 172.21.15.138:0/3427215122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d010a6b0 0x7f82d010aaf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009900 tx=0x7f82bc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 -- 172.21.15.138:0/3427215122 shutdown_connections 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 --2- 172.21.15.138:0/3427215122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010b030 0x7f82d0111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 --2- 172.21.15.138:0/3427215122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d010a6b0 0x7f82d010aaf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 --2- 172.21.15.138:0/3427215122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d0104ca0 0x7f82d0105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 -- 172.21.15.138:0/3427215122 >> 172.21.15.138:0/3427215122 conn(0x7f82d01006f0 msgr2=0x7f82d0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 -- 172.21.15.138:0/3427215122 shutdown_connections 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.641+0000 7f82d6800640 1 -- 172.21.15.138:0/3427215122 wait complete. 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.642+0000 7f82d6800640 1 Processor -- start 2024-08-20T01:08:14.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 -- start start 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010a6b0 0x7f82d019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d010b030 0x7f82d01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0114d40 con 0x7f82d0104ca0 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0114bc0 con 0x7f82d010a6b0 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82d6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0114ec0 con 0x7f82d010b030 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34760/0 (socket says 172.21.15.138:34760) 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82cf400640 1 -- 172.21.15.138:0/432208703 learned_addr learned my addr 172.21.15.138:0/432208703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:14.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82cea00640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010a6b0 0x7f82d019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.643+0000 7f82cfe00640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d010b030 0x7f82d01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 -- 172.21.15.138:0/432208703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d010b030 msgr2=0x7f82d01a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d010b030 0x7f82d01a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 -- 172.21.15.138:0/432208703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010a6b0 msgr2=0x7f82d019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010a6b0 0x7f82d019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 -- 172.21.15.138:0/432208703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d01a1200 con 0x7f82d0104ca0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82cf400640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009d50 tx=0x7f82c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0003b70 con 0x7f82d0104ca0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c0003d10 con 0x7f82d0104ca0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d01a14f0 con 0x7f82d0104ca0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.644+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00108a0 con 0x7f82d0104ca0 2024-08-20T01:08:14.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.645+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01a7bb0 con 0x7f82d0104ca0 2024-08-20T01:08:14.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.646+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d0105080 con 0x7f82d0104ca0 2024-08-20T01:08:14.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.648+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f82c0004000 con 0x7f82d0104ca0 2024-08-20T01:08:14.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.649+0000 7f82c7e00640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 0x7f82b40b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:14.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.649+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82c00e4800 con 0x7f82d0104ca0 2024-08-20T01:08:14.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.649+0000 7f82cea00640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 0x7f82b40b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:14.649 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.650+0000 7f82cea00640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 0x7f82b40b60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005ab0 tx=0x7f82bc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:14.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.653+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f82c00ab980 con 0x7f82d0104ca0 2024-08-20T01:08:14.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.873+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82d01a7fb0 con 0x7f82d0104ca0 2024-08-20T01:08:14.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.874+0000 7f82c7e00640 1 -- 172.21.15.138:0/432208703 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f82c00ab280 con 0x7f82d0104ca0 2024-08-20T01:08:14.873 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:14.874 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:08:14.874 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.876+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 msgr2=0x7f82b40b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.876+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 0x7f82b40b60e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f82bc005ab0 tx=0x7f82bc005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.876+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 msgr2=0x7f82d019b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.876+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009d50 tx=0x7f82c000c6a0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 shutdown_connections 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82b40b3c20 0x7f82b40b60e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82d010b030 0x7f82d01a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82d010a6b0 0x7f82d019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 --2- 172.21.15.138:0/432208703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82d0104ca0 0x7f82d019b150 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:14.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 >> 172.21.15.138:0/432208703 conn(0x7f82d01006f0 msgr2=0x7f82d0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:14.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 shutdown_connections 2024-08-20T01:08:14.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:14.877+0000 7f82d6800640 1 -- 172.21.15.138:0/432208703 wait complete. 2024-08-20T01:08:15.689 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:08:15.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:15 smithi167 ceph-mon[21965]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:15 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/432208703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:15 smithi142 ceph-mon[21799]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:15 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/432208703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:15 smithi138 ceph-mon[18076]: pgmap v366: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:15 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/432208703' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:16.061 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:16.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 -- 172.21.15.138:0/2106445779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075b90 msgr2=0x7f181c1132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 --2- 172.21.15.138:0/2106445779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075b90 0x7f181c1132d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1808009900 tx=0x7f180802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 -- 172.21.15.138:0/2106445779 shutdown_connections 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 --2- 172.21.15.138:0/2106445779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 0x7f181c115c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 --2- 172.21.15.138:0/2106445779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075b90 0x7f181c1132d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 --2- 172.21.15.138:0/2106445779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075270 0x7f181c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 -- 172.21.15.138:0/2106445779 >> 172.21.15.138:0/2106445779 conn(0x7f181c0fe500 msgr2=0x7f181c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 -- 172.21.15.138:0/2106445779 shutdown_connections 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.683+0000 7f1822400640 1 -- 172.21.15.138:0/2106445779 wait complete. 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.684+0000 7f1822400640 1 Processor -- start 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.684+0000 7f1822400640 1 -- start start 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.684+0000 7f1822400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:16.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f1822400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075b90 0x7f181c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f1822400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 0x7f181c1a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f1822400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f181c119090 con 0x7f181c075270 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f1822400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f181c118f10 con 0x7f181c075b90 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f1822400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f181c119210 con 0x7f181c113810 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34786/0 (socket says 172.21.15.138:34786) 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 -- 172.21.15.138:0/1518236749 learned_addr learned my addr 172.21.15.138:0/1518236749 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181be00640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 0x7f181c1a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181aa00640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075b90 0x7f181c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 -- 172.21.15.138:0/1518236749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 msgr2=0x7f181c1a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 0x7f181c1a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 -- 172.21.15.138:0/1518236749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075b90 msgr2=0x7f181c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:16.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075b90 0x7f181c19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.685+0000 7f181b400640 1 -- 172.21.15.138:0/1518236749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181c1a56d0 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f181b400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f180c00e9f0 tx=0x7f180c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c00cd80 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f180c0108e0 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c018980 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181c1a59c0 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.686+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f181c1ac080 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.687+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181c075650 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.689+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f180c010430 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.690+0000 7f1813e00640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 0x7f18000b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.691+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f180c0e4360 con 0x7f181c075270 2024-08-20T01:08:16.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.691+0000 7f181aa00640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 0x7f18000b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:16.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.691+0000 7f181aa00640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 0x7f18000b6160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1808031440 tx=0x7f1808005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:16.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.695+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f180c0aaef0 con 0x7f181c075270 2024-08-20T01:08:16.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.915+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f181c1a0750 con 0x7f181c075270 2024-08-20T01:08:16.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.916+0000 7f1813e00640 1 -- 172.21.15.138:0/1518236749 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11932 (secure 0 0 0) 0x7f180c0af440 con 0x7f181c075270 2024-08-20T01:08:16.915 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:16.916 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":24,"btime":"2024-08-20T01:06:24:166182+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:06:23.367255+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:08:16.916 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 24 2024-08-20T01:08:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 msgr2=0x7f18000b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:16.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 0x7f18000b6160 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1808031440 tx=0x7f1808005b50 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 msgr2=0x7f181c19f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f180c00e9f0 tx=0x7f180c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 shutdown_connections 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18000b3ca0 0x7f18000b6160 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f181c113810 0x7f181c1a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f181c075b90 0x7f181c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 --2- 172.21.15.138:0/1518236749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f181c075270 0x7f181c19f620 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 >> 172.21.15.138:0/1518236749 conn(0x7f181c0fe500 msgr2=0x7f181c0ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 shutdown_connections 2024-08-20T01:08:16.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:16.919+0000 7f1822400640 1 -- 172.21.15.138:0/1518236749 wait complete. 2024-08-20T01:08:17.729 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:08:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:17 smithi167 ceph-mon[21965]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:17.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:17 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1518236749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:17 smithi142 ceph-mon[21799]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:17 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1518236749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:17 smithi138 ceph-mon[18076]: pgmap v367: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:17 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1518236749' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:18.096 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:18.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 -- 172.21.15.138:0/1290154861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108330 msgr2=0x7fa3f0108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:18.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/1290154861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108330 0x7fa3f0108770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009900 tx=0x7fa3dc02a410 comp rx=0 tx=0).stop 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 -- 172.21.15.138:0/1290154861 shutdown_connections 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/1290154861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0108cb0 0x7fa3f010f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/1290154861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108330 0x7fa3f0108770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/1290154861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0102920 0x7fa3f0102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 -- 172.21.15.138:0/1290154861 >> 172.21.15.138:0/1290154861 conn(0x7fa3f00fe350 msgr2=0x7fa3f0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.719+0000 7fa3f4a00640 1 -- 172.21.15.138:0/1290154861 shutdown_connections 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.720+0000 7fa3f4a00640 1 -- 172.21.15.138:0/1290154861 wait complete. 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.720+0000 7fa3f4a00640 1 Processor -- start 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 -- start start 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0102920 0x7fa3f00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0108330 0x7fa3f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:18.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112980 con 0x7fa3f0108cb0 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112800 con 0x7fa3f0108330 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3f4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112b00 con 0x7fa3f0102920 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3ef200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3ef200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57452/0 (socket says 172.21.15.138:57452) 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3ef200640 1 -- 172.21.15.138:0/296687870 learned_addr learned my addr 172.21.15.138:0/296687870 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3ede00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0108330 0x7fa3f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.721+0000 7fa3ee800640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0102920 0x7fa3f00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 -- 172.21.15.138:0/296687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0102920 msgr2=0x7fa3f00751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0102920 0x7fa3f00751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 -- 172.21.15.138:0/296687870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0108330 msgr2=0x7fa3f0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0108330 0x7fa3f0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 -- 172.21.15.138:0/296687870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f01a7150 con 0x7fa3f0108cb0 2024-08-20T01:08:18.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3ef200640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800eb10 tx=0x7fa3e800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004b20 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e8004cc0 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8010670 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01a7440 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.722+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01a7950 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.723+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0102d00 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.726+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fa3e8005020 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.727+0000 7fa3e7400640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 0x7fa3d40b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.727+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fa3e80e46b0 con 0x7fa3f0108cb0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.727+0000 7fa3ee800640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 0x7fa3d40b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.728+0000 7fa3ee800640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 0x7fa3d40b6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009bd0 tx=0x7fa3e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:18.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.730+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fa3e80ab830 con 0x7fa3f0108cb0 2024-08-20T01:08:18.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.918+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3f01a7db0 con 0x7fa3f0108cb0 2024-08-20T01:08:18.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.919+0000 7fa3e7400640 1 -- 172.21.15.138:0/296687870 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7fa3e80ab130 con 0x7fa3f0108cb0 2024-08-20T01:08:18.918 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:18.918 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 msgr2=0x7fa3d40b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 0x7fa3d40b6180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009bd0 tx=0x7fa3e0005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 msgr2=0x7fa3f0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800eb10 tx=0x7fa3e800c2d0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 shutdown_connections 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa3d40b3cc0 0x7fa3d40b6180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3f0108cb0 0x7fa3f0075c50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f0108330 0x7fa3f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 --2- 172.21.15.138:0/296687870 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa3f0102920 0x7fa3f00751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:18.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 >> 172.21.15.138:0/296687870 conn(0x7fa3f00fe350 msgr2=0x7fa3f0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:18.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 shutdown_connections 2024-08-20T01:08:18.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:18.922+0000 7fa3f4a00640 1 -- 172.21.15.138:0/296687870 wait complete. 2024-08-20T01:08:19.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T01:05:27.648119+0000', 'modified': '2024-08-20T01:06:23.367255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24743, 'mds_1': 24706, 'mds_2': 24704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2224495590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2224495590}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2224495590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2270224611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270224611}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270224611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/2113916191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 2113916191}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 2113916191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.142:6837/1103910039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1103910039}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1103910039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.138:6839/1714719372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1714719372}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1714719372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.142:6839/80285616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 80285616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 80285616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24743, 'qdb_cluster': [24704, 24743, 24706]} 2024-08-20T01:08:19.820 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-20T01:08:19.820 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T01:08:19.868 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:19 smithi138 ceph-mon[18076]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:19.869 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:19 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/296687870' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:19 smithi167 ceph-mon[21965]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:19 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/296687870' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:19 smithi142 ceph-mon[21799]: pgmap v368: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:19 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/296687870' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:20.189 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:20.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.790+0000 7fd563e00640 1 -- 172.21.15.138:0/1736978733 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580318a0 con 0x7fd564102ab0 2024-08-20T01:08:20.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 -- 172.21.15.138:0/1736978733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 msgr2=0x7fd564102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 --2- 172.21.15.138:0/1736978733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564102e90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd558009900 tx=0x7fd55802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 -- 172.21.15.138:0/1736978733 shutdown_connections 2024-08-20T01:08:20.872 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 --2- 172.21.15.138:0/1736978733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd564108e40 0x7fd56410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 --2- 172.21.15.138:0/1736978733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5641084c0 0x7fd564108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 --2- 172.21.15.138:0/1736978733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564102e90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 -- 172.21.15.138:0/1736978733 >> 172.21.15.138:0/1736978733 conn(0x7fd5640fe500 msgr2=0x7fd564100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 -- 172.21.15.138:0/1736978733 shutdown_connections 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.791+0000 7fd56c400640 1 -- 172.21.15.138:0/1736978733 wait complete. 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 Processor -- start 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 -- start start 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5641084c0 0x7fd5641994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd564108e40 0x7fd56419e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd564112b00 con 0x7fd564102ab0 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd564112980 con 0x7fd564108e40 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56c400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd564112c80 con 0x7fd5641084c0 2024-08-20T01:08:20.873 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd569a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd569a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57476/0 (socket says 172.21.15.138:57476) 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd569a00640 1 -- 172.21.15.138:0/3566441332 learned_addr learned my addr 172.21.15.138:0/3566441332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.792+0000 7fd56a400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd564108e40 0x7fd56419e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569000640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5641084c0 0x7fd5641994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 -- 172.21.15.138:0/3566441332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5641084c0 msgr2=0x7fd5641994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5641084c0 0x7fd5641994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 -- 172.21.15.138:0/3566441332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd564108e40 msgr2=0x7fd56419e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd564108e40 0x7fd56419e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 -- 172.21.15.138:0/3566441332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56419f020 con 0x7fd564102ab0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd569a00640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd558005940 tx=0x7fd5580040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd562000640 1 -- 172.21.15.138:0/3566441332 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55803f030 con 0x7fd564102ab0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd562000640 1 -- 172.21.15.138:0/3566441332 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5580044c0 con 0x7fd564102ab0 2024-08-20T01:08:20.874 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56419f2b0 con 0x7fd564102ab0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.793+0000 7fd562000640 1 -- 172.21.15.138:0/3566441332 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558035070 con 0x7fd564102ab0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.794+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5641a59d0 con 0x7fd564102ab0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.796+0000 7fd56eea2740 1 Processor -- start 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 -- start start 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd562000640 1 -- 172.21.15.138:0/3566441332 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fd558042030 con 0x7fd564102ab0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bfbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607a2b63840 con 0x5607a2becf10 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607a2b52d40 con 0x5607a2bf97c0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56eea2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607a2b56d80 con 0x5607a2bed830 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57490/0 (socket says 172.21.15.138:57490) 2024-08-20T01:08:20.875 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd56a400640 1 -- 172.21.15.138:0/168781737 learned_addr learned my addr 172.21.15.138:0/168781737 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd569000640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bfbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.797+0000 7fd569a00640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 -- 172.21.15.138:0/168781737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 msgr2=0x5607a2bf9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 -- 172.21.15.138:0/168781737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 msgr2=0x5607a2bfbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bfbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 -- 172.21.15.138:0/168781737 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a2af91c0 con 0x5607a2becf10 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd569000640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd56a400640 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd55c011440 tx=0x7fd55c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cecc58e8594ae9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd556a00640 1 -- 172.21.15.138:0/168781737 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c004d20 con 0x5607a2becf10 2024-08-20T01:08:20.876 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd556a00640 1 -- 172.21.15.138:0/168781737 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c004050 con 0x5607a2becf10 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.798+0000 7fd556a00640 1 -- 172.21.15.138:0/168781737 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c00f7a0 con 0x5607a2becf10 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd562000640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 0x7fd54c0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd562000640 1 -- 172.21.15.138:0/3566441332 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd5581043c0 con 0x7fd564102ab0 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd569a00640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 0x7fd54c0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 -- 172.21.15.138:0/168781737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 msgr2=0x5607a2bed2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd55c011440 tx=0x7fd55c011900 comp rx=0 tx=0).stop 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 -- 172.21.15.138:0/168781737 shutdown_connections 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bfbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 --2- 172.21.15.138:0/168781737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2becf10 0x5607a2bed2f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 -- 172.21.15.138:0/168781737 >> 172.21.15.138:0/168781737 conn(0x5607a2bead40 msgr2=0x5607a2bebee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.799+0000 7fd56eea2740 1 -- 172.21.15.138:0/168781737 shutdown_connections 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.800+0000 7fd56eea2740 1 -- 172.21.15.138:0/168781737 wait complete. 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.800+0000 7fd569a00640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 0x7fd54c0b6160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd5580cabf0 tx=0x7fd558107e00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.800+0000 7fd56eea2740 1 -- start start 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.800+0000 7fd56eea2740 1 Processor -- start 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2becf10 0x5607a2bf8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bf6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607a29b55f0 con 0x5607a2bed830 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607a2aee170 con 0x5607a2bf97c0 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607a2bfc4c0 con 0x5607a2becf10 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56eea2740 10 client.0 _ll_register_callbacks cb 0x5607a2b4d8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57504/0 (socket says 172.21.15.138:57504) 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 -- 172.21.15.138:0/500128387 learned_addr learned my addr 172.21.15.138:0/500128387 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56a400640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2becf10 0x5607a2bf8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56a400640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bf6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 -- 172.21.15.138:0/500128387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2becf10 msgr2=0x5607a2bf8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2becf10 0x5607a2bf8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 -- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 msgr2=0x5607a2bf6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bf6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd569000640 1 -- 172.21.15.138:0/500128387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a28b2540 con 0x5607a2bed830 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.801+0000 7fd56a400640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bf6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd569000640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd550012700 tx=0x7fd550012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fe41e397372088 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd54b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550004030 con 0x5607a2bed830 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd5500041d0 con 0x5607a2bed830 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55c004050 con 0x5607a2bed830 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd550025050 con 0x5607a2bed830 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd56eea2740 10 client.15297 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:20.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5607a2c07d90 con 0x5607a2bed830 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.802+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c004050 con 0x5607a2bed830 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd550003920 con 0x5607a2bed830 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 10 client.15297 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 10 client.15297 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 20 client.15297 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c0a3a0 0x5607a2c0b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 4 client.15297 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607a2b65790 con 0x5607a2c0a3a0 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.803+0000 7fd56a400640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c0a3a0 0x5607a2c0b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.804+0000 7fd56a400640 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c0a3a0 0x5607a2c0b810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eed7c42373c0a3e4 server_cookie=a8e8361026cc8b12 in_seq=0 out_seq=0 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.804+0000 7fd54b400640 10 client.15297 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd54b400640 1 -- 172.21.15.138:0/500128387 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607a2b65790 con 0x5607a2c0a3a0 2024-08-20T01:08:20.881 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd54b400640 10 client.15297 handle_command_reply: tid=0 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 client.15297 shutdown 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 msgr2=0x5607a2bf6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd550012700 tx=0x7fd550012bc0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 shutdown_connections 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c0a3a0 0x5607a2c0b810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2bf97c0 0x5607a2bf6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bed830 0x5607a2bf6410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.882 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 --2- 172.21.15.138:0/500128387 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2becf10 0x5607a2bf8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd54b400640 0 client.15297 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 >> 172.21.15.138:0/500128387 conn(0x5607a2bead40 msgr2=0x5607a2bf08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 shutdown_connections 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 1 -- 172.21.15.138:0/500128387 wait complete. 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.805+0000 7fd56eea2740 20 client.15297 trim_cache size 0 max 16384 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 Processor -- start 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 -- start start 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c09b50 con 0x5607a2bf97c0 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c0a220 con 0x5607a2becf10 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56eea2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c0a8f0 con 0x5607a2bed830 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57516/0 (socket says 172.21.15.138:57516) 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56a400640 1 -- 172.21.15.138:0/3192892923 learned_addr learned my addr 172.21.15.138:0/3192892923 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd56a400640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.824+0000 7fd569000640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 -- 172.21.15.138:0/3192892923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 msgr2=0x5607a2c0aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 -- 172.21.15.138:0/3192892923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 msgr2=0x5607a2c13af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 -- 172.21.15.138:0/3192892923 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607a2beb7c0 con 0x5607a2bed830 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd56a400640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd569000640 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf68d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd550003ee0 tx=0x7fd550009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c46cb6c368f5f756 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd556000640 1 -- 172.21.15.138:0/3192892923 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550022040 con 0x5607a2bed830 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd556000640 1 -- 172.21.15.138:0/3192892923 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd550003aa0 con 0x5607a2bed830 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.825+0000 7fd556000640 1 -- 172.21.15.138:0/3192892923 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55000fa70 con 0x5607a2bed830 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- 172.21.15.138:0/3192892923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 msgr2=0x5607a2bf68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf68d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd550003ee0 tx=0x7fd550009640 comp rx=0 tx=0).stop 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- 172.21.15.138:0/3192892923 shutdown_connections 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c13af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2bf68d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 --2- 172.21.15.138:0/3192892923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- 172.21.15.138:0/3192892923 >> 172.21.15.138:0/3192892923 conn(0x5607a2bead40 msgr2=0x5607a2bf61c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- 172.21.15.138:0/3192892923 shutdown_connections 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- 172.21.15.138:0/3192892923 wait complete. 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.826+0000 7fd56eea2740 1 -- start start 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 Processor -- start 2024-08-20T01:08:20.885 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2c0fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c0b690 con 0x5607a2bf97c0 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c0afc0 con 0x5607a2becf10 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607a2c0bd60 con 0x5607a2bed830 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56eea2740 10 client.0 _ll_register_callbacks cb 0x7fd550009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56a400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56a400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57524/0 (socket says 172.21.15.138:57524) 2024-08-20T01:08:20.886 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd56a400640 1 -- 172.21.15.138:0/2661122894 learned_addr learned my addr 172.21.15.138:0/2661122894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.827+0000 7fd569000640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2c0fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 -- 172.21.15.138:0/2661122894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 msgr2=0x5607a2c0fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2c0fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 -- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 msgr2=0x5607a2c0f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 -- 172.21.15.138:0/2661122894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607a2c73ae0 con 0x5607a2bf97c0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd569000640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2c0fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56a400640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0048b0 tx=0x7fd55c011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2845a87f498d9ae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd54b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c01d040 con 0x5607a2bf97c0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550003aa0 con 0x5607a2bf97c0 2024-08-20T01:08:20.887 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.828+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd55c004250 con 0x5607a2bf97c0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd55c004470 con 0x5607a2bf97c0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 10 client.15303 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5607a2c74de0 con 0x5607a2bf97c0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd550003aa0 con 0x5607a2bf97c0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fd55c00f540 con 0x5607a2bf97c0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 10 client.15303 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 20 client.15303 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c797e0 0x5607a2c784f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 4 client.15303 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.829+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "status"}) -- 0x5607a2b61230 con 0x5607a2c797e0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd569000640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c797e0 0x5607a2c784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:20.888 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd569000640 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c797e0 0x5607a2c784f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c8ae7107465f3da server_cookie=d917cf90f01ca216 in_seq=0 out_seq=0 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd54b400640 10 client.15303 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd54b400640 1 -- 172.21.15.138:0/2661122894 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5607a2b61230 con 0x5607a2c797e0 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd54b400640 10 client.15303 handle_command_reply: tid=0 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.830+0000 7fd56eea2740 1 client.15303 shutdown 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 msgr2=0x5607a2c10380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0048b0 tx=0x7fd55c011900 comp rx=0 tx=0).stop 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 shutdown_connections 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607a2c797e0 0x5607a2c784f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607a2bf97c0 0x5607a2c10380 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607a2bed830 0x5607a2c0fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 --2- 172.21.15.138:0/2661122894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607a2becf10 0x5607a2c0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 >> 172.21.15.138:0/2661122894 conn(0x5607a2bead40 msgr2=0x5607a2be95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd54b400640 0 client.15303 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:20.889 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 shutdown_connections 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 1 -- 172.21.15.138:0/2661122894 wait complete. 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.831+0000 7fd56eea2740 20 client.15303 trim_cache size 0 max 16384 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 2, 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24704, 2024-08-20T01:08:20.890 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 121.22934230200001, 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 199.700819817 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 msgr2=0x7fd54c0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 0x7fd54c0b6160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd5580cabf0 tx=0x7fd558107e00 comp rx=0 tx=0).stop 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 msgr2=0x7fd564198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:20.891 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd558005940 tx=0x7fd5580040c0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 shutdown_connections 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd54c0b3ca0 0x7fd54c0b6160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd564108e40 0x7fd56419e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd5641084c0 0x7fd5641994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 --2- 172.21.15.138:0/3566441332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd564102ab0 0x7fd564198f70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 >> 172.21.15.138:0/3566441332 conn(0x7fd5640fe500 msgr2=0x7fd5641008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 shutdown_connections 2024-08-20T01:08:20.892 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:20.832+0000 7fd56c400640 1 -- 172.21.15.138:0/3566441332 wait complete. 2024-08-20T01:08:21.758 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.g status 2024-08-20T01:08:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:21 smithi167 ceph-mon[21965]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:21 smithi142 ceph-mon[21799]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:22.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:21 smithi138 ceph-mon[18076]: pgmap v369: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:22.124 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:22.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 -- 172.21.15.138:0/3764189644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410eaf0 msgr2=0x7fa1a410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/3764189644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410eaf0 0x7fa1a410ef30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009860 tx=0x7fa18c02a340 comp rx=0 tx=0).stop 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 -- 172.21.15.138:0/3764189644 shutdown_connections 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/3764189644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a410f470 0x7fa1a4115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/3764189644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410eaf0 0x7fa1a410ef30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/3764189644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a41090e0 0x7fa1a41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 -- 172.21.15.138:0/3764189644 >> 172.21.15.138:0/3764189644 conn(0x7fa1a4078760 msgr2=0x7fa1a407ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 -- 172.21.15.138:0/3764189644 shutdown_connections 2024-08-20T01:08:22.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.744+0000 7fa1a8c00640 1 -- 172.21.15.138:0/3764189644 wait complete. 2024-08-20T01:08:22.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.745+0000 7fa1a8c00640 1 Processor -- start 2024-08-20T01:08:22.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.745+0000 7fa1a8c00640 1 -- start start 2024-08-20T01:08:22.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a410eaf0 0x7fa1a419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 0x7fa1a41a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4119090 con 0x7fa1a41090e0 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4118f10 con 0x7fa1a410f470 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a8c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4119210 con 0x7fa1a410eaf0 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57544/0 (socket says 172.21.15.138:57544) 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa199400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a410eaf0 0x7fa1a419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a1e00640 1 -- 172.21.15.138:0/1054439086 learned_addr learned my addr 172.21.15.138:0/1054439086 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 0x7fa1a41a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 -- 172.21.15.138:0/1054439086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a410eaf0 msgr2=0x7fa1a419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a410eaf0 0x7fa1a419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 -- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 msgr2=0x7fa1a419f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a2800640 1 -- 172.21.15.138:0/1054439086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a41a56d0 con 0x7fa1a410f470 2024-08-20T01:08:22.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.746+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:22.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa1a2800640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 0x7fa1a41a4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ef00 tx=0x7fa19c00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa19b400640 1 -- 172.21.15.138:0/1054439086 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c004b20 con 0x7fa1a410f470 2024-08-20T01:08:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa19b400640 1 -- 172.21.15.138:0/1054439086 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c003e20 con 0x7fa1a410f470 2024-08-20T01:08:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa19b400640 1 -- 172.21.15.138:0/1054439086 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00b570 con 0x7fa1a410f470 2024-08-20T01:08:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a41a59c0 con 0x7fa1a410f470 2024-08-20T01:08:22.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.747+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41ac080 con 0x7fa1a410f470 2024-08-20T01:08:22.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.750+0000 7fa1ab653740 1 Processor -- start 2024-08-20T01:08:22.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa19b400640 1 -- 172.21.15.138:0/1054439086 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7fa19c01d020 con 0x7fa1a410f470 2024-08-20T01:08:22.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 -- start start 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8cf10 0x5651b0a8d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 0x5651b0a9bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651b0a03840 con 0x5651b0a997c0 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5651b09f2d40 con 0x5651b0a8cf10 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.751+0000 7fa1ab653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651b09f6d80 con 0x5651b0a8d830 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa199400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 0x5651b0a9bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59548/0 (socket says 172.21.15.138:59548) 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 -- 172.21.15.138:0/728427148 learned_addr learned my addr 172.21.15.138:0/728427148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa199400640 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8cf10 0x5651b0a8d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 -- 172.21.15.138:0/728427148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8cf10 msgr2=0x5651b0a8d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8cf10 0x5651b0a8d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 -- 172.21.15.138:0/728427148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 msgr2=0x5651b0a9bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 0x5651b0a9bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 -- 172.21.15.138:0/728427148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b09991c0 con 0x5651b0a8d830 2024-08-20T01:08:22.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa199400640 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 0x5651b0a9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa19b400640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 0x7fa1880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.752+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa194011660 tx=0x7fa194011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f41e7c322a71f43 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa19b400640 1 -- 172.21.15.138:0/1054439086 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fa19c0e3f30 con 0x7fa1a410f470 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa199400640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 0x7fa1880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa193000640 1 -- 172.21.15.138:0/728427148 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19401e070 con 0x5651b0a8d830 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa193000640 1 -- 172.21.15.138:0/728427148 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19400fba0 con 0x5651b0a8d830 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa193000640 1 -- 172.21.15.138:0/728427148 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19400e610 con 0x5651b0a8d830 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa1ab653740 1 -- 172.21.15.138:0/728427148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 msgr2=0x5651b0a99280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa1ab653740 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa194011660 tx=0x7fa194011b20 comp rx=0 tx=0).stop 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.753+0000 7fa199400640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 0x7fa1880b60e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa18c02a930 tx=0x7fa18c02a340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 -- 172.21.15.138:0/728427148 shutdown_connections 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a997c0 0x5651b0a9bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a99280 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 --2- 172.21.15.138:0/728427148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8cf10 0x5651b0a8d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 -- 172.21.15.138:0/728427148 >> 172.21.15.138:0/728427148 conn(0x5651b0a8ad40 msgr2=0x5651b0a8bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 -- 172.21.15.138:0/728427148 shutdown_connections 2024-08-20T01:08:22.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 -- 172.21.15.138:0/728427148 wait complete. 2024-08-20T01:08:22.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 -- start start 2024-08-20T01:08:22.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.754+0000 7fa1ab653740 1 Processor -- start 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a963a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a997c0 0x5651b0a968e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651b08555f0 con 0x5651b0a8cf10 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5651b098e170 con 0x5651b0a997c0 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651b0a9c470 con 0x5651b0a8d830 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1ab653740 10 client.0 _ll_register_callbacks cb 0x5651b09ed8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57562/0 (socket says 172.21.15.138:57562) 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2848674045 learned_addr learned my addr 172.21.15.138:0/2848674045 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:22.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.755+0000 7fa199400640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a2800640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a997c0 0x5651b0a968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2848674045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 msgr2=0x5651b0a963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2848674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a997c0 msgr2=0x5651b0a968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a997c0 0x5651b0a968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2848674045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b0752540 con 0x5651b0a8cf10 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa194011630 tx=0x7fa194019930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b90fc4449016e3c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa187400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194026070 con 0x5651b0a8cf10 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fa19400fd50 con 0x5651b0a8cf10 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.756+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19400fba0 con 0x5651b0a8cf10 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa194032050 con 0x5651b0a8cf10 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 10 client.15315 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:22.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651b0aa7c40 con 0x5651b0a8cf10 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19400fba0 con 0x5651b0a8cf10 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fa194004b30 con 0x5651b0a8cf10 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 10 client.15315 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 10 client.15315 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 20 client.15315 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0aaa020 0x5651b0aab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 4 client.15315 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.757+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651b0a05790 con 0x5651b0aaa020 2024-08-20T01:08:22.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.758+0000 7fa199400640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0aaa020 0x5651b0aab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.758+0000 7fa187400640 10 client.15315 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:22.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.758+0000 7fa199400640 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0aaa020 0x5651b0aab490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2474cd6f4ee3d7c7 server_cookie=9edb52df26d0c199 in_seq=0 out_seq=0 2024-08-20T01:08:22.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa187400640 1 -- 172.21.15.138:0/2848674045 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5651b0a05790 con 0x5651b0aaa020 2024-08-20T01:08:22.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa187400640 10 client.15315 handle_command_reply: tid=0 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 client.15315 shutdown 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 msgr2=0x5651b0a98f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa194011630 tx=0x7fa194019930 comp rx=0 tx=0).stop 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 shutdown_connections 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a997c0 0x5651b0a968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0aaa020 0x5651b0aab490 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a8d830 0x5651b0a963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2848674045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a98f50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa187400640 0 client.15315 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 >> 172.21.15.138:0/2848674045 conn(0x5651b0a8ad40 msgr2=0x5651b0a908e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 shutdown_connections 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.759+0000 7fa1ab653740 1 -- 172.21.15.138:0/2848674045 wait complete. 2024-08-20T01:08:22.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.760+0000 7fa1ab653740 20 client.15315 trim_cache size 0 max 16384 2024-08-20T01:08:22.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.778+0000 7fa1ab653740 1 Processor -- start 2024-08-20T01:08:22.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.778+0000 7fa1ab653740 1 -- start start 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0a8c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0a935b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aa97d0 con 0x5651b0a8cf10 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aa9ea0 con 0x5651b0a8d830 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1ab653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aaa570 con 0x5651b0a997c0 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57568/0 (socket says 172.21.15.138:57568) 2024-08-20T01:08:22.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2816674615 learned_addr learned my addr 172.21.15.138:0/2816674615 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa199400640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0a8c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.779+0000 7fa1a2800640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0a935b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2816674615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 msgr2=0x5651b0a935b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0a935b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2816674615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 msgr2=0x5651b0a8c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0a8c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2816674615 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651b0a95300 con 0x5651b0a8cf10 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa199400640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0a8c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:22.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa194009f30 tx=0x7fa194019b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c57f5bf542fbf3c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa192600640 1 -- 172.21.15.138:0/2816674615 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19402f020 con 0x5651b0a8cf10 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa192600640 1 -- 172.21.15.138:0/2816674615 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1940226a0 con 0x5651b0a8cf10 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.780+0000 7fa192600640 1 -- 172.21.15.138:0/2816674615 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194026070 con 0x5651b0a8cf10 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- 172.21.15.138:0/2816674615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 msgr2=0x5651b0a94c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa194009f30 tx=0x7fa194019b30 comp rx=0 tx=0).stop 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- 172.21.15.138:0/2816674615 shutdown_connections 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0a935b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0a8c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2816674615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0a94c20 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- 172.21.15.138:0/2816674615 >> 172.21.15.138:0/2816674615 conn(0x5651b0a8ad40 msgr2=0x5651b0a8ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- 172.21.15.138:0/2816674615 shutdown_connections 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- 172.21.15.138:0/2816674615 wait complete. 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.781+0000 7fa1ab653740 1 -- start start 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 Processor -- start 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0aaf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0aaf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aab310 con 0x5651b0a8cf10 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aaac40 con 0x5651b0a8d830 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5651b0aab9e0 con 0x5651b0a997c0 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1ab653740 10 client.0 _ll_register_callbacks cb 0x7fa19401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:22.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1a1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57576/0 (socket says 172.21.15.138:57576) 2024-08-20T01:08:22.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa1a1e00640 1 -- 172.21.15.138:0/2699388804 learned_addr learned my addr 172.21.15.138:0/2699388804 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:22.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.782+0000 7fa199400640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0aaf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa1a2800640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0aaf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 -- 172.21.15.138:0/2699388804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 msgr2=0x5651b0aaf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0aaf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 -- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 msgr2=0x5651b0ab3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:22.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 -- 172.21.15.138:0/2699388804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651b0aafd90 con 0x5651b0a8d830 2024-08-20T01:08:22.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:22.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa199400640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0aaf160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005940 tx=0x7fa18c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365d1189d6e16af4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:22.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa187400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:22.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c037040 con 0x5651b0a8d830 2024-08-20T01:08:22.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.783+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1940226a0 con 0x5651b0a8d830 2024-08-20T01:08:22.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fa18c0037f0 con 0x5651b0a8d830 2024-08-20T01:08:22.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa18c066050 con 0x5651b0a8d830 2024-08-20T01:08:22.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 10 client.25349 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:22.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5651b0b14680 con 0x5651b0a8d830 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1940226a0 con 0x5651b0a8d830 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7fa18c0409b0 con 0x5651b0a8d830 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 10 client.25349 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 10 client.25349 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 20 client.25349 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:22.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0b19230 0x5651b0b17f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.784+0000 7fa1ab653740 4 client.25349 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "status"}) -- 0x5651b0a01230 con 0x5651b0b19230 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0b19230 0x5651b0b17f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa1a1e00640 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0b19230 0x5651b0b17f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de443e9a2bc69ca4 server_cookie=1be738a9e2d5bd6a in_seq=0 out_seq=0 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa187400640 10 client.25349 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa187400640 1 -- 172.21.15.138:0/2699388804 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5651b0a01230 con 0x5651b0b19230 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.785+0000 7fa187400640 10 client.25349 handle_command_reply: tid=0 2024-08-20T01:08:22.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 client.25349 shutdown 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 msgr2=0x5651b0aaf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0aaf160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa18c005940 tx=0x7fa18c005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 shutdown_connections 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5651b0b19230 0x5651b0b17f40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5651b0a997c0 0x5651b0aaf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5651b0a8d830 0x5651b0aaf160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 --2- 172.21.15.138:0/2699388804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5651b0a8cf10 0x5651b0ab3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa187400640 0 client.25349 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:22.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 >> 172.21.15.138:0/2699388804 conn(0x5651b0a8ad40 msgr2=0x5651b0a8ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 shutdown_connections 2024-08-20T01:08:22.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 1 -- 172.21.15.138:0/2699388804 wait complete. 2024-08-20T01:08:22.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.786+0000 7fa1ab653740 20 client.25349 trim_cache size 0 max 16384 2024-08-20T01:08:22.795 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 1, 2024-08-20T01:08:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24706, 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 124.19368457900001, 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:08:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:08:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 195.25627996399999 2024-08-20T01:08:22.797 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:22.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 msgr2=0x7fa1880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 0x7fa1880b60e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa18c02a930 tx=0x7fa18c02a340 comp rx=0 tx=0).stop 2024-08-20T01:08:22.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 msgr2=0x7fa1a41a4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:22.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 0x7fa1a41a4f00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00ef00 tx=0x7fa19c00c740 comp rx=0 tx=0).stop 2024-08-20T01:08:22.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 shutdown_connections 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa1880b3c20 0x7fa1880b60e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1a410f470 0x7fa1a41a4f00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa1a410eaf0 0x7fa1a419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 --2- 172.21.15.138:0/1054439086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa1a41090e0 0x7fa1a419f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 >> 172.21.15.138:0/1054439086 conn(0x7fa1a4078760 msgr2=0x7fa1a407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 shutdown_connections 2024-08-20T01:08:22.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:22.787+0000 7fa1a8c00640 1 -- 172.21.15.138:0/1054439086 wait complete. 2024-08-20T01:08:23.709 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T01:08:23.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:23 smithi167 ceph-mon[21965]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:23.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:23 smithi142 ceph-mon[21799]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:23 smithi138 ceph-mon[18076]: pgmap v370: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:24.082 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:24.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 -- 172.21.15.138:0/3427103589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410eaf0 msgr2=0x7f5ea410ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/3427103589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410eaf0 0x7f5ea410ef30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5e90009900 tx=0x7f5e9002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 -- 172.21.15.138:0/3427103589 shutdown_connections 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/3427103589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea410f470 0x7f5ea4115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/3427103589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410eaf0 0x7f5ea410ef30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/3427103589 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea41090e0 0x7f5ea41094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 -- 172.21.15.138:0/3427103589 >> 172.21.15.138:0/3427103589 conn(0x7f5ea4078760 msgr2=0x7f5ea407ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 -- 172.21.15.138:0/3427103589 shutdown_connections 2024-08-20T01:08:24.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.673+0000 7f5ea9a00640 1 -- 172.21.15.138:0/3427103589 wait complete. 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 Processor -- start 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 -- start start 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 0x7f5ea419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410f470 0x7f5ea41a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea41190e0 con 0x7f5ea410f470 2024-08-20T01:08:24.673 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4118f60 con 0x7f5ea41090e0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.674+0000 7f5ea9a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea4119260 con 0x7f5ea410eaf0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410f470 0x7f5ea41a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59606/0 (socket says 172.21.15.138:59606) 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 -- 172.21.15.138:0/585967718 learned_addr learned my addr 172.21.15.138:0/585967718 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 0x7f5ea419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 -- 172.21.15.138:0/585967718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 msgr2=0x7f5ea419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 0x7f5ea419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 -- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410f470 msgr2=0x7f5ea41a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410f470 0x7f5ea41a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 -- 172.21.15.138:0/585967718 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea41a56e0 con 0x7f5ea410eaf0 2024-08-20T01:08:24.674 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 0x7f5ea419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.675+0000 7f5ea2000640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005ab0 tx=0x7f5e90003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.676+0000 7f5e9b400640 1 -- 172.21.15.138:0/585967718 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9003f070 con 0x7f5ea410eaf0 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.676+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea41a5970 con 0x7f5ea410eaf0 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.676+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea41ac090 con 0x7f5ea410eaf0 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.676+0000 7f5e9b400640 1 -- 172.21.15.138:0/585967718 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9003b910 con 0x7f5ea410eaf0 2024-08-20T01:08:24.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.676+0000 7f5e9b400640 1 -- 172.21.15.138:0/585967718 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90049060 con 0x7f5ea410eaf0 2024-08-20T01:08:24.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.679+0000 7f5eac478740 1 Processor -- start 2024-08-20T01:08:24.677 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.679+0000 7f5eac478740 1 -- start start 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.679+0000 7f5eac478740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.679+0000 7f5eac478740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7da280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5e9b400640 1 -- 172.21.15.138:0/585967718 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f5e90042070 con 0x7f5ea410eaf0 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5eac478740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5eac478740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d744840 con 0x55ac7d7cdf10 2024-08-20T01:08:24.678 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5eac478740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d733d40 con 0x55ac7d7da7c0 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5eac478740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d737d80 con 0x55ac7d7ce830 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57618/0 (socket says 172.21.15.138:57618) 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 -- 172.21.15.138:0/4094377779 learned_addr learned my addr 172.21.15.138:0/4094377779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7da280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea2000640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 -- 172.21.15.138:0/4094377779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 msgr2=0x55ac7d7da280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7da280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 -- 172.21.15.138:0/4094377779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 msgr2=0x55ac7d7dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea3400640 1 -- 172.21.15.138:0/4094377779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7d6da1c0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:24.681 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.680+0000 7f5ea2000640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5ea3400640 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009f80 tx=0x7f5e9c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1861f70d2dbdad11 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5e8fe00640 1 -- 172.21.15.138:0/4094377779 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c004d20 con 0x55ac7d7cdf10 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5e8fe00640 1 -- 172.21.15.138:0/4094377779 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c004050 con 0x55ac7d7cdf10 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5e8fe00640 1 -- 172.21.15.138:0/4094377779 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c005280 con 0x55ac7d7cdf10 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5e9b400640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 0x7f5e880b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.681+0000 7f5e9b400640 1 -- 172.21.15.138:0/585967718 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5e901045a0 con 0x7f5ea410eaf0 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 0x7f5e880b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 -- 172.21.15.138:0/4094377779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 msgr2=0x55ac7d7ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009f80 tx=0x7f5e9c0118d0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 -- 172.21.15.138:0/4094377779 shutdown_connections 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7da280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 --2- 172.21.15.138:0/4094377779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7ce2f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 -- 172.21.15.138:0/4094377779 >> 172.21.15.138:0/4094377779 conn(0x55ac7d7cbd40 msgr2=0x55ac7d7ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 -- 172.21.15.138:0/4094377779 shutdown_connections 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5eac478740 1 -- 172.21.15.138:0/4094377779 wait complete. 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.682+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 0x7f5e880b60e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009fd0 tx=0x7f5e94005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 -- start start 2024-08-20T01:08:24.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 Processor -- start 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d5965f0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d6cf170 con 0x55ac7d7ce830 2024-08-20T01:08:24.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7dd470 con 0x55ac7d7da7c0 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.683+0000 7f5eac478740 10 client.0 _ll_register_callbacks cb 0x55ac7d72e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57620/0 (socket says 172.21.15.138:57620) 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2a00640 1 -- 172.21.15.138:0/1192711820 learned_addr learned my addr 172.21.15.138:0/1192711820 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea3400640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 -- 172.21.15.138:0/1192711820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 msgr2=0x55ac7d7d7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7d7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 -- 172.21.15.138:0/1192711820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 msgr2=0x55ac7d7d9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 -- 172.21.15.138:0/1192711820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7d493540 con 0x55ac7d7ce830 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:24.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7d73c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ac7d7cdec0 tx=0x7f5e900cafa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6d822f77485b34 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.684+0000 7f5e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e90104030 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9c004050 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5e901041d0 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5e901043d0 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5eac478740 10 client.25361 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ac7d7e8cf0 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9c004050 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5e900d3c80 con 0x55ac7d7ce830 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.685+0000 7f5eac478740 10 client.25361 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5eac478740 10 client.25361 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5eac478740 20 client.25361 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:24.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d7eb360 0x55ac7d7ec7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5eac478740 4 client.25361 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac7d746790 con 0x55ac7d7eb360 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d7eb360 0x55ac7d7ec7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d7eb360 0x55ac7d7ec7d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc6d2377f0b60ade server_cookie=d9a3806b93fd914e in_seq=0 out_seq=0 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.686+0000 7f5e8cc00640 10 client.25361 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.687+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1192711820 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ac7d746790 con 0x55ac7d7eb360 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.687+0000 7f5e8cc00640 10 client.25361 handle_command_reply: tid=0 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.687+0000 7f5eac478740 1 client.25361 shutdown 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.687+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 msgr2=0x55ac7d7d73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.687+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7d73c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55ac7d7cdec0 tx=0x7f5e900cafa0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 shutdown_connections 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d7eb360 0x55ac7d7ec7d0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7d7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7d73c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5e8cc00640 0 client.25361 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 --2- 172.21.15.138:0/1192711820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 >> 172.21.15.138:0/1192711820 conn(0x55ac7d7cbd40 msgr2=0x55ac7d7d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 shutdown_connections 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 1 -- 172.21.15.138:0/1192711820 wait complete. 2024-08-20T01:08:24.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.688+0000 7f5eac478740 20 client.25361 trim_cache size 0 max 16384 2024-08-20T01:08:24.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.706+0000 7f5eac478740 1 Processor -- start 2024-08-20T01:08:24.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.706+0000 7f5eac478740 1 -- start start 2024-08-20T01:08:24.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7eab10 con 0x55ac7d7cdf10 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7eb1e0 con 0x55ac7d7ce830 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5eac478740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7eb8b0 con 0x55ac7d7da7c0 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57626/0 (socket says 172.21.15.138:57626) 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 -- 172.21.15.138:0/2810820906 learned_addr learned my addr 172.21.15.138:0/2810820906 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea3400640 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 -- 172.21.15.138:0/2810820906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 msgr2=0x55ac7d7f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 -- 172.21.15.138:0/2810820906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 msgr2=0x55ac7d7f4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2a00640 1 -- 172.21.15.138:0/2810820906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac7d7dcef0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.707+0000 7f5ea2000640 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:24.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ac7d7dd4a0 tx=0x7f5e9401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f220ba0e3caefe31 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5e8f400640 1 -- 172.21.15.138:0/2810820906 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9400fdd0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5e8f400640 1 -- 172.21.15.138:0/2810820906 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9401fdd0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5e8f400640 1 -- 172.21.15.138:0/2810820906 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e94005650 con 0x55ac7d7cdf10 2024-08-20T01:08:24.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5eac478740 1 -- 172.21.15.138:0/2810820906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 msgr2=0x55ac7d7d7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.708+0000 7f5eac478740 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55ac7d7dd4a0 tx=0x7f5e9401a000 comp rx=0 tx=0).stop 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 -- 172.21.15.138:0/2810820906 shutdown_connections 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7da7c0 0x55ac7d7f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 --2- 172.21.15.138:0/2810820906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7d7880 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 -- 172.21.15.138:0/2810820906 >> 172.21.15.138:0/2810820906 conn(0x55ac7d7cbd40 msgr2=0x55ac7d7d7170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 -- 172.21.15.138:0/2810820906 shutdown_connections 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 -- 172.21.15.138:0/2810820906 wait complete. 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 -- start start 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.709+0000 7f5eac478740 1 Processor -- start 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d853a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7ec650 con 0x55ac7d7cdf10 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7ebf80 con 0x55ac7d7da7c0 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ac7d7ecd20 con 0x55ac7d7ce830 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5eac478740 10 client.0 _ll_register_callbacks cb 0x7f5e90005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57636/0 (socket says 172.21.15.138:57636) 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 -- 172.21.15.138:0/1668666245 learned_addr learned my addr 172.21.15.138:0/1668666245 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea3400640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d853a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 -- 172.21.15.138:0/1668666245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 msgr2=0x55ac7d7f0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 -- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 msgr2=0x55ac7d853a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d853a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea2a00640 1 -- 172.21.15.138:0/1668666245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac7d8541a0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.710+0000 7f5ea3400640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d853a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5ea2a00640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e94012e80 tx=0x7f5e9400f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98287b70b2e2b4e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5e8cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9401f8a0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e9401fdd0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5e9401fa40 con 0x55ac7d7cdf10 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5e94046050 con 0x55ac7d7cdf10 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5eac478740 10 client.15339 fetch_fsmap learned FSMap version 24 2024-08-20T01:08:24.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.711+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ac7d8554a0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e9401fdd0 con 0x55ac7d7cdf10 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 24) ==== 8354+0+0 (secure 0 0 0) 0x7f5e94003a00 con 0x55ac7d7cdf10 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 10 client.15339 fetch_fsmap finished waiting for FSMap version 24 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 10 client.15339 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 20 client.15339 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d859e50 0x55ac7d858b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 4 client.15339 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "status"}) -- 0x55ac7d742230 con 0x55ac7d859e50 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.712+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d859e50 0x55ac7d858b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:24.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5ea2000640 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d859e50 0x55ac7d858b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd50f237255b39ed server_cookie=fac1e350b361fbc4 in_seq=0 out_seq=0 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5e8cc00640 10 client.15339 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5e8cc00640 1 -- 172.21.15.138:0/1668666245 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55ac7d742230 con 0x55ac7d859e50 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5e8cc00640 10 client.15339 handle_command_reply: tid=0 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 client.15339 shutdown 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 msgr2=0x55ac7d7f01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5e94012e80 tx=0x7f5e9400f990 comp rx=0 tx=0).stop 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 shutdown_connections 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ac7d859e50 0x55ac7d858b60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ac7d7da7c0 0x55ac7d853a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ac7d7ce830 0x55ac7d7f0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 --2- 172.21.15.138:0/1668666245 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ac7d7cdf10 0x55ac7d7f01a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5e8cc00640 0 client.15339 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:24.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 >> 172.21.15.138:0/1668666245 conn(0x55ac7d7cbd40 msgr2=0x55ac7d7d7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 shutdown_connections 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 1 -- 172.21.15.138:0/1668666245 wait complete. 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.713+0000 7f5eac478740 20 client.15339 trim_cache size 0 max 16384 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24743, 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 176.107651702, 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 24, 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 86, 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 190.50354252 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 msgr2=0x7f5e880b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 0x7f5e880b60e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5e94009fd0 tx=0x7f5e94005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:24.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 msgr2=0x7f5ea419fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5e90005ab0 tx=0x7f5e90003c10 comp rx=0 tx=0).stop 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 shutdown_connections 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e880b3c20 0x7f5e880b60e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ea410f470 0x7f5ea41a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ea410eaf0 0x7f5ea419fb70 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 --2- 172.21.15.138:0/585967718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ea41090e0 0x7f5ea419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 >> 172.21.15.138:0/585967718 conn(0x7f5ea4078760 msgr2=0x7f5ea407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 shutdown_connections 2024-08-20T01:08:24.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:24.714+0000 7f5ea9a00640 1 -- 172.21.15.138:0/585967718 wait complete. 2024-08-20T01:08:25.806 INFO:teuthology.run_tasks:Running task sequential... 2024-08-20T01:08:25.816 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-08-20T01:08:25.816 INFO:teuthology.task.sequential:In sequential, running task print... 2024-08-20T01:08:25.816 INFO:teuthology.task.print:Enabling mgr modules 2024-08-20T01:08:25.817 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-08-20T01:08:25.817 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T01:08:25.817 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:08:25.818 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-08-20T01:08:25.866 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:25 smithi138 ceph-mon[18076]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:25 smithi167 ceph-mon[21965]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:25.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 -- 172.21.15.138:0/1912701499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac41083f0 msgr2=0x7f5ac4108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 --2- 172.21.15.138:0/1912701499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac41083f0 0x7f5ac4108830 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009900 tx=0x7f5ab002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 -- 172.21.15.138:0/1912701499 shutdown_connections 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 --2- 172.21.15.138:0/1912701499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac4108d70 0x7f5ac410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 --2- 172.21.15.138:0/1912701499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac41083f0 0x7f5ac4108830 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 --2- 172.21.15.138:0/1912701499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 0x7f5ac4102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 -- 172.21.15.138:0/1912701499 >> 172.21.15.138:0/1912701499 conn(0x7f5ac40fe430 msgr2=0x7f5ac4100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 -- 172.21.15.138:0/1912701499 shutdown_connections 2024-08-20T01:08:25.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.973+0000 7f5acc200640 1 -- 172.21.15.138:0/1912701499 wait complete. 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 Processor -- start 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 -- start start 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 0x7f5ac419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac41083f0 0x7f5ac419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112a40 con 0x7f5ac4108d70 2024-08-20T01:08:25.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac41128c0 con 0x7f5ac41083f0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.974+0000 7f5acc200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac4112bc0 con 0x7f5ac41029f0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57648/0 (socket says 172.21.15.138:57648) 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 -- 172.21.15.138:0/1940482081 learned_addr learned my addr 172.21.15.138:0/1940482081 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5ac9800640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 0x7f5ac419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5ac8e00640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac41083f0 0x7f5ac419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 -- 172.21.15.138:0/1940482081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 msgr2=0x7f5ac419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 0x7f5ac419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 -- 172.21.15.138:0/1940482081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac41083f0 msgr2=0x7f5ac419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac41083f0 0x7f5ac419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 -- 172.21.15.138:0/1940482081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac41a1220 con 0x7f5ac4108d70 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.975+0000 7f5aca200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00eb10 tx=0x7f5abc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.976+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc004b20 con 0x7f5ac4108d70 2024-08-20T01:08:25.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.976+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac41a1510 con 0x7f5ac4108d70 2024-08-20T01:08:25.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.976+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5abc004cc0 con 0x7f5ac4108d70 2024-08-20T01:08:25.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.976+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5abc010670 con 0x7f5ac4108d70 2024-08-20T01:08:25.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.976+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac41a7bd0 con 0x7f5ac4108d70 2024-08-20T01:08:25.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.977+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac4102dd0 con 0x7f5ac4108d70 2024-08-20T01:08:25.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.979+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 22) ==== 151077+0+0 (secure 0 0 0) 0x7f5abc005020 con 0x7f5ac4108d70 2024-08-20T01:08:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.980+0000 7f5ac2000640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 0x7f5aac0b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:25.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.980+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5abc0e4640 con 0x7f5ac4108d70 2024-08-20T01:08:25.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.980+0000 7f5ac9800640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 0x7f5aac0b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:25.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.981+0000 7f5ac9800640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 0x7f5aac0b6160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009bd0 tx=0x7f5ab8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:25.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:25.984+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5abc0ab7c0 con 0x7f5ac4108d70 2024-08-20T01:08:25.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:25 smithi142 ceph-mon[21799]: pgmap v371: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:26.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.213+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f5ac402e340 con 0x7f5ac4108d70 2024-08-20T01:08:26.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.621+0000 7f5ac2000640 1 -- 172.21.15.138:0/1940482081 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 module 'snap_schedule' is already enabled v23) ==== 133+0+0 (secure 0 0 0) 0x7f5abc0ab0c0 con 0x7f5ac4108d70 2024-08-20T01:08:26.619 INFO:teuthology.orchestra.run.smithi138.stderr:module 'snap_schedule' is already enabled 2024-08-20T01:08:26.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.623+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 msgr2=0x7f5aac0b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.623+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 0x7f5aac0b6160 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8009bd0 tx=0x7f5ab8005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.623+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 msgr2=0x7f5ac41a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.623+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00eb10 tx=0x7f5abc00c2d0 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 shutdown_connections 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5aac0b3ca0 0x7f5aac0b6160 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ac4108d70 0x7f5ac41a0a50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ac41083f0 0x7f5ac419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 --2- 172.21.15.138:0/1940482081 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ac41029f0 0x7f5ac419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:26.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 >> 172.21.15.138:0/1940482081 conn(0x7f5ac40fe430 msgr2=0x7f5ac41059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:26.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 shutdown_connections 2024-08-20T01:08:26.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:26.624+0000 7f5acc200640 1 -- 172.21.15.138:0/1940482081 wait complete. 2024-08-20T01:08:26.643 INFO:teuthology.run_tasks:Running task kclient... 2024-08-20T01:08:26.656 INFO:tasks.kclient:Mounting kernel clients... 2024-08-20T01:08:26.657 INFO:tasks.kclient:config is {'client.0': {}} 2024-08-20T01:08:26.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:26.657 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-08-20T01:08:26.692 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-08-20T01:08:26.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:26.693 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-08-20T01:08:26.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:26.791 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-08-20T01:08:26.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:26 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T01:08:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:26 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T01:08:27.011 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:26 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-08-20T01:08:27.094 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-08-20T01:08:27.094 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-08-20T01:08:27.094 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:27.095 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs ls 2024-08-20T01:08:27.464 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:27.956 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:27 smithi138 ceph-mon[18076]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:27.956 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:27 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T01:08:27.956 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:27 smithi138 ceph-mon[18076]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-20T01:08:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:27 smithi167 ceph-mon[21965]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:27 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T01:08:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:27 smithi167 ceph-mon[21965]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-20T01:08:27.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:27 smithi142 ceph-mon[21799]: pgmap v372: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:27 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1940482081' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-08-20T01:08:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:27 smithi142 ceph-mon[21799]: mgrmap e23: x(active, since 9m), standbys: z, y 2024-08-20T01:08:28.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 -- 172.21.15.138:0/2067278340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 msgr2=0x7f38a010f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 --2- 172.21.15.138:0/2067278340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a010f490 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3898009ca0 tx=0x7f389802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 -- 172.21.15.138:0/2067278340 shutdown_connections 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 --2- 172.21.15.138:0/2067278340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a010f490 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 --2- 172.21.15.138:0/2067278340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0108310 0x7f38a0108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 --2- 172.21.15.138:0/2067278340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0102900 0x7f38a0102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 -- 172.21.15.138:0/2067278340 >> 172.21.15.138:0/2067278340 conn(0x7f38a00fe390 msgr2=0x7f38a01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 -- 172.21.15.138:0/2067278340 shutdown_connections 2024-08-20T01:08:28.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.068+0000 7f38a4a00640 1 -- 172.21.15.138:0/2067278340 wait complete. 2024-08-20T01:08:28.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 Processor -- start 2024-08-20T01:08:28.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 -- start start 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0108310 0x7f38a0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f38a01129a0 con 0x7f38a0108c90 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0112820 con 0x7f38a0108310 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.069+0000 7f38a4a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0112b20 con 0x7f38a0102900 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:53630/0 (socket says 172.21.15.138:53630) 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 -- 172.21.15.138:0/1779047647 learned_addr learned my addr 172.21.15.138:0/1779047647 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389de00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0108310 0x7f38a0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 -- 172.21.15.138:0/1779047647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0108310 msgr2=0x7f38a0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0108310 0x7f38a0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 -- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 msgr2=0x7f38a0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389e800640 1 -- 172.21.15.138:0/1779047647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a01a70f0 con 0x7f38a0102900 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.070+0000 7f389f200640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f389e800640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f389000e9f0 tx=0x7f389000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389000cd80 con 0x7f38a0102900 2024-08-20T01:08:28.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a01a73e0 con 0x7f38a0102900 2024-08-20T01:08:28.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38900108e0 con 0x7f38a0102900 2024-08-20T01:08:28.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3890018980 con 0x7f38a0102900 2024-08-20T01:08:28.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.071+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01a7920 con 0x7f38a0102900 2024-08-20T01:08:28.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.072+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a0102ce0 con 0x7f38a0102900 2024-08-20T01:08:28.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.074+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f3890010430 con 0x7f38a0102900 2024-08-20T01:08:28.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.076+0000 7f3897400640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 0x7f38840b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:28.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.076+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f38900e47b0 con 0x7f38a0102900 2024-08-20T01:08:28.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.076+0000 7f389de00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 0x7f38840b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:28.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.076+0000 7f389de00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 0x7f38840b6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f388c0095f0 tx=0x7f388c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:28.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.078+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f38900ab930 con 0x7f38a0102900 2024-08-20T01:08:28.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.302+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f38a01a7c50 con 0x7f38a0102900 2024-08-20T01:08:28.301 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.303+0000 7f3897400640 1 -- 172.21.15.138:0/1779047647 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+88 (secure 0 0 0) 0x7f38900ab230 con 0x7f38a0102900 2024-08-20T01:08:28.301 INFO:teuthology.orchestra.run.smithi138.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 msgr2=0x7f38840b6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 0x7f38840b6180 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f388c0095f0 tx=0x7f388c005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 msgr2=0x7f38a00751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f389000e9f0 tx=0x7f389000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 shutdown_connections 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f38840b3cc0 0x7f38840b6180 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38a0108c90 0x7f38a0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38a0108310 0x7f38a0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 --2- 172.21.15.138:0/1779047647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f38a0102900 0x7f38a00751d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:28.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 >> 172.21.15.138:0/1779047647 conn(0x7f38a00fe390 msgr2=0x7f38a0104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:28.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.305+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 shutdown_connections 2024-08-20T01:08:28.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:28.306+0000 7f38a4a00640 1 -- 172.21.15.138:0/1779047647 wait complete. 2024-08-20T01:08:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:28 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1779047647' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:08:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:28 smithi167 ceph-mon[21965]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:28 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1779047647' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:08:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:28 smithi142 ceph-mon[21799]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:29.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:28 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1779047647' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-08-20T01:08:29.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:28 smithi138 ceph-mon[18076]: pgmap v373: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-08-20T01:08:29.261 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-08-20T01:08:29.262 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-08-20T01:08:29.262 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-08-20T01:08:29.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:29.262 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:08:29.299 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42119sec preferred_lft 42119sec 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::ec4:7aff:fe88:6f71/64 scope link 2024-08-20T01:08:29.300 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-08-20T01:08:29.301 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-08-20T01:08:29.301 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:08:29.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:29.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:29.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:29.525 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-08-20T01:08:29.582 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-08-20T01:08:29.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:29.584 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-08-20T01:08:29.620 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-08-20T01:08:29.620 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-08-20T01:08:29.620 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-08-20T01:08:29.620 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-08-20T01:08:29.620 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-08-20T01:08:29.620 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:08:29.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:29.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:29.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:29.819 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-08-20T01:08:29.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:29.855 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-08-20T01:08:29.930 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:08:29.930 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:08:29.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T01:08:29.930 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-08-20T01:08:29.930 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:08:30.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:30.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:30.109 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-08-20T01:08:30.109 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:08:30.109 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:08:30.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-08-20T01:08:30.109 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-08-20T01:08:30.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-08-20T01:08:30.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-08-20T01:08:30.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-08-20T01:08:30.110 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:08:30.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:30.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:30.328 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-08-20T01:08:30.328 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-08-20T01:08:30.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-08-20T01:08:30.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-08-20T01:08:30.328 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-08-20T01:08:30.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:30.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:30.495 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T01:08:30.495 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:30.532 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-08-20T01:08:30.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-08-20T01:08:30.532 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:30.607 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-08-20T01:08:30.608 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-08-20T01:08:30.608 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/61a922e4-5ca6-4b73-ad1d-5bdb03bd564e 2024-08-20T01:08:30.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:30.608 DEBUG:teuthology.orchestra.run.smithi138:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/61a922e4-5ca6-4b73-ad1d-5bdb03bd564e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-08-20T01:08:30.738 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-08-20T01:08:30.738 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-08-20T01:08:30.738 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync". 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:invalid new device string format 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: resolved to: "172.21.15.138:3300,172.21.15.142:3300,172.21.15.167:3300" 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.138:3300,172.21.15.142:3300,172.21.15.167:3300:/volumes/qa/sv_1/61a922e4-5ca6-4b73-ad1d-5bdb03bd564e 2024-08-20T01:08:31.446 INFO:teuthology.orchestra.run.smithi138.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,nowsync,key=0,fsid=d876cd00-5e8e-11ef-bccf-c7b262605968" will pass to kernel 2024-08-20T01:08:31.449 INFO:teuthology.orchestra.run.smithi138.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-08-20T01:08:31.449 INFO:teuthology.orchestra.run.smithi138.stdout: You just mounted a file system that supports labels which does not 2024-08-20T01:08:31.449 INFO:teuthology.orchestra.run.smithi138.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-08-20T01:08:31.449 INFO:teuthology.orchestra.run.smithi138.stdout: applications will generate AVC messages and not be allowed access to 2024-08-20T01:08:31.449 INFO:teuthology.orchestra.run.smithi138.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-08-20T01:08:31.452 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-08-20T01:08:31.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:08:31.453 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:31.514 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T01:08:31.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T01:08:31.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:31.607 INFO:teuthology.orchestra.run.smithi138.stdout:client25382 2024-08-20T01:08:31.607 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:08:31.607 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T01:08:31.644 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T01:08:31.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T01:08:31.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:31.774 INFO:teuthology.orchestra.run.smithi138.stdout:client25382 2024-08-20T01:08:31.774 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-08-20T01:08:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:31 smithi138 ceph-mon[18076]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:31.852 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-08-20T01:08:31.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:31.863 INFO:teuthology.orchestra.run.smithi138.stdout:client25382 2024-08-20T01:08:31.864 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968.client25382/status') 2024-08-20T01:08:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:31 smithi167 ceph-mon[21965]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:31.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-08-20T01:08:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:31 smithi142 ceph-mon[21799]: pgmap v374: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-08-20T01:08:32.007 INFO:teuthology.orchestra.run.smithi138.stdout:instance: client.25382 (3)172.21.15.138:0/3056806185 2024-08-20T01:08:32.007 INFO:teuthology.orchestra.run.smithi138.stdout:blocklisted: false 2024-08-20T01:08:32.007 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records in 2024-08-20T01:08:32.007 INFO:teuthology.orchestra.run.smithi138.stderr:0+1 records out 2024-08-20T01:08:32.007 INFO:teuthology.orchestra.run.smithi138.stderr:72 bytes copied, 5.8521e-05 s, 1.2 MB/s 2024-08-20T01:08:32.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-08-20T01:08:32.017 INFO:teuthology.run_tasks:Running task exec... 2024-08-20T01:08:32.028 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T01:08:32.028 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:08:32.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-08-20T01:08:32.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 -- 172.21.15.138:0/3456641330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac10eff0 msgr2=0x7f61ac1113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 --2- 172.21.15.138:0/3456641330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac10eff0 0x7f61ac1113e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009ca0 tx=0x7f61a402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 -- 172.21.15.138:0/3456641330 shutdown_connections 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 --2- 172.21.15.138:0/3456641330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac10eff0 0x7f61ac1113e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 --2- 172.21.15.138:0/3456641330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 0x7f61ac10eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 --2- 172.21.15.138:0/3456641330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac0751f0 0x7f61ac0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 -- 172.21.15.138:0/3456641330 >> 172.21.15.138:0/3456641330 conn(0x7f61ac0fe430 msgr2=0x7f61ac100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 -- 172.21.15.138:0/3456641330 shutdown_connections 2024-08-20T01:08:32.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.182+0000 7f61b2600640 1 -- 172.21.15.138:0/3456641330 wait complete. 2024-08-20T01:08:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.183+0000 7f61b2600640 1 Processor -- start 2024-08-20T01:08:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.183+0000 7f61b2600640 1 -- start start 2024-08-20T01:08:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 0x7f61ac19b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 0x7f61ac1a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114c20 con 0x7f61ac0751f0 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114aa0 con 0x7f61ac075b10 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61b2600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac114da0 con 0x7f61ac10eff0 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43824/0 (socket says 172.21.15.138:43824) 2024-08-20T01:08:32.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 -- 172.21.15.138:0/1289568673 learned_addr learned my addr 172.21.15.138:0/1289568673 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61a2a00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 0x7f61ac19b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61abe00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 0x7f61ac1a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 -- 172.21.15.138:0/1289568673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 msgr2=0x7f61ac1a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 0x7f61ac1a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 -- 172.21.15.138:0/1289568673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 msgr2=0x7f61ac19b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 0x7f61ac19b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 -- 172.21.15.138:0/1289568673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac1a1180 con 0x7f61ac0751f0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61abe00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 0x7f61ac1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.184+0000 7f61ab400640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.185+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cd80 con 0x7f61ac0751f0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.185+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f619c0108e0 con 0x7f61ac0751f0 2024-08-20T01:08:32.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.185+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac1a1470 con 0x7f61ac0751f0 2024-08-20T01:08:32.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.185+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c018920 con 0x7f61ac0751f0 2024-08-20T01:08:32.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.185+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1a7b30 con 0x7f61ac0751f0 2024-08-20T01:08:32.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.186+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac0755d0 con 0x7f61ac0751f0 2024-08-20T01:08:32.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.188+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f619c010430 con 0x7f61ac0751f0 2024-08-20T01:08:32.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.189+0000 7f61a8c00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 0x7f61900b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.190+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f619c0e44a0 con 0x7f61ac0751f0 2024-08-20T01:08:32.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.190+0000 7f61a2a00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 0x7f61900b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.190+0000 7f61a2a00640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 0x7f61900b60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61ac100c60 tx=0x7f6194005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:32.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.193+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f619c0ab620 con 0x7f61ac0751f0 2024-08-20T01:08:32.413 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.415+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f61ac1a7f80 con 0x7f61ac0751f0 2024-08-20T01:08:32.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.584+0000 7f61a8c00640 1 -- 172.21.15.138:0/1289568673 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f619c0aaf20 con 0x7f61ac0751f0 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.587+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 msgr2=0x7f61900b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.587+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 0x7f61900b60e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f61ac100c60 tx=0x7f6194005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.587+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 msgr2=0x7f61ac19b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.587+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 shutdown_connections 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f61900b3c20 0x7f61900b60e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f61ac10eff0 0x7f61ac1a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f61ac075b10 0x7f61ac19b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 --2- 172.21.15.138:0/1289568673 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61ac0751f0 0x7f61ac19b0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 >> 172.21.15.138:0/1289568673 conn(0x7f61ac0fe430 msgr2=0x7f61ac1111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:32.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 shutdown_connections 2024-08-20T01:08:32.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.588+0000 7f61b2600640 1 -- 172.21.15.138:0/1289568673 wait complete. 2024-08-20T01:08:32.602 INFO:teuthology.run_tasks:Running task check-counter... 2024-08-20T01:08:32.613 INFO:teuthology.run_tasks:Running task exec... 2024-08-20T01:08:32.644 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T01:08:32.644 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:08:32.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-08-20T01:08:32.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.799+0000 7ff05a000640 1 -- 172.21.15.138:0/2761165160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff054104a10 msgr2=0x7ff054104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.799+0000 7ff05a000640 1 --2- 172.21.15.138:0/2761165160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff054104a10 0x7ff054104df0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff044009900 tx=0x7ff04402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 -- 172.21.15.138:0/2761165160 shutdown_connections 2024-08-20T01:08:32.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 --2- 172.21.15.138:0/2761165160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410ad90 0x7ff054111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 --2- 172.21.15.138:0/2761165160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff05410a410 0x7ff05410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 --2- 172.21.15.138:0/2761165160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff054104a10 0x7ff054104df0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 -- 172.21.15.138:0/2761165160 >> 172.21.15.138:0/2761165160 conn(0x7ff054100450 msgr2=0x7ff054102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 -- 172.21.15.138:0/2761165160 shutdown_connections 2024-08-20T01:08:32.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 -- 172.21.15.138:0/2761165160 wait complete. 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.800+0000 7ff05a000640 1 Processor -- start 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 -- start start 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff054104a10 0x7ff05419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff054114aa0 con 0x7ff05410ad90 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff054114920 con 0x7ff05410a410 2024-08-20T01:08:32.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff05a000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff054114c20 con 0x7ff054104a10 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff059600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff04be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff058c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff054104a10 0x7ff05419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff04be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:44956/0 (socket says 172.21.15.138:44956) 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff059600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43838/0 (socket says 172.21.15.138:43838) 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.801+0000 7ff04be00640 1 -- 172.21.15.138:0/1575455212 learned_addr learned my addr 172.21.15.138:0/1575455212 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 -- 172.21.15.138:0/1575455212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff054104a10 msgr2=0x7ff05419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff054104a10 0x7ff05419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 -- 172.21.15.138:0/1575455212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 msgr2=0x7ff05419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 -- 172.21.15.138:0/1575455212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0541a1000 con 0x7ff05410ad90 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff04be00640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:32.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff059600640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009fd0 tx=0x7ff04c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00ce20 con 0x7ff05410ad90 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c005020 con 0x7ff05410ad90 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0541a12f0 con 0x7ff05410ad90 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c010690 con 0x7ff05410ad90 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.802+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0541a79b0 con 0x7ff05410ad90 2024-08-20T01:08:32.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.803+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff054104df0 con 0x7ff05410ad90 2024-08-20T01:08:32.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.807+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7ff04c0054d0 con 0x7ff05410ad90 2024-08-20T01:08:32.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.808+0000 7ff052000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 0x7ff0380b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.809+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff04c0e4700 con 0x7ff05410ad90 2024-08-20T01:08:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.809+0000 7ff058c00640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 0x7ff0380b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:32.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.809+0000 7ff058c00640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 0x7ff0380b6160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff044005ab0 tx=0x7ff044005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:32.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:32.810+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff04c0ab8e0 con 0x7ff05410ad90 2024-08-20T01:08:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:32 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-20T01:08:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:32 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-20T01:08:33.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.016+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ff0540632c0 con 0x7ff0380b3ca0 2024-08-20T01:08:33.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:32 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-08-20T01:08:33.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.181+0000 7ff052000640 1 -- 172.21.15.138:0/1575455212 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff0540632c0 con 0x7ff0380b3ca0 2024-08-20T01:08:33.180 INFO:teuthology.orchestra.run.smithi138.stdout:Schedule set for path / 2024-08-20T01:08:33.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 msgr2=0x7ff0380b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 0x7ff0380b6160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff044005ab0 tx=0x7ff044005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 msgr2=0x7ff0541a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009fd0 tx=0x7ff04c00ef10 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 shutdown_connections 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0380b3ca0 0x7ff0380b6160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff05410ad90 0x7ff0541a0830 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff05410a410 0x7ff05419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 --2- 172.21.15.138:0/1575455212 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff054104a10 0x7ff05419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 >> 172.21.15.138:0/1575455212 conn(0x7ff054100450 msgr2=0x7ff054102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 shutdown_connections 2024-08-20T01:08:33.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.183+0000 7ff05a000640 1 -- 172.21.15.138:0/1575455212 wait complete. 2024-08-20T01:08:33.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 -- 172.21.15.138:0/2857051596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 msgr2=0x7f5e58111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2857051596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e58111590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009ca0 tx=0x7f5e5002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 -- 172.21.15.138:0/2857051596 shutdown_connections 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2857051596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e58111590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2857051596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e5810a410 0x7f5e5810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2857051596 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e58104a10 0x7f5e58104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 -- 172.21.15.138:0/2857051596 >> 172.21.15.138:0/2857051596 conn(0x7f5e58100450 msgr2=0x7f5e58102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.351+0000 7f5e5f000640 1 -- 172.21.15.138:0/2857051596 shutdown_connections 2024-08-20T01:08:33.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.352+0000 7f5e5f000640 1 -- 172.21.15.138:0/2857051596 wait complete. 2024-08-20T01:08:33.352 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.352+0000 7f5e5f000640 1 Processor -- start 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 -- start start 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e58104a10 0x7f5e5819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 0x7f5e5819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58114aa0 con 0x7f5e5810ad90 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58114920 con 0x7f5e58104a10 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5e58114c20 con 0x7f5e5810a410 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e58104a10 0x7f5e5819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5e600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43874/0 (socket says 172.21.15.138:43874) 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5e600640 1 -- 172.21.15.138:0/2983572214 learned_addr learned my addr 172.21.15.138:0/2983572214 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:33.353 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.353+0000 7f5e5d200640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 0x7f5e5819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 -- 172.21.15.138:0/2983572214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 msgr2=0x7f5e5819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 0x7f5e5819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 -- 172.21.15.138:0/2983572214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e58104a10 msgr2=0x7f5e5819aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e58104a10 0x7f5e5819aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 -- 172.21.15.138:0/2983572214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e581a0f60 con 0x7f5e5810ad90 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5d200640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 0x7f5e5819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5e600640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009430 tx=0x7f5e500317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50003b70 con 0x7f5e5810ad90 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e50003d10 con 0x7f5e5810ad90 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5002fa30 con 0x7f5e5810ad90 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e581a11f0 con 0x7f5e5810ad90 2024-08-20T01:08:33.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.354+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e581a7910 con 0x7f5e5810ad90 2024-08-20T01:08:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.355+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e58104df0 con 0x7f5e5810ad90 2024-08-20T01:08:33.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.357+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7f5e5002fbd0 con 0x7f5e5810ad90 2024-08-20T01:08:33.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.359+0000 7f5e56000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 0x7f5e400b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.360+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5e50105460 con 0x7f5e5810ad90 2024-08-20T01:08:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.360+0000 7f5e5dc00640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 0x7f5e400b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.360+0000 7f5e5dc00640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 0x7f5e400b6160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009bd0 tx=0x7f5e4c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:33.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.363+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5e500cc6e0 con 0x7f5e5810ad90 2024-08-20T01:08:33.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.567+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f5e581079d0 con 0x7f5e400b3ca0 2024-08-20T01:08:33.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.639+0000 7f5e56000640 1 -- 172.21.15.138:0/2983572214 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f5e581079d0 con 0x7f5e400b3ca0 2024-08-20T01:08:33.637 INFO:teuthology.orchestra.run.smithi138.stdout:Retention added to path / 2024-08-20T01:08:33.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 msgr2=0x7f5e400b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 0x7f5e400b6160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009bd0 tx=0x7f5e4c005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 msgr2=0x7f5e581a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009430 tx=0x7f5e500317c0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 shutdown_connections 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5e400b3ca0 0x7f5e400b6160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e5810ad90 0x7f5e581a0790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5e5810a410 0x7f5e5819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 --2- 172.21.15.138:0/2983572214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e58104a10 0x7f5e5819aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.641+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 >> 172.21.15.138:0/2983572214 conn(0x7f5e58100450 msgr2=0x7f5e58102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.642+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 shutdown_connections 2024-08-20T01:08:33.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.642+0000 7f5e5f000640 1 -- 172.21.15.138:0/2983572214 wait complete. 2024-08-20T01:08:33.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-20T01:08:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 -- 172.21.15.138:0/1131303719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 msgr2=0x7feb70104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 --2- 172.21.15.138:0/1131303719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb70104ff0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feb60009900 tx=0x7feb6002a410 comp rx=0 tx=0).stop 2024-08-20T01:08:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 -- 172.21.15.138:0/1131303719 shutdown_connections 2024-08-20T01:08:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 --2- 172.21.15.138:0/1131303719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 0x7feb70111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 --2- 172.21.15.138:0/1131303719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 0x7feb7010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 --2- 172.21.15.138:0/1131303719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb70104ff0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 -- 172.21.15.138:0/1131303719 >> 172.21.15.138:0/1131303719 conn(0x7feb70100610 msgr2=0x7feb70102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 -- 172.21.15.138:0/1131303719 shutdown_connections 2024-08-20T01:08:33.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.810+0000 7feb76c00640 1 -- 172.21.15.138:0/1131303719 wait complete. 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 Processor -- start 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 -- start start 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 0x7feb7019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 0x7feb701a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7feb70114c70 con 0x7feb70104c10 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7feb70114af0 con 0x7feb7010af90 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.811+0000 7feb76c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7feb70114df0 con 0x7feb7010a610 2024-08-20T01:08:33.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43886/0 (socket says 172.21.15.138:43886) 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 -- 172.21.15.138:0/2780962191 learned_addr learned my addr 172.21.15.138:0/2780962191 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb67400640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 0x7feb7019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb74c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 0x7feb701a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 -- 172.21.15.138:0/2780962191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 msgr2=0x7feb7019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 0x7feb7019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 -- 172.21.15.138:0/2780962191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 msgr2=0x7feb701a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 0x7feb701a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 -- 172.21.15.138:0/2780962191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701a11f0 con 0x7feb70104c10 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.812+0000 7feb6fe00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb60005ab0 tx=0x7feb60003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.813+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6003f070 con 0x7feb70104c10 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.813+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb701a1480 con 0x7feb70104c10 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.813+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701a7ba0 con 0x7feb70104c10 2024-08-20T01:08:33.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.813+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb60031d60 con 0x7feb70104c10 2024-08-20T01:08:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.813+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60049060 con 0x7feb70104c10 2024-08-20T01:08:33.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.814+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb70104ff0 con 0x7feb70104c10 2024-08-20T01:08:33.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.816+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7feb60042070 con 0x7feb70104c10 2024-08-20T01:08:33.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.817+0000 7feb6d600640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 0x7feb540b6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.817+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7feb601044b0 con 0x7feb70104c10 2024-08-20T01:08:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.818+0000 7feb67400640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 0x7feb540b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:33.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.818+0000 7feb67400640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 0x7feb540b6160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb70103f40 tx=0x7feb5800b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:33.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:33.821+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7feb600cb040 con 0x7feb70104c10 2024-08-20T01:08:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:33 smithi167 ceph-mon[21965]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:33 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-20T01:08:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:33 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:33 smithi142 ceph-mon[21799]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:33 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-20T01:08:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:33 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:34.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.024+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7feb700632c0 con 0x7feb540b3ca0 2024-08-20T01:08:34.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.032+0000 7feb6d600640 1 -- 172.21.15.138:0/2780962191 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7feb700632c0 con 0x7feb540b3ca0 2024-08-20T01:08:34.030 INFO:teuthology.orchestra.run.smithi138.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-20T00:00:00", "created": "2024-08-20T01:08:33", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.034+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 msgr2=0x7feb540b6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.034+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 0x7feb540b6160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb70103f40 tx=0x7feb5800b390 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 msgr2=0x7feb7019b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7feb60005ab0 tx=0x7feb60003c80 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 shutdown_connections 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7feb540b3ca0 0x7feb540b6160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feb7010af90 0x7feb701a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7feb7010a610 0x7feb7019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 --2- 172.21.15.138:0/2780962191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb70104c10 0x7feb7019b150 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 >> 172.21.15.138:0/2780962191 conn(0x7feb70100610 msgr2=0x7feb70102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 shutdown_connections 2024-08-20T01:08:34.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.035+0000 7feb76c00640 1 -- 172.21.15.138:0/2780962191 wait complete. 2024-08-20T01:08:34.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:33 smithi138 ceph-mon[18076]: pgmap v375: 129 pgs: 129 active+clean; 644 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:08:34.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:33 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1289568673' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-08-20T01:08:34.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:33 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:34.050 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-20T01:08:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 -- 172.21.15.138:0/1859167194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5840751f0 msgr2=0x7fd5840755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 --2- 172.21.15.138:0/1859167194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5840751f0 0x7fd5840755d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd574009900 tx=0x7fd57402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 -- 172.21.15.138:0/1859167194 shutdown_connections 2024-08-20T01:08:34.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 --2- 172.21.15.138:0/1859167194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd58410eff0 0x7fd5841113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 --2- 172.21.15.138:0/1859167194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd584075b10 0x7fd58410eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 --2- 172.21.15.138:0/1859167194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd5840751f0 0x7fd5840755d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 -- 172.21.15.138:0/1859167194 >> 172.21.15.138:0/1859167194 conn(0x7fd5840fe430 msgr2=0x7fd584100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:34.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 -- 172.21.15.138:0/1859167194 shutdown_connections 2024-08-20T01:08:34.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.205+0000 7fd589e00640 1 -- 172.21.15.138:0/1859167194 wait complete. 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 Processor -- start 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 -- start start 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5840751f0 0x7fd58419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584075b10 0x7fd58419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114bf0 con 0x7fd58410eff0 2024-08-20T01:08:34.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114a70 con 0x7fd5840751f0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.206+0000 7fd589e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd584114d70 con 0x7fd584075b10 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43898/0 (socket says 172.21.15.138:43898) 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 -- 172.21.15.138:0/269713643 learned_addr learned my addr 172.21.15.138:0/269713643 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd582a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5840751f0 0x7fd58419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd57be00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584075b10 0x7fd58419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 -- 172.21.15.138:0/269713643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584075b10 msgr2=0x7fd58419b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584075b10 0x7fd58419b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 -- 172.21.15.138:0/269713643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5840751f0 msgr2=0x7fd58419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5840751f0 0x7fd58419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 -- 172.21.15.138:0/269713643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5841a11f0 con 0x7fd58410eff0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.207+0000 7fd583400640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00dac0 tx=0x7fd57c00df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.208+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c017070 con 0x7fd58410eff0 2024-08-20T01:08:34.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.208+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5841a14e0 con 0x7fd58410eff0 2024-08-20T01:08:34.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.208+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57c00be30 con 0x7fd58410eff0 2024-08-20T01:08:34.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.208+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57c00f820 con 0x7fd58410eff0 2024-08-20T01:08:34.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.208+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5841a7ba0 con 0x7fd58410eff0 2024-08-20T01:08:34.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.209+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5840755d0 con 0x7fd58410eff0 2024-08-20T01:08:34.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.212+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 23) ==== 151077+0+0 (secure 0 0 0) 0x7fd57c010ac0 con 0x7fd58410eff0 2024-08-20T01:08:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.214+0000 7fd588a00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 0x7fd5680b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.214+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd57c0e3e40 con 0x7fd58410eff0 2024-08-20T01:08:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.214+0000 7fd582a00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 0x7fd5680b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.215+0000 7fd582a00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 0x7fd5680b60e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd574005ab0 tx=0x7fd574005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.215+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd57c0aa9d0 con 0x7fd58410eff0 2024-08-20T01:08:34.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.424+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fd584100750 con 0x7fd5680b3c20 2024-08-20T01:08:34.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.431+0000 7fd588a00640 1 -- 172.21.15.138:0/269713643 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fd584100750 con 0x7fd5680b3c20 2024-08-20T01:08:34.429 INFO:teuthology.orchestra.run.smithi138.stdout:/ 1m 6m3h 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 msgr2=0x7fd5680b60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 0x7fd5680b60e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd574005ab0 tx=0x7fd574005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 msgr2=0x7fd5841a0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00dac0 tx=0x7fd57c00df80 comp rx=0 tx=0).stop 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 shutdown_connections 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd5680b3c20 0x7fd5680b60e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd58410eff0 0x7fd5841a0a20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd584075b10 0x7fd58419b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 --2- 172.21.15.138:0/269713643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5840751f0 0x7fd58419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 >> 172.21.15.138:0/269713643 conn(0x7fd5840fe430 msgr2=0x7fd5840ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 shutdown_connections 2024-08-20T01:08:34.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:34.434+0000 7fd589e00640 1 -- 172.21.15.138:0/269713643 wait complete. 2024-08-20T01:08:34.449 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-08-20T01:08:34.507 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-08-20T01:08:34.517 INFO:teuthology.run_tasks:Running task background_exec... 2024-08-20T01:08:34.597 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:08:34.597 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-08-20T01:08:34.601 INFO:teuthology.run_tasks:Running task quiescer... 2024-08-20T01:08:34.685 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-08-20T01:08:34.685 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:08:34.734 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:34 smithi138 ceph-mon[18076]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:34.734 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:34 smithi138 ceph-mon[18076]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:34 smithi167 ceph-mon[21965]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:34 smithi167 ceph-mon[21965]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:34 smithi142 ceph-mon[21799]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:34 smithi142 ceph-mon[21799]: from='client.15375 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.057 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:35.674 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:35 smithi138 ceph-mon[18076]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.675 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:35 smithi138 ceph-mon[18076]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.675 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:35 smithi138 ceph-mon[18076]: pgmap v376: 129 pgs: 129 active+clean; 1.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 101 KiB/s wr, 41 op/s 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.708+0000 7fb379c00640 1 -- 172.21.15.138:0/2438841100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 msgr2=0x7fb37410c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.708+0000 7fb379c00640 1 --2- 172.21.15.138:0/2438841100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb37410c620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb360009860 tx=0x7fb36002a340 comp rx=0 tx=0).stop 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 -- 172.21.15.138:0/2438841100 shutdown_connections 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 --2- 172.21.15.138:0/2438841100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb37410cc70 0x7fb37410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 --2- 172.21.15.138:0/2438841100 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb37410c620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 --2- 172.21.15.138:0/2438841100 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb374103810 0x7fb374103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 -- 172.21.15.138:0/2438841100 >> 172.21.15.138:0/2438841100 conn(0x7fb3740fd2d0 msgr2=0x7fb3740ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 -- 172.21.15.138:0/2438841100 shutdown_connections 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.709+0000 7fb379c00640 1 -- 172.21.15.138:0/2438841100 wait complete. 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 Processor -- start 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 -- start start 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb374101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb37410cc70 0x7fb374101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb3741128e0 con 0x7fb374103810 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb374112760 con 0x7fb374104130 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb379c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb374112a60 con 0x7fb37410cc70 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb373e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb373e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43910/0 (socket says 172.21.15.138:43910) 2024-08-20T01:08:35.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb373e00640 1 -- 172.21.15.138:0/1796006525 learned_addr learned my addr 172.21.15.138:0/1796006525 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.710+0000 7fb373400640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb374101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb379200640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb37410cc70 0x7fb374101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 -- 172.21.15.138:0/1796006525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb37410cc70 msgr2=0x7fb374101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb37410cc70 0x7fb374101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 -- 172.21.15.138:0/1796006525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 msgr2=0x7fb374101a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb374101a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 -- 172.21.15.138:0/1796006525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3741a7230 con 0x7fb374103810 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373400640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb374101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.711+0000 7fb373e00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb36400e9f0 tx=0x7fb36400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.712+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36400ccc0 con 0x7fb374103810 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.712+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3741a7520 con 0x7fb374103810 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.712+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3640054d0 con 0x7fb374103810 2024-08-20T01:08:35.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.712+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb364010430 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.712+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3741a7a30 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.713+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb374103bf0 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.715+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fb3640046c0 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.717+0000 7fb370c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 0x7fb3580b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.717+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fb3640e4670 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.717+0000 7fb373400640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 0x7fb3580b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.718+0000 7fb373400640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 0x7fb3580b62a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb3600303f0 tx=0x7fb360030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.719+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb364055e20 con 0x7fb374103810 2024-08-20T01:08:35.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.904+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3740632c0 con 0x7fb3580b3de0 2024-08-20T01:08:35.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.918+0000 7fb370c00640 1 -- 172.21.15.138:0/1796006525 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340793 (secure 0 0 0) 0x7fb3740632c0 con 0x7fb3580b3de0 2024-08-20T01:08:35.928 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:35.931 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 msgr2=0x7fb3580b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 0x7fb3580b62a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb3600303f0 tx=0x7fb360030340 comp rx=0 tx=0).stop 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 msgr2=0x7fb3741014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb36400e9f0 tx=0x7fb36400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 shutdown_connections 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb3580b3de0 0x7fb3580b62a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb37410cc70 0x7fb374101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb374104130 0x7fb374101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 --2- 172.21.15.138:0/1796006525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb374103810 0x7fb3741014c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:35.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 >> 172.21.15.138:0/1796006525 conn(0x7fb3740fd2d0 msgr2=0x7fb3740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:35.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 shutdown_connections 2024-08-20T01:08:35.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:35.923+0000 7fb379c00640 1 -- 172.21.15.138:0/1796006525 wait complete. 2024-08-20T01:08:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:35 smithi167 ceph-mon[21965]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:35 smithi167 ceph-mon[21965]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:35 smithi167 ceph-mon[21965]: pgmap v376: 129 pgs: 129 active+clean; 1.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 101 KiB/s wr, 41 op/s 2024-08-20T01:08:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:35 smithi142 ceph-mon[21799]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:35 smithi142 ceph-mon[21799]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:08:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:35 smithi142 ceph-mon[21799]: pgmap v376: 129 pgs: 129 active+clean; 1.0 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 101 KiB/s wr, 41 op/s 2024-08-20T01:08:36.805 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:36 smithi138 ceph-mon[18076]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-20T01:08:36.842 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-08-20T01:08:34.523440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1052827,"num_objects":65,"num_object_clones":0,"num_object_copies":197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":0,"num_read":1181,"num_read_kb":806,"num_write":454,"num_write_kb":3495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":828,"ondisk_log_size":828,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":352908,"kb_used_data":11976,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588916,"statfs":{"total":1151940427776,"available":1151579049984,"internally_reserved":0,"allocated":12263424,"data_stored":9341442,"data_compressed":483827,"data_compressed_allocated":651264,"data_compressed_original":3809280,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":393216,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":352,"num_read_kb":289,"num_write":152,"num_write_kb":1217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003465"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621129+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621129+0000","last_peered":"2024-08-20T01:06:36.621129+0000","last_clean":"2024-08-20T01:06:36.621129+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621129+0000","last_undegraded":"2024-08-20T01:06:36.621129+0000","last_fullsized":"2024-08-20T01:06:36.621129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731638+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:07:34.731638+0000","last_peered":"2024-08-20T01:07:34.731638+0000","last_clean":"2024-08-20T01:07:34.731638+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:07:34.731638+0000","last_undegraded":"2024-08-20T01:07:34.731638+0000","last_fullsized":"2024-08-20T01:07:34.731638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.427003+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.427003+0000","last_peered":"2024-08-20T01:06:37.427003+0000","last_clean":"2024-08-20T01:06:37.427003+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.427003+0000","last_undegraded":"2024-08-20T01:06:37.427003+0000","last_fullsized":"2024-08-20T01:06:37.427003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564918+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564918+0000","last_peered":"2024-08-20T01:06:36.564918+0000","last_clean":"2024-08-20T01:06:36.564918+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564918+0000","last_undegraded":"2024-08-20T01:06:36.564918+0000","last_fullsized":"2024-08-20T01:06:36.564918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747371+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:07:34.747371+0000","last_peered":"2024-08-20T01:07:34.747371+0000","last_clean":"2024-08-20T01:07:34.747371+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:07:34.747371+0000","last_undegraded":"2024-08-20T01:07:34.747371+0000","last_fullsized":"2024-08-20T01:07:34.747371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564478+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564478+0000","last_peered":"2024-08-20T01:06:36.564478+0000","last_clean":"2024-08-20T01:06:36.564478+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564478+0000","last_undegraded":"2024-08-20T01:06:36.564478+0000","last_fullsized":"2024-08-20T01:06:36.564478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566603+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566603+0000","last_peered":"2024-08-20T01:06:36.566603+0000","last_clean":"2024-08-20T01:06:36.566603+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566603+0000","last_undegraded":"2024-08-20T01:06:36.566603+0000","last_fullsized":"2024-08-20T01:06:36.566603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746216+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:07:34.746216+0000","last_peered":"2024-08-20T01:07:34.746216+0000","last_clean":"2024-08-20T01:07:34.746216+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:07:34.746216+0000","last_undegraded":"2024-08-20T01:07:34.746216+0000","last_fullsized":"2024-08-20T01:07:34.746216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649471+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649471+0000","last_peered":"2024-08-20T01:06:36.649471+0000","last_clean":"2024-08-20T01:06:36.649471+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649471+0000","last_undegraded":"2024-08-20T01:06:36.649471+0000","last_fullsized":"2024-08-20T01:06:36.649471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567327+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567327+0000","last_peered":"2024-08-20T01:06:36.567327+0000","last_clean":"2024-08-20T01:06:36.567327+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567327+0000","last_undegraded":"2024-08-20T01:06:36.567327+0000","last_fullsized":"2024-08-20T01:06:36.567327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730346+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:07:34.730346+0000","last_peered":"2024-08-20T01:07:34.730346+0000","last_clean":"2024-08-20T01:07:34.730346+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:07:34.730346+0000","last_undegraded":"2024-08-20T01:07:34.730346+0000","last_fullsized":"2024-08-20T01:07:34.730346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564624+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564624+0000","last_peered":"2024-08-20T01:06:36.564624+0000","last_clean":"2024-08-20T01:06:36.564624+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564624+0000","last_undegraded":"2024-08-20T01:06:36.564624+0000","last_fullsized":"2024-08-20T01:06:36.564624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733459+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:07:34.733459+0000","last_peered":"2024-08-20T01:07:34.733459+0000","last_clean":"2024-08-20T01:07:34.733459+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:07:34.733459+0000","last_undegraded":"2024-08-20T01:07:34.733459+0000","last_fullsized":"2024-08-20T01:07:34.733459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564869+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564869+0000","last_peered":"2024-08-20T01:06:36.564869+0000","last_clean":"2024-08-20T01:06:36.564869+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564869+0000","last_undegraded":"2024-08-20T01:06:36.564869+0000","last_fullsized":"2024-08-20T01:06:36.564869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739679+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:07:34.739679+0000","last_peered":"2024-08-20T01:07:34.739679+0000","last_clean":"2024-08-20T01:07:34.739679+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:07:34.739679+0000","last_undegraded":"2024-08-20T01:07:34.739679+0000","last_fullsized":"2024-08-20T01:07:34.739679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566999+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:06:36.566999+0000","last_peered":"2024-08-20T01:06:36.566999+0000","last_clean":"2024-08-20T01:06:36.566999+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:06:36.566999+0000","last_undegraded":"2024-08-20T01:06:36.566999+0000","last_fullsized":"2024-08-20T01:06:36.566999+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564675+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564675+0000","last_peered":"2024-08-20T01:06:36.564675+0000","last_clean":"2024-08-20T01:06:36.564675+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564675+0000","last_undegraded":"2024-08-20T01:06:36.564675+0000","last_fullsized":"2024-08-20T01:06:36.564675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741181+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:07:34.741181+0000","last_peered":"2024-08-20T01:07:34.741181+0000","last_clean":"2024-08-20T01:07:34.741181+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:07:34.741181+0000","last_undegraded":"2024-08-20T01:07:34.741181+0000","last_fullsized":"2024-08-20T01:07:34.741181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564578+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564578+0000","last_peered":"2024-08-20T01:06:36.564578+0000","last_clean":"2024-08-20T01:06:36.564578+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564578+0000","last_undegraded":"2024-08-20T01:06:36.564578+0000","last_fullsized":"2024-08-20T01:06:36.564578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620950+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620950+0000","last_peered":"2024-08-20T01:06:36.620950+0000","last_clean":"2024-08-20T01:06:36.620950+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620950+0000","last_undegraded":"2024-08-20T01:06:36.620950+0000","last_fullsized":"2024-08-20T01:06:36.620950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727978+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:07:34.727978+0000","last_peered":"2024-08-20T01:07:34.727978+0000","last_clean":"2024-08-20T01:07:34.727978+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:07:34.727978+0000","last_undegraded":"2024-08-20T01:07:34.727978+0000","last_fullsized":"2024-08-20T01:07:34.727978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642277+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642277+0000","last_peered":"2024-08-20T01:06:36.642277+0000","last_clean":"2024-08-20T01:06:36.642277+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642277+0000","last_undegraded":"2024-08-20T01:06:36.642277+0000","last_fullsized":"2024-08-20T01:06:36.642277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565004+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.565004+0000","last_peered":"2024-08-20T01:06:36.565004+0000","last_clean":"2024-08-20T01:06:36.565004+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.565004+0000","last_undegraded":"2024-08-20T01:06:36.565004+0000","last_fullsized":"2024-08-20T01:06:36.565004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738711+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:07:34.738711+0000","last_peered":"2024-08-20T01:07:34.738711+0000","last_clean":"2024-08-20T01:07:34.738711+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:07:34.738711+0000","last_undegraded":"2024-08-20T01:07:34.738711+0000","last_fullsized":"2024-08-20T01:07:34.738711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565514+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565514+0000","last_peered":"2024-08-20T01:06:36.565514+0000","last_clean":"2024-08-20T01:06:36.565514+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565514+0000","last_undegraded":"2024-08-20T01:06:36.565514+0000","last_fullsized":"2024-08-20T01:06:36.565514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567004+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.567004+0000","last_peered":"2024-08-20T01:06:36.567004+0000","last_clean":"2024-08-20T01:06:36.567004+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.567004+0000","last_undegraded":"2024-08-20T01:06:36.567004+0000","last_fullsized":"2024-08-20T01:06:36.567004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727759+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:07:34.727759+0000","last_peered":"2024-08-20T01:07:34.727759+0000","last_clean":"2024-08-20T01:07:34.727759+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:07:34.727759+0000","last_undegraded":"2024-08-20T01:07:34.727759+0000","last_fullsized":"2024-08-20T01:07:34.727759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'95","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:34.030555+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:08:34.030555+0000","last_peered":"2024-08-20T01:08:34.030555+0000","last_clean":"2024-08-20T01:08:34.030555+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:08:34.030555+0000","last_undegraded":"2024-08-20T01:08:34.030555+0000","last_fullsized":"2024-08-20T01:08:34.030555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":175,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564152+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564152+0000","last_peered":"2024-08-20T01:06:36.564152+0000","last_clean":"2024-08-20T01:06:36.564152+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564152+0000","last_undegraded":"2024-08-20T01:06:36.564152+0000","last_fullsized":"2024-08-20T01:06:36.564152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729501+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:07:34.729501+0000","last_peered":"2024-08-20T01:07:34.729501+0000","last_clean":"2024-08-20T01:07:34.729501+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:07:34.729501+0000","last_undegraded":"2024-08-20T01:07:34.729501+0000","last_fullsized":"2024-08-20T01:07:34.729501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566711+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566711+0000","last_peered":"2024-08-20T01:06:36.566711+0000","last_clean":"2024-08-20T01:06:36.566711+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566711+0000","last_undegraded":"2024-08-20T01:06:36.566711+0000","last_fullsized":"2024-08-20T01:06:36.566711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338363+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338363+0000","last_peered":"2024-08-20T01:06:37.338363+0000","last_clean":"2024-08-20T01:06:37.338363+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338363+0000","last_undegraded":"2024-08-20T01:06:37.338363+0000","last_fullsized":"2024-08-20T01:06:37.338363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732734+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:07:34.732734+0000","last_peered":"2024-08-20T01:07:34.732734+0000","last_clean":"2024-08-20T01:07:34.732734+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:07:34.732734+0000","last_undegraded":"2024-08-20T01:07:34.732734+0000","last_fullsized":"2024-08-20T01:07:34.732734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620842+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620842+0000","last_peered":"2024-08-20T01:06:36.620842+0000","last_clean":"2024-08-20T01:06:36.620842+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620842+0000","last_undegraded":"2024-08-20T01:06:36.620842+0000","last_fullsized":"2024-08-20T01:06:36.620842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566765+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566765+0000","last_peered":"2024-08-20T01:06:36.566765+0000","last_clean":"2024-08-20T01:06:36.566765+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566765+0000","last_undegraded":"2024-08-20T01:06:36.566765+0000","last_fullsized":"2024-08-20T01:06:36.566765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740243+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:07:34.740243+0000","last_peered":"2024-08-20T01:07:34.740243+0000","last_clean":"2024-08-20T01:07:34.740243+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:07:34.740243+0000","last_undegraded":"2024-08-20T01:07:34.740243+0000","last_fullsized":"2024-08-20T01:07:34.740243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900778+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900778+0000","last_peered":"2024-08-20T01:06:36.900778+0000","last_clean":"2024-08-20T01:06:36.900778+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900778+0000","last_undegraded":"2024-08-20T01:06:36.900778+0000","last_fullsized":"2024-08-20T01:06:36.900778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566915+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566915+0000","last_peered":"2024-08-20T01:06:36.566915+0000","last_clean":"2024-08-20T01:06:36.566915+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566915+0000","last_undegraded":"2024-08-20T01:06:36.566915+0000","last_fullsized":"2024-08-20T01:06:36.566915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731513+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:07:34.731513+0000","last_peered":"2024-08-20T01:07:34.731513+0000","last_clean":"2024-08-20T01:07:34.731513+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:07:34.731513+0000","last_undegraded":"2024-08-20T01:07:34.731513+0000","last_fullsized":"2024-08-20T01:07:34.731513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":176,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:28.817055+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:08:28.817055+0000","last_peered":"2024-08-20T01:08:28.817055+0000","last_clean":"2024-08-20T01:08:28.817055+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:08:28.817055+0000","last_undegraded":"2024-08-20T01:08:28.817055+0000","last_fullsized":"2024-08-20T01:08:28.817055+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":130,"num_read_kb":130,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567516+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567516+0000","last_peered":"2024-08-20T01:06:36.567516+0000","last_clean":"2024-08-20T01:06:36.567516+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567516+0000","last_undegraded":"2024-08-20T01:06:36.567516+0000","last_fullsized":"2024-08-20T01:06:36.567516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727983+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:07:34.727983+0000","last_peered":"2024-08-20T01:07:34.727983+0000","last_clean":"2024-08-20T01:07:34.727983+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:07:34.727983+0000","last_undegraded":"2024-08-20T01:07:34.727983+0000","last_fullsized":"2024-08-20T01:07:34.727983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900807+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900807+0000","last_peered":"2024-08-20T01:06:36.900807+0000","last_clean":"2024-08-20T01:06:36.900807+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900807+0000","last_undegraded":"2024-08-20T01:06:36.900807+0000","last_fullsized":"2024-08-20T01:06:36.900807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647669+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647669+0000","last_peered":"2024-08-20T01:06:36.647669+0000","last_clean":"2024-08-20T01:06:36.647669+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647669+0000","last_undegraded":"2024-08-20T01:06:36.647669+0000","last_fullsized":"2024-08-20T01:06:36.647669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747444+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:07:34.747444+0000","last_peered":"2024-08-20T01:07:34.747444+0000","last_clean":"2024-08-20T01:07:34.747444+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:07:34.747444+0000","last_undegraded":"2024-08-20T01:07:34.747444+0000","last_fullsized":"2024-08-20T01:07:34.747444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565335+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565335+0000","last_peered":"2024-08-20T01:06:36.565335+0000","last_clean":"2024-08-20T01:06:36.565335+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565335+0000","last_undegraded":"2024-08-20T01:06:36.565335+0000","last_fullsized":"2024-08-20T01:06:36.565335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900855+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900855+0000","last_peered":"2024-08-20T01:06:36.900855+0000","last_clean":"2024-08-20T01:06:36.900855+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900855+0000","last_undegraded":"2024-08-20T01:06:36.900855+0000","last_fullsized":"2024-08-20T01:06:36.900855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732158+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:07:34.732158+0000","last_peered":"2024-08-20T01:07:34.732158+0000","last_clean":"2024-08-20T01:07:34.732158+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:07:34.732158+0000","last_undegraded":"2024-08-20T01:07:34.732158+0000","last_fullsized":"2024-08-20T01:07:34.732158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567812+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567812+0000","last_peered":"2024-08-20T01:06:36.567812+0000","last_clean":"2024-08-20T01:06:36.567812+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567812+0000","last_undegraded":"2024-08-20T01:06:36.567812+0000","last_fullsized":"2024-08-20T01:06:36.567812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642230+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642230+0000","last_peered":"2024-08-20T01:06:36.642230+0000","last_clean":"2024-08-20T01:06:36.642230+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642230+0000","last_undegraded":"2024-08-20T01:06:36.642230+0000","last_fullsized":"2024-08-20T01:06:36.642230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731242+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:07:34.731242+0000","last_peered":"2024-08-20T01:07:34.731242+0000","last_clean":"2024-08-20T01:07:34.731242+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:07:34.731242+0000","last_undegraded":"2024-08-20T01:07:34.731242+0000","last_fullsized":"2024-08-20T01:07:34.731242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647722+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:06:36.647722+0000","last_peered":"2024-08-20T01:06:36.647722+0000","last_clean":"2024-08-20T01:06:36.647722+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:06:36.647722+0000","last_undegraded":"2024-08-20T01:06:36.647722+0000","last_fullsized":"2024-08-20T01:06:36.647722+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425109+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425109+0000","last_peered":"2024-08-20T01:06:37.425109+0000","last_clean":"2024-08-20T01:06:37.425109+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425109+0000","last_undegraded":"2024-08-20T01:06:37.425109+0000","last_fullsized":"2024-08-20T01:06:37.425109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731342+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:07:34.731342+0000","last_peered":"2024-08-20T01:07:34.731342+0000","last_clean":"2024-08-20T01:07:34.731342+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:07:34.731342+0000","last_undegraded":"2024-08-20T01:07:34.731342+0000","last_fullsized":"2024-08-20T01:07:34.731342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646941+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646941+0000","last_peered":"2024-08-20T01:06:36.646941+0000","last_clean":"2024-08-20T01:06:36.646941+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646941+0000","last_undegraded":"2024-08-20T01:06:36.646941+0000","last_fullsized":"2024-08-20T01:06:36.646941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739075+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:07:34.739075+0000","last_peered":"2024-08-20T01:07:34.739075+0000","last_clean":"2024-08-20T01:07:34.739075+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:07:34.739075+0000","last_undegraded":"2024-08-20T01:07:34.739075+0000","last_fullsized":"2024-08-20T01:07:34.739075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566920+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566920+0000","last_peered":"2024-08-20T01:06:36.566920+0000","last_clean":"2024-08-20T01:06:36.566920+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566920+0000","last_undegraded":"2024-08-20T01:06:36.566920+0000","last_fullsized":"2024-08-20T01:06:36.566920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731827+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:07:34.731827+0000","last_peered":"2024-08-20T01:07:34.731827+0000","last_clean":"2024-08-20T01:07:34.731827+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:07:34.731827+0000","last_undegraded":"2024-08-20T01:07:34.731827+0000","last_fullsized":"2024-08-20T01:07:34.731827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338193+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338193+0000","last_peered":"2024-08-20T01:06:37.338193+0000","last_clean":"2024-08-20T01:06:37.338193+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338193+0000","last_undegraded":"2024-08-20T01:06:37.338193+0000","last_fullsized":"2024-08-20T01:06:37.338193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340579+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340579+0000","last_peered":"2024-08-20T01:06:37.340579+0000","last_clean":"2024-08-20T01:06:37.340579+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340579+0000","last_undegraded":"2024-08-20T01:06:37.340579+0000","last_fullsized":"2024-08-20T01:06:37.340579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724142+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:07:34.724142+0000","last_peered":"2024-08-20T01:07:34.724142+0000","last_clean":"2024-08-20T01:07:34.724142+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:07:34.724142+0000","last_undegraded":"2024-08-20T01:07:34.724142+0000","last_fullsized":"2024-08-20T01:07:34.724142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":177,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:28.817893+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:08:28.817893+0000","last_peered":"2024-08-20T01:08:28.817893+0000","last_clean":"2024-08-20T01:08:28.817893+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:08:28.817893+0000","last_undegraded":"2024-08-20T01:08:28.817893+0000","last_fullsized":"2024-08-20T01:08:28.817893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621011+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.621011+0000","last_peered":"2024-08-20T01:06:36.621011+0000","last_clean":"2024-08-20T01:06:36.621011+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.621011+0000","last_undegraded":"2024-08-20T01:06:36.621011+0000","last_fullsized":"2024-08-20T01:06:36.621011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739960+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:07:34.739960+0000","last_peered":"2024-08-20T01:07:34.739960+0000","last_clean":"2024-08-20T01:07:34.739960+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:07:34.739960+0000","last_undegraded":"2024-08-20T01:07:34.739960+0000","last_fullsized":"2024-08-20T01:07:34.739960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647459+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647459+0000","last_peered":"2024-08-20T01:06:36.647459+0000","last_clean":"2024-08-20T01:06:36.647459+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647459+0000","last_undegraded":"2024-08-20T01:06:36.647459+0000","last_fullsized":"2024-08-20T01:06:36.647459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338356+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338356+0000","last_peered":"2024-08-20T01:06:37.338356+0000","last_clean":"2024-08-20T01:06:37.338356+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338356+0000","last_undegraded":"2024-08-20T01:06:37.338356+0000","last_fullsized":"2024-08-20T01:06:37.338356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725099+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:07:34.725099+0000","last_peered":"2024-08-20T01:07:34.725099+0000","last_clean":"2024-08-20T01:07:34.725099+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:07:34.725099+0000","last_undegraded":"2024-08-20T01:07:34.725099+0000","last_fullsized":"2024-08-20T01:07:34.725099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566749+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566749+0000","last_peered":"2024-08-20T01:06:36.566749+0000","last_clean":"2024-08-20T01:06:36.566749+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566749+0000","last_undegraded":"2024-08-20T01:06:36.566749+0000","last_fullsized":"2024-08-20T01:06:36.566749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647424+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647424+0000","last_peered":"2024-08-20T01:06:36.647424+0000","last_clean":"2024-08-20T01:06:36.647424+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647424+0000","last_undegraded":"2024-08-20T01:06:36.647424+0000","last_fullsized":"2024-08-20T01:06:36.647424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.719431+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:07:34.719431+0000","last_peered":"2024-08-20T01:07:34.719431+0000","last_clean":"2024-08-20T01:07:34.719431+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:07:34.719431+0000","last_undegraded":"2024-08-20T01:07:34.719431+0000","last_fullsized":"2024-08-20T01:07:34.719431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'38","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:28.817809+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:08:28.817809+0000","last_peered":"2024-08-20T01:08:28.817809+0000","last_clean":"2024-08-20T01:08:28.817809+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:08:28.817809+0000","last_undegraded":"2024-08-20T01:08:28.817809+0000","last_fullsized":"2024-08-20T01:08:28.817809+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64867,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":270,"num_read_kb":199,"num_write":39,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620266+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620266+0000","last_peered":"2024-08-20T01:06:36.620266+0000","last_clean":"2024-08-20T01:06:36.620266+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620266+0000","last_undegraded":"2024-08-20T01:06:36.620266+0000","last_fullsized":"2024-08-20T01:06:36.620266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747475+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:07:34.747475+0000","last_peered":"2024-08-20T01:07:34.747475+0000","last_clean":"2024-08-20T01:07:34.747475+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:07:34.747475+0000","last_undegraded":"2024-08-20T01:07:34.747475+0000","last_fullsized":"2024-08-20T01:07:34.747475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647695+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647695+0000","last_peered":"2024-08-20T01:06:36.647695+0000","last_clean":"2024-08-20T01:06:36.647695+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647695+0000","last_undegraded":"2024-08-20T01:06:36.647695+0000","last_fullsized":"2024-08-20T01:06:36.647695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565025+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.565025+0000","last_peered":"2024-08-20T01:06:36.565025+0000","last_clean":"2024-08-20T01:06:36.565025+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.565025+0000","last_undegraded":"2024-08-20T01:06:36.565025+0000","last_fullsized":"2024-08-20T01:06:36.565025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740128+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:07:34.740128+0000","last_peered":"2024-08-20T01:07:34.740128+0000","last_clean":"2024-08-20T01:07:34.740128+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:07:34.740128+0000","last_undegraded":"2024-08-20T01:07:34.740128+0000","last_fullsized":"2024-08-20T01:07:34.740128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901002+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.901002+0000","last_peered":"2024-08-20T01:06:36.901002+0000","last_clean":"2024-08-20T01:06:36.901002+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.901002+0000","last_undegraded":"2024-08-20T01:06:36.901002+0000","last_fullsized":"2024-08-20T01:06:36.901002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620177+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620177+0000","last_peered":"2024-08-20T01:06:36.620177+0000","last_clean":"2024-08-20T01:06:36.620177+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620177+0000","last_undegraded":"2024-08-20T01:06:36.620177+0000","last_fullsized":"2024-08-20T01:06:36.620177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746528+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:07:34.746528+0000","last_peered":"2024-08-20T01:07:34.746528+0000","last_clean":"2024-08-20T01:07:34.746528+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:07:34.746528+0000","last_undegraded":"2024-08-20T01:07:34.746528+0000","last_fullsized":"2024-08-20T01:07:34.746528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":175,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:27.815952+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:08:27.815952+0000","last_peered":"2024-08-20T01:08:27.815952+0000","last_clean":"2024-08-20T01:08:27.815952+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:08:27.815952+0000","last_undegraded":"2024-08-20T01:08:27.815952+0000","last_fullsized":"2024-08-20T01:08:27.815952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564777+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564777+0000","last_peered":"2024-08-20T01:06:36.564777+0000","last_clean":"2024-08-20T01:06:36.564777+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564777+0000","last_undegraded":"2024-08-20T01:06:36.564777+0000","last_fullsized":"2024-08-20T01:06:36.564777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425144+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425144+0000","last_peered":"2024-08-20T01:06:37.425144+0000","last_clean":"2024-08-20T01:06:37.425144+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425144+0000","last_undegraded":"2024-08-20T01:06:37.425144+0000","last_fullsized":"2024-08-20T01:06:37.425144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731657+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:07:34.731657+0000","last_peered":"2024-08-20T01:07:34.731657+0000","last_clean":"2024-08-20T01:07:34.731657+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:07:34.731657+0000","last_undegraded":"2024-08-20T01:07:34.731657+0000","last_fullsized":"2024-08-20T01:07:34.731657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647535+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647535+0000","last_peered":"2024-08-20T01:06:36.647535+0000","last_clean":"2024-08-20T01:06:36.647535+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647535+0000","last_undegraded":"2024-08-20T01:06:36.647535+0000","last_fullsized":"2024-08-20T01:06:36.647535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338530+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338530+0000","last_peered":"2024-08-20T01:06:37.338530+0000","last_clean":"2024-08-20T01:06:37.338530+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338530+0000","last_undegraded":"2024-08-20T01:06:37.338530+0000","last_fullsized":"2024-08-20T01:06:37.338530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728742+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:07:34.728742+0000","last_peered":"2024-08-20T01:07:34.728742+0000","last_clean":"2024-08-20T01:07:34.728742+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:07:34.728742+0000","last_undegraded":"2024-08-20T01:07:34.728742+0000","last_fullsized":"2024-08-20T01:07:34.728742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566516+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566516+0000","last_peered":"2024-08-20T01:06:36.566516+0000","last_clean":"2024-08-20T01:06:36.566516+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566516+0000","last_undegraded":"2024-08-20T01:06:36.566516+0000","last_fullsized":"2024-08-20T01:06:36.566516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644361+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644361+0000","last_peered":"2024-08-20T01:06:36.644361+0000","last_clean":"2024-08-20T01:06:36.644361+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644361+0000","last_undegraded":"2024-08-20T01:06:36.644361+0000","last_fullsized":"2024-08-20T01:06:36.644361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732168+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:07:34.732168+0000","last_peered":"2024-08-20T01:07:34.732168+0000","last_clean":"2024-08-20T01:07:34.732168+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:07:34.732168+0000","last_undegraded":"2024-08-20T01:07:34.732168+0000","last_fullsized":"2024-08-20T01:07:34.732168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566548+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566548+0000","last_peered":"2024-08-20T01:06:36.566548+0000","last_clean":"2024-08-20T01:06:36.566548+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566548+0000","last_undegraded":"2024-08-20T01:06:36.566548+0000","last_fullsized":"2024-08-20T01:06:36.566548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'7","reported_seq":182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:27.816674+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:08:27.816674+0000","last_peered":"2024-08-20T01:08:27.816674+0000","last_clean":"2024-08-20T01:08:27.816674+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:08:27.816674+0000","last_undegraded":"2024-08-20T01:08:27.816674+0000","last_fullsized":"2024-08-20T01:08:27.816674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732844+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:07:34.732844+0000","last_peered":"2024-08-20T01:07:34.732844+0000","last_clean":"2024-08-20T01:07:34.732844+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:07:34.732844+0000","last_undegraded":"2024-08-20T01:07:34.732844+0000","last_fullsized":"2024-08-20T01:07:34.732844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567404+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567404+0000","last_peered":"2024-08-20T01:06:36.567404+0000","last_clean":"2024-08-20T01:06:36.567404+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567404+0000","last_undegraded":"2024-08-20T01:06:36.567404+0000","last_fullsized":"2024-08-20T01:06:36.567404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732743+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:07:34.732743+0000","last_peered":"2024-08-20T01:07:34.732743+0000","last_clean":"2024-08-20T01:07:34.732743+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:07:34.732743+0000","last_undegraded":"2024-08-20T01:07:34.732743+0000","last_fullsized":"2024-08-20T01:07:34.732743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733589+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:07:34.733589+0000","last_peered":"2024-08-20T01:07:34.733589+0000","last_clean":"2024-08-20T01:07:34.733589+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:07:34.733589+0000","last_undegraded":"2024-08-20T01:07:34.733589+0000","last_fullsized":"2024-08-20T01:07:34.733589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739063+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:07:34.739063+0000","last_peered":"2024-08-20T01:07:34.739063+0000","last_clean":"2024-08-20T01:07:34.739063+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:07:34.739063+0000","last_undegraded":"2024-08-20T01:07:34.739063+0000","last_fullsized":"2024-08-20T01:07:34.739063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739518+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:07:34.739518+0000","last_peered":"2024-08-20T01:07:34.739518+0000","last_clean":"2024-08-20T01:07:34.739518+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:07:34.739518+0000","last_undegraded":"2024-08-20T01:07:34.739518+0000","last_fullsized":"2024-08-20T01:07:34.739518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740881+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:07:34.740881+0000","last_peered":"2024-08-20T01:07:34.740881+0000","last_clean":"2024-08-20T01:07:34.740881+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:07:34.740881+0000","last_undegraded":"2024-08-20T01:07:34.740881+0000","last_fullsized":"2024-08-20T01:07:34.740881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729504+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:07:34.729504+0000","last_peered":"2024-08-20T01:07:34.729504+0000","last_clean":"2024-08-20T01:07:34.729504+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:07:34.729504+0000","last_undegraded":"2024-08-20T01:07:34.729504+0000","last_fullsized":"2024-08-20T01:07:34.729504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725362+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:07:34.725362+0000","last_peered":"2024-08-20T01:07:34.725362+0000","last_clean":"2024-08-20T01:07:34.725362+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:07:34.725362+0000","last_undegraded":"2024-08-20T01:07:34.725362+0000","last_fullsized":"2024-08-20T01:07:34.725362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728179+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:07:34.728179+0000","last_peered":"2024-08-20T01:07:34.728179+0000","last_clean":"2024-08-20T01:07:34.728179+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:07:34.728179+0000","last_undegraded":"2024-08-20T01:07:34.728179+0000","last_fullsized":"2024-08-20T01:07:34.728179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732606+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:07:34.732606+0000","last_peered":"2024-08-20T01:07:34.732606+0000","last_clean":"2024-08-20T01:07:34.732606+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:07:34.732606+0000","last_undegraded":"2024-08-20T01:07:34.732606+0000","last_fullsized":"2024-08-20T01:07:34.732606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729298+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:07:34.729298+0000","last_peered":"2024-08-20T01:07:34.729298+0000","last_clean":"2024-08-20T01:07:34.729298+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:07:34.729298+0000","last_undegraded":"2024-08-20T01:07:34.729298+0000","last_fullsized":"2024-08-20T01:07:34.729298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747462+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:07:34.747462+0000","last_peered":"2024-08-20T01:07:34.747462+0000","last_clean":"2024-08-20T01:07:34.747462+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:07:34.747462+0000","last_undegraded":"2024-08-20T01:07:34.747462+0000","last_fullsized":"2024-08-20T01:07:34.747462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741436+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:07:34.741436+0000","last_peered":"2024-08-20T01:07:34.741436+0000","last_clean":"2024-08-20T01:07:34.741436+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:07:34.741436+0000","last_undegraded":"2024-08-20T01:07:34.741436+0000","last_fullsized":"2024-08-20T01:07:34.741436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731490+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:07:34.731490+0000","last_peered":"2024-08-20T01:07:34.731490+0000","last_clean":"2024-08-20T01:07:34.731490+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:07:34.731490+0000","last_undegraded":"2024-08-20T01:07:34.731490+0000","last_fullsized":"2024-08-20T01:07:34.731490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732616+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:07:34.732616+0000","last_peered":"2024-08-20T01:07:34.732616+0000","last_clean":"2024-08-20T01:07:34.732616+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:07:34.732616+0000","last_undegraded":"2024-08-20T01:07:34.732616+0000","last_fullsized":"2024-08-20T01:07:34.732616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.735563+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:07:34.735563+0000","last_peered":"2024-08-20T01:07:34.735563+0000","last_clean":"2024-08-20T01:07:34.735563+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:07:34.735563+0000","last_undegraded":"2024-08-20T01:07:34.735563+0000","last_fullsized":"2024-08-20T01:07:34.735563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742307+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:07:34.742307+0000","last_peered":"2024-08-20T01:07:34.742307+0000","last_clean":"2024-08-20T01:07:34.742307+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:07:34.742307+0000","last_undegraded":"2024-08-20T01:07:34.742307+0000","last_fullsized":"2024-08-20T01:07:34.742307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747020+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:07:34.747020+0000","last_peered":"2024-08-20T01:07:34.747020+0000","last_clean":"2024-08-20T01:07:34.747020+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:07:34.747020+0000","last_undegraded":"2024-08-20T01:07:34.747020+0000","last_fullsized":"2024-08-20T01:07:34.747020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740301+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:07:34.740301+0000","last_peered":"2024-08-20T01:07:34.740301+0000","last_clean":"2024-08-20T01:07:34.740301+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:07:34.740301+0000","last_undegraded":"2024-08-20T01:07:34.740301+0000","last_fullsized":"2024-08-20T01:07:34.740301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740343+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:07:34.740343+0000","last_peered":"2024-08-20T01:07:34.740343+0000","last_clean":"2024-08-20T01:07:34.740343+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:07:34.740343+0000","last_undegraded":"2024-08-20T01:07:34.740343+0000","last_fullsized":"2024-08-20T01:07:34.740343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738693+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:07:34.738693+0000","last_peered":"2024-08-20T01:07:34.738693+0000","last_clean":"2024-08-20T01:07:34.738693+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:07:34.738693+0000","last_undegraded":"2024-08-20T01:07:34.738693+0000","last_fullsized":"2024-08-20T01:07:34.738693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.736334+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:07:34.736334+0000","last_peered":"2024-08-20T01:07:34.736334+0000","last_clean":"2024-08-20T01:07:34.736334+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:07:34.736334+0000","last_undegraded":"2024-08-20T01:07:34.736334+0000","last_fullsized":"2024-08-20T01:07:34.736334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731015+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:07:34.731015+0000","last_peered":"2024-08-20T01:07:34.731015+0000","last_clean":"2024-08-20T01:07:34.731015+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:07:34.731015+0000","last_undegraded":"2024-08-20T01:07:34.731015+0000","last_fullsized":"2024-08-20T01:07:34.731015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740057+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:07:34.740057+0000","last_peered":"2024-08-20T01:07:34.740057+0000","last_clean":"2024-08-20T01:07:34.740057+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:07:34.740057+0000","last_undegraded":"2024-08-20T01:07:34.740057+0000","last_fullsized":"2024-08-20T01:07:34.740057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.744507+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:07:34.744507+0000","last_peered":"2024-08-20T01:07:34.744507+0000","last_clean":"2024-08-20T01:07:34.744507+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:07:34.744507+0000","last_undegraded":"2024-08-20T01:07:34.744507+0000","last_fullsized":"2024-08-20T01:07:34.744507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734496+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:07:34.734496+0000","last_peered":"2024-08-20T01:07:34.734496+0000","last_clean":"2024-08-20T01:07:34.734496+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:07:34.734496+0000","last_undegraded":"2024-08-20T01:07:34.734496+0000","last_fullsized":"2024-08-20T01:07:34.734496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730134+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:07:34.730134+0000","last_peered":"2024-08-20T01:07:34.730134+0000","last_clean":"2024-08-20T01:07:34.730134+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:07:34.730134+0000","last_undegraded":"2024-08-20T01:07:34.730134+0000","last_fullsized":"2024-08-20T01:07:34.730134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740683+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:07:34.740683+0000","last_peered":"2024-08-20T01:07:34.740683+0000","last_clean":"2024-08-20T01:07:34.740683+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:07:34.740683+0000","last_undegraded":"2024-08-20T01:07:34.740683+0000","last_fullsized":"2024-08-20T01:07:34.740683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734991+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:07:34.734991+0000","last_peered":"2024-08-20T01:07:34.734991+0000","last_clean":"2024-08-20T01:07:34.734991+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:07:34.734991+0000","last_undegraded":"2024-08-20T01:07:34.734991+0000","last_fullsized":"2024-08-20T01:07:34.734991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733118+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:07:34.733118+0000","last_peered":"2024-08-20T01:07:34.733118+0000","last_clean":"2024-08-20T01:07:34.733118+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:07:34.733118+0000","last_undegraded":"2024-08-20T01:07:34.733118+0000","last_fullsized":"2024-08-20T01:07:34.733118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747505+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:07:34.747505+0000","last_peered":"2024-08-20T01:07:34.747505+0000","last_clean":"2024-08-20T01:07:34.747505+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:07:34.747505+0000","last_undegraded":"2024-08-20T01:07:34.747505+0000","last_fullsized":"2024-08-20T01:07:34.747505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739436+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:07:34.739436+0000","last_peered":"2024-08-20T01:07:34.739436+0000","last_clean":"2024-08-20T01:07:34.739436+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:07:34.739436+0000","last_undegraded":"2024-08-20T01:07:34.739436+0000","last_fullsized":"2024-08-20T01:07:34.739436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742422+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:07:34.742422+0000","last_peered":"2024-08-20T01:07:34.742422+0000","last_clean":"2024-08-20T01:07:34.742422+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:07:34.742422+0000","last_undegraded":"2024-08-20T01:07:34.742422+0000","last_fullsized":"2024-08-20T01:07:34.742422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":462225,"num_objects":61,"num_object_clones":0,"num_object_copies":183,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":0,"num_read":1100,"num_read_kb":737,"num_write":329,"num_write_kb":1343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":1387546,"data_compressed":56669,"data_compressed_allocated":73728,"data_compressed_original":1155072,"omap_allocated":0,"internal_metadata":0},"log_size":234,"ondisk_log_size":234,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678067,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874296,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32276,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712876,"statfs":{"total":95995035648,"available":95961985024,"internally_reserved":0,"allocated":1118208,"data_stored":1169315,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":914728,"data_compressed":45926,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430279,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1113386,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593806,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790035,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":1108974,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953560,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":1299778,"data_compressed":64017,"data_compressed_allocated":86016,"data_compressed_original":835584,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378723,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1060864,"data_stored":579251,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":577680,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":66445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:08:36.848 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:08:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:36 smithi167 ceph-mon[21965]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-20T01:08:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:36 smithi142 ceph-mon[21799]: mgrmap e24: x(active, since 10m), standbys: z, y 2024-08-20T01:08:37.220 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 -- 172.21.15.138:0/846539073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 msgr2=0x7f2b34115a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 --2- 172.21.15.138:0/846539073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b34115a80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009be0 tx=0x7f2b2c02a6c0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 -- 172.21.15.138:0/846539073 shutdown_connections 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 --2- 172.21.15.138:0/846539073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b34115a80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 --2- 172.21.15.138:0/846539073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b34113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 --2- 172.21.15.138:0/846539073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 0x7f2b340777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 -- 172.21.15.138:0/846539073 >> 172.21.15.138:0/846539073 conn(0x7f2b341006f0 msgr2=0x7f2b34102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 -- 172.21.15.138:0/846539073 shutdown_connections 2024-08-20T01:08:37.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.822+0000 7f2b38e00640 1 -- 172.21.15.138:0/846539073 wait complete. 2024-08-20T01:08:37.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.823+0000 7f2b38e00640 1 Processor -- start 2024-08-20T01:08:37.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.823+0000 7f2b38e00640 1 -- start start 2024-08-20T01:08:37.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 0x7f2b3419f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:37.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b341a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34119070 con 0x7f2b34077d20 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2b34118ef0 con 0x7f2b34113690 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b38e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2b341191f0 con 0x7f2b34077400 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36300/0 (socket says 172.21.15.138:36300) 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 -- 172.21.15.138:0/3207565576 learned_addr learned my addr 172.21.15.138:0/3207565576 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b32800640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b341a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 0x7f2b3419f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 -- 172.21.15.138:0/3207565576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 msgr2=0x7f2b3419f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 0x7f2b3419f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.824 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 -- 172.21.15.138:0/3207565576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 msgr2=0x7f2b341a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b341a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.824+0000 7f2b31400640 1 -- 172.21.15.138:0/3207565576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b341a5680 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b31400640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9c0 tx=0x7f2b2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b2000cde0 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b341a5970 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b200035f0 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b20010690 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.825+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b341ac200 con 0x7f2b34077d20 2024-08-20T01:08:37.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.826+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b340777e0 con 0x7f2b34077d20 2024-08-20T01:08:37.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.828+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f2b20004000 con 0x7f2b34077d20 2024-08-20T01:08:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.830+0000 7f2b2aa00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 0x7f2b180b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.830+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2b200e4430 con 0x7f2b34077d20 2024-08-20T01:08:37.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.830+0000 7f2b31e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 0x7f2b180b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:37.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.831+0000 7f2b31e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 0x7f2b180b6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009bd0 tx=0x7f2b24005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:37.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:37.834+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2b20009510 con 0x7f2b34077d20 2024-08-20T01:08:37.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:37 smithi167 ceph-mon[21965]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:37 smithi167 ceph-mon[21965]: pgmap v377: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-08-20T01:08:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:37 smithi142 ceph-mon[21799]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:37 smithi142 ceph-mon[21799]: pgmap v377: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-08-20T01:08:38.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.013+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2b3410e850 con 0x7f2b180b3cb0 2024-08-20T01:08:38.011 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:37 smithi138 ceph-mon[18076]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:38.011 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:37 smithi138 ceph-mon[18076]: pgmap v377: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 53 op/s 2024-08-20T01:08:38.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.018+0000 7f2b2aa00640 1 -- 172.21.15.138:0/3207565576 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340842 (secure 0 0 0) 0x7f2b3410e850 con 0x7f2b180b3cb0 2024-08-20T01:08:38.021 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:38.022 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:08:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 msgr2=0x7f2b180b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 0x7f2b180b6170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009bd0 tx=0x7f2b24005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:38.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 msgr2=0x7f2b3419fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9c0 tx=0x7f2b2000ee80 comp rx=0 tx=0).stop 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 shutdown_connections 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2b180b3cb0 0x7f2b180b6170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b34113690 0x7f2b341a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34077d20 0x7f2b3419fb10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 --2- 172.21.15.138:0/3207565576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2b34077400 0x7f2b3419f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 >> 172.21.15.138:0/3207565576 conn(0x7f2b341006f0 msgr2=0x7f2b34102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 shutdown_connections 2024-08-20T01:08:38.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:38.023+0000 7f2b38e00640 1 -- 172.21.15.138:0/3207565576 wait complete. 2024-08-20T01:08:38.995 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-08-20T01:08:36.524048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1451539,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1245,"num_read_kb":854,"num_write":531,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":893,"ondisk_log_size":893,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353032,"kb_used_data":12084,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588792,"statfs":{"total":1151940427776,"available":1151578923008,"internally_reserved":0,"allocated":12374016,"data_stored":10134863,"data_compressed":520009,"data_compressed_allocated":700416,"data_compressed_original":4546560,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":791928,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":416,"num_read_kb":337,"num_write":229,"num_write_kb":2099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003537"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621129+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621129+0000","last_peered":"2024-08-20T01:06:36.621129+0000","last_clean":"2024-08-20T01:06:36.621129+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621129+0000","last_undegraded":"2024-08-20T01:06:36.621129+0000","last_fullsized":"2024-08-20T01:06:36.621129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731638+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:07:34.731638+0000","last_peered":"2024-08-20T01:07:34.731638+0000","last_clean":"2024-08-20T01:07:34.731638+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:07:34.731638+0000","last_undegraded":"2024-08-20T01:07:34.731638+0000","last_fullsized":"2024-08-20T01:07:34.731638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.427003+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.427003+0000","last_peered":"2024-08-20T01:06:37.427003+0000","last_clean":"2024-08-20T01:06:37.427003+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.427003+0000","last_undegraded":"2024-08-20T01:06:37.427003+0000","last_fullsized":"2024-08-20T01:06:37.427003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564918+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564918+0000","last_peered":"2024-08-20T01:06:36.564918+0000","last_clean":"2024-08-20T01:06:36.564918+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564918+0000","last_undegraded":"2024-08-20T01:06:36.564918+0000","last_fullsized":"2024-08-20T01:06:36.564918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747371+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:07:34.747371+0000","last_peered":"2024-08-20T01:07:34.747371+0000","last_clean":"2024-08-20T01:07:34.747371+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:07:34.747371+0000","last_undegraded":"2024-08-20T01:07:34.747371+0000","last_fullsized":"2024-08-20T01:07:34.747371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564478+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564478+0000","last_peered":"2024-08-20T01:06:36.564478+0000","last_clean":"2024-08-20T01:06:36.564478+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564478+0000","last_undegraded":"2024-08-20T01:06:36.564478+0000","last_fullsized":"2024-08-20T01:06:36.564478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566603+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566603+0000","last_peered":"2024-08-20T01:06:36.566603+0000","last_clean":"2024-08-20T01:06:36.566603+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566603+0000","last_undegraded":"2024-08-20T01:06:36.566603+0000","last_fullsized":"2024-08-20T01:06:36.566603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746216+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:07:34.746216+0000","last_peered":"2024-08-20T01:07:34.746216+0000","last_clean":"2024-08-20T01:07:34.746216+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:07:34.746216+0000","last_undegraded":"2024-08-20T01:07:34.746216+0000","last_fullsized":"2024-08-20T01:07:34.746216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649471+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649471+0000","last_peered":"2024-08-20T01:06:36.649471+0000","last_clean":"2024-08-20T01:06:36.649471+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649471+0000","last_undegraded":"2024-08-20T01:06:36.649471+0000","last_fullsized":"2024-08-20T01:06:36.649471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567327+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567327+0000","last_peered":"2024-08-20T01:06:36.567327+0000","last_clean":"2024-08-20T01:06:36.567327+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567327+0000","last_undegraded":"2024-08-20T01:06:36.567327+0000","last_fullsized":"2024-08-20T01:06:36.567327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730346+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:07:34.730346+0000","last_peered":"2024-08-20T01:07:34.730346+0000","last_clean":"2024-08-20T01:07:34.730346+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:07:34.730346+0000","last_undegraded":"2024-08-20T01:07:34.730346+0000","last_fullsized":"2024-08-20T01:07:34.730346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564624+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564624+0000","last_peered":"2024-08-20T01:06:36.564624+0000","last_clean":"2024-08-20T01:06:36.564624+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564624+0000","last_undegraded":"2024-08-20T01:06:36.564624+0000","last_fullsized":"2024-08-20T01:06:36.564624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733459+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:07:34.733459+0000","last_peered":"2024-08-20T01:07:34.733459+0000","last_clean":"2024-08-20T01:07:34.733459+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:07:34.733459+0000","last_undegraded":"2024-08-20T01:07:34.733459+0000","last_fullsized":"2024-08-20T01:07:34.733459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564869+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564869+0000","last_peered":"2024-08-20T01:06:36.564869+0000","last_clean":"2024-08-20T01:06:36.564869+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564869+0000","last_undegraded":"2024-08-20T01:06:36.564869+0000","last_fullsized":"2024-08-20T01:06:36.564869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739679+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:07:34.739679+0000","last_peered":"2024-08-20T01:07:34.739679+0000","last_clean":"2024-08-20T01:07:34.739679+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:07:34.739679+0000","last_undegraded":"2024-08-20T01:07:34.739679+0000","last_fullsized":"2024-08-20T01:07:34.739679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:36.101490+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:07:36.101490+0000","last_peered":"2024-08-20T01:07:36.101490+0000","last_clean":"2024-08-20T01:07:36.101490+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:07:36.101490+0000","last_undegraded":"2024-08-20T01:07:36.101490+0000","last_fullsized":"2024-08-20T01:07:36.101490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564675+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564675+0000","last_peered":"2024-08-20T01:06:36.564675+0000","last_clean":"2024-08-20T01:06:36.564675+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564675+0000","last_undegraded":"2024-08-20T01:06:36.564675+0000","last_fullsized":"2024-08-20T01:06:36.564675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741181+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:07:34.741181+0000","last_peered":"2024-08-20T01:07:34.741181+0000","last_clean":"2024-08-20T01:07:34.741181+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:07:34.741181+0000","last_undegraded":"2024-08-20T01:07:34.741181+0000","last_fullsized":"2024-08-20T01:07:34.741181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564578+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564578+0000","last_peered":"2024-08-20T01:06:36.564578+0000","last_clean":"2024-08-20T01:06:36.564578+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564578+0000","last_undegraded":"2024-08-20T01:06:36.564578+0000","last_fullsized":"2024-08-20T01:06:36.564578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620950+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620950+0000","last_peered":"2024-08-20T01:06:36.620950+0000","last_clean":"2024-08-20T01:06:36.620950+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620950+0000","last_undegraded":"2024-08-20T01:06:36.620950+0000","last_fullsized":"2024-08-20T01:06:36.620950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727978+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:07:34.727978+0000","last_peered":"2024-08-20T01:07:34.727978+0000","last_clean":"2024-08-20T01:07:34.727978+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:07:34.727978+0000","last_undegraded":"2024-08-20T01:07:34.727978+0000","last_fullsized":"2024-08-20T01:07:34.727978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642277+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642277+0000","last_peered":"2024-08-20T01:06:36.642277+0000","last_clean":"2024-08-20T01:06:36.642277+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642277+0000","last_undegraded":"2024-08-20T01:06:36.642277+0000","last_fullsized":"2024-08-20T01:06:36.642277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565004+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.565004+0000","last_peered":"2024-08-20T01:06:36.565004+0000","last_clean":"2024-08-20T01:06:36.565004+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.565004+0000","last_undegraded":"2024-08-20T01:06:36.565004+0000","last_fullsized":"2024-08-20T01:06:36.565004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738711+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:07:34.738711+0000","last_peered":"2024-08-20T01:07:34.738711+0000","last_clean":"2024-08-20T01:07:34.738711+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:07:34.738711+0000","last_undegraded":"2024-08-20T01:07:34.738711+0000","last_fullsized":"2024-08-20T01:07:34.738711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565514+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565514+0000","last_peered":"2024-08-20T01:06:36.565514+0000","last_clean":"2024-08-20T01:06:36.565514+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565514+0000","last_undegraded":"2024-08-20T01:06:36.565514+0000","last_fullsized":"2024-08-20T01:06:36.565514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567004+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.567004+0000","last_peered":"2024-08-20T01:06:36.567004+0000","last_clean":"2024-08-20T01:06:36.567004+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.567004+0000","last_undegraded":"2024-08-20T01:06:36.567004+0000","last_fullsized":"2024-08-20T01:06:36.567004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727759+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:07:34.727759+0000","last_peered":"2024-08-20T01:07:34.727759+0000","last_clean":"2024-08-20T01:07:34.727759+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:07:34.727759+0000","last_undegraded":"2024-08-20T01:07:34.727759+0000","last_fullsized":"2024-08-20T01:07:34.727759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'95","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:34.030555+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:08:34.030555+0000","last_peered":"2024-08-20T01:08:34.030555+0000","last_clean":"2024-08-20T01:08:34.030555+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:08:34.030555+0000","last_undegraded":"2024-08-20T01:08:34.030555+0000","last_fullsized":"2024-08-20T01:08:34.030555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":175,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564152+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564152+0000","last_peered":"2024-08-20T01:06:36.564152+0000","last_clean":"2024-08-20T01:06:36.564152+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564152+0000","last_undegraded":"2024-08-20T01:06:36.564152+0000","last_fullsized":"2024-08-20T01:06:36.564152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729501+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:07:34.729501+0000","last_peered":"2024-08-20T01:07:34.729501+0000","last_clean":"2024-08-20T01:07:34.729501+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:07:34.729501+0000","last_undegraded":"2024-08-20T01:07:34.729501+0000","last_fullsized":"2024-08-20T01:07:34.729501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566711+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566711+0000","last_peered":"2024-08-20T01:06:36.566711+0000","last_clean":"2024-08-20T01:06:36.566711+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566711+0000","last_undegraded":"2024-08-20T01:06:36.566711+0000","last_fullsized":"2024-08-20T01:06:36.566711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338363+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338363+0000","last_peered":"2024-08-20T01:06:37.338363+0000","last_clean":"2024-08-20T01:06:37.338363+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338363+0000","last_undegraded":"2024-08-20T01:06:37.338363+0000","last_fullsized":"2024-08-20T01:06:37.338363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732734+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:07:34.732734+0000","last_peered":"2024-08-20T01:07:34.732734+0000","last_clean":"2024-08-20T01:07:34.732734+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:07:34.732734+0000","last_undegraded":"2024-08-20T01:07:34.732734+0000","last_fullsized":"2024-08-20T01:07:34.732734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620842+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620842+0000","last_peered":"2024-08-20T01:06:36.620842+0000","last_clean":"2024-08-20T01:06:36.620842+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620842+0000","last_undegraded":"2024-08-20T01:06:36.620842+0000","last_fullsized":"2024-08-20T01:06:36.620842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566765+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566765+0000","last_peered":"2024-08-20T01:06:36.566765+0000","last_clean":"2024-08-20T01:06:36.566765+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566765+0000","last_undegraded":"2024-08-20T01:06:36.566765+0000","last_fullsized":"2024-08-20T01:06:36.566765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740243+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:07:34.740243+0000","last_peered":"2024-08-20T01:07:34.740243+0000","last_clean":"2024-08-20T01:07:34.740243+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:07:34.740243+0000","last_undegraded":"2024-08-20T01:07:34.740243+0000","last_fullsized":"2024-08-20T01:07:34.740243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900778+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900778+0000","last_peered":"2024-08-20T01:06:36.900778+0000","last_clean":"2024-08-20T01:06:36.900778+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900778+0000","last_undegraded":"2024-08-20T01:06:36.900778+0000","last_fullsized":"2024-08-20T01:06:36.900778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566915+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566915+0000","last_peered":"2024-08-20T01:06:36.566915+0000","last_clean":"2024-08-20T01:06:36.566915+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566915+0000","last_undegraded":"2024-08-20T01:06:36.566915+0000","last_fullsized":"2024-08-20T01:06:36.566915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731513+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:07:34.731513+0000","last_peered":"2024-08-20T01:07:34.731513+0000","last_clean":"2024-08-20T01:07:34.731513+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:07:34.731513+0000","last_undegraded":"2024-08-20T01:07:34.731513+0000","last_fullsized":"2024-08-20T01:07:34.731513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":181,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.825138+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:08:33.825138+0000","last_peered":"2024-08-20T01:08:33.825138+0000","last_clean":"2024-08-20T01:08:33.825138+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:08:33.825138+0000","last_undegraded":"2024-08-20T01:08:33.825138+0000","last_fullsized":"2024-08-20T01:08:33.825138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567516+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567516+0000","last_peered":"2024-08-20T01:06:36.567516+0000","last_clean":"2024-08-20T01:06:36.567516+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567516+0000","last_undegraded":"2024-08-20T01:06:36.567516+0000","last_fullsized":"2024-08-20T01:06:36.567516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727983+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:07:34.727983+0000","last_peered":"2024-08-20T01:07:34.727983+0000","last_clean":"2024-08-20T01:07:34.727983+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:07:34.727983+0000","last_undegraded":"2024-08-20T01:07:34.727983+0000","last_fullsized":"2024-08-20T01:07:34.727983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900807+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900807+0000","last_peered":"2024-08-20T01:06:36.900807+0000","last_clean":"2024-08-20T01:06:36.900807+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900807+0000","last_undegraded":"2024-08-20T01:06:36.900807+0000","last_fullsized":"2024-08-20T01:06:36.900807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647669+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647669+0000","last_peered":"2024-08-20T01:06:36.647669+0000","last_clean":"2024-08-20T01:06:36.647669+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647669+0000","last_undegraded":"2024-08-20T01:06:36.647669+0000","last_fullsized":"2024-08-20T01:06:36.647669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747444+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:07:34.747444+0000","last_peered":"2024-08-20T01:07:34.747444+0000","last_clean":"2024-08-20T01:07:34.747444+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:07:34.747444+0000","last_undegraded":"2024-08-20T01:07:34.747444+0000","last_fullsized":"2024-08-20T01:07:34.747444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565335+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565335+0000","last_peered":"2024-08-20T01:06:36.565335+0000","last_clean":"2024-08-20T01:06:36.565335+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565335+0000","last_undegraded":"2024-08-20T01:06:36.565335+0000","last_fullsized":"2024-08-20T01:06:36.565335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900855+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900855+0000","last_peered":"2024-08-20T01:06:36.900855+0000","last_clean":"2024-08-20T01:06:36.900855+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900855+0000","last_undegraded":"2024-08-20T01:06:36.900855+0000","last_fullsized":"2024-08-20T01:06:36.900855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732158+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:07:34.732158+0000","last_peered":"2024-08-20T01:07:34.732158+0000","last_clean":"2024-08-20T01:07:34.732158+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:07:34.732158+0000","last_undegraded":"2024-08-20T01:07:34.732158+0000","last_fullsized":"2024-08-20T01:07:34.732158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567812+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567812+0000","last_peered":"2024-08-20T01:06:36.567812+0000","last_clean":"2024-08-20T01:06:36.567812+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567812+0000","last_undegraded":"2024-08-20T01:06:36.567812+0000","last_fullsized":"2024-08-20T01:06:36.567812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642230+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642230+0000","last_peered":"2024-08-20T01:06:36.642230+0000","last_clean":"2024-08-20T01:06:36.642230+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642230+0000","last_undegraded":"2024-08-20T01:06:36.642230+0000","last_fullsized":"2024-08-20T01:06:36.642230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731242+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:07:34.731242+0000","last_peered":"2024-08-20T01:07:34.731242+0000","last_clean":"2024-08-20T01:07:34.731242+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:07:34.731242+0000","last_undegraded":"2024-08-20T01:07:34.731242+0000","last_fullsized":"2024-08-20T01:07:34.731242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'60","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:34.030121+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:08:34.030121+0000","last_peered":"2024-08-20T01:08:34.030121+0000","last_clean":"2024-08-20T01:08:34.030121+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:08:34.030121+0000","last_undegraded":"2024-08-20T01:08:34.030121+0000","last_fullsized":"2024-08-20T01:08:34.030121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425109+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425109+0000","last_peered":"2024-08-20T01:06:37.425109+0000","last_clean":"2024-08-20T01:06:37.425109+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425109+0000","last_undegraded":"2024-08-20T01:06:37.425109+0000","last_fullsized":"2024-08-20T01:06:37.425109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731342+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:07:34.731342+0000","last_peered":"2024-08-20T01:07:34.731342+0000","last_clean":"2024-08-20T01:07:34.731342+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:07:34.731342+0000","last_undegraded":"2024-08-20T01:07:34.731342+0000","last_fullsized":"2024-08-20T01:07:34.731342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646941+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646941+0000","last_peered":"2024-08-20T01:06:36.646941+0000","last_clean":"2024-08-20T01:06:36.646941+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646941+0000","last_undegraded":"2024-08-20T01:06:36.646941+0000","last_fullsized":"2024-08-20T01:06:36.646941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739075+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:07:34.739075+0000","last_peered":"2024-08-20T01:07:34.739075+0000","last_clean":"2024-08-20T01:07:34.739075+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:07:34.739075+0000","last_undegraded":"2024-08-20T01:07:34.739075+0000","last_fullsized":"2024-08-20T01:07:34.739075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566920+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566920+0000","last_peered":"2024-08-20T01:06:36.566920+0000","last_clean":"2024-08-20T01:06:36.566920+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566920+0000","last_undegraded":"2024-08-20T01:06:36.566920+0000","last_fullsized":"2024-08-20T01:06:36.566920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731827+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:07:34.731827+0000","last_peered":"2024-08-20T01:07:34.731827+0000","last_clean":"2024-08-20T01:07:34.731827+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:07:34.731827+0000","last_undegraded":"2024-08-20T01:07:34.731827+0000","last_fullsized":"2024-08-20T01:07:34.731827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338193+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338193+0000","last_peered":"2024-08-20T01:06:37.338193+0000","last_clean":"2024-08-20T01:06:37.338193+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338193+0000","last_undegraded":"2024-08-20T01:06:37.338193+0000","last_fullsized":"2024-08-20T01:06:37.338193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340579+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340579+0000","last_peered":"2024-08-20T01:06:37.340579+0000","last_clean":"2024-08-20T01:06:37.340579+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340579+0000","last_undegraded":"2024-08-20T01:06:37.340579+0000","last_fullsized":"2024-08-20T01:06:37.340579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724142+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:07:34.724142+0000","last_peered":"2024-08-20T01:07:34.724142+0000","last_clean":"2024-08-20T01:07:34.724142+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:07:34.724142+0000","last_undegraded":"2024-08-20T01:07:34.724142+0000","last_fullsized":"2024-08-20T01:07:34.724142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.825690+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:08:33.825690+0000","last_peered":"2024-08-20T01:08:33.825690+0000","last_clean":"2024-08-20T01:08:33.825690+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:08:33.825690+0000","last_undegraded":"2024-08-20T01:08:33.825690+0000","last_fullsized":"2024-08-20T01:08:33.825690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621011+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.621011+0000","last_peered":"2024-08-20T01:06:36.621011+0000","last_clean":"2024-08-20T01:06:36.621011+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.621011+0000","last_undegraded":"2024-08-20T01:06:36.621011+0000","last_fullsized":"2024-08-20T01:06:36.621011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739960+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:07:34.739960+0000","last_peered":"2024-08-20T01:07:34.739960+0000","last_clean":"2024-08-20T01:07:34.739960+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:07:34.739960+0000","last_undegraded":"2024-08-20T01:07:34.739960+0000","last_fullsized":"2024-08-20T01:07:34.739960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647459+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647459+0000","last_peered":"2024-08-20T01:06:36.647459+0000","last_clean":"2024-08-20T01:06:36.647459+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647459+0000","last_undegraded":"2024-08-20T01:06:36.647459+0000","last_fullsized":"2024-08-20T01:06:36.647459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338356+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338356+0000","last_peered":"2024-08-20T01:06:37.338356+0000","last_clean":"2024-08-20T01:06:37.338356+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338356+0000","last_undegraded":"2024-08-20T01:06:37.338356+0000","last_fullsized":"2024-08-20T01:06:37.338356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725099+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:07:34.725099+0000","last_peered":"2024-08-20T01:07:34.725099+0000","last_clean":"2024-08-20T01:07:34.725099+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:07:34.725099+0000","last_undegraded":"2024-08-20T01:07:34.725099+0000","last_fullsized":"2024-08-20T01:07:34.725099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566749+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566749+0000","last_peered":"2024-08-20T01:06:36.566749+0000","last_clean":"2024-08-20T01:06:36.566749+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566749+0000","last_undegraded":"2024-08-20T01:06:36.566749+0000","last_fullsized":"2024-08-20T01:06:36.566749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647424+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647424+0000","last_peered":"2024-08-20T01:06:36.647424+0000","last_clean":"2024-08-20T01:06:36.647424+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647424+0000","last_undegraded":"2024-08-20T01:06:36.647424+0000","last_fullsized":"2024-08-20T01:06:36.647424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.719431+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:07:34.719431+0000","last_peered":"2024-08-20T01:07:34.719431+0000","last_clean":"2024-08-20T01:07:34.719431+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:07:34.719431+0000","last_undegraded":"2024-08-20T01:07:34.719431+0000","last_fullsized":"2024-08-20T01:07:34.719431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'41","reported_seq":364,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.826628+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:08:33.826628+0000","last_peered":"2024-08-20T01:08:33.826628+0000","last_clean":"2024-08-20T01:08:33.826628+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:08:33.826628+0000","last_undegraded":"2024-08-20T01:08:33.826628+0000","last_fullsized":"2024-08-20T01:08:33.826628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69492,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":281,"num_read_kb":205,"num_write":42,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620266+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620266+0000","last_peered":"2024-08-20T01:06:36.620266+0000","last_clean":"2024-08-20T01:06:36.620266+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620266+0000","last_undegraded":"2024-08-20T01:06:36.620266+0000","last_fullsized":"2024-08-20T01:06:36.620266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747475+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:07:34.747475+0000","last_peered":"2024-08-20T01:07:34.747475+0000","last_clean":"2024-08-20T01:07:34.747475+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:07:34.747475+0000","last_undegraded":"2024-08-20T01:07:34.747475+0000","last_fullsized":"2024-08-20T01:07:34.747475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647695+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647695+0000","last_peered":"2024-08-20T01:06:36.647695+0000","last_clean":"2024-08-20T01:06:36.647695+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647695+0000","last_undegraded":"2024-08-20T01:06:36.647695+0000","last_fullsized":"2024-08-20T01:06:36.647695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565025+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.565025+0000","last_peered":"2024-08-20T01:06:36.565025+0000","last_clean":"2024-08-20T01:06:36.565025+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.565025+0000","last_undegraded":"2024-08-20T01:06:36.565025+0000","last_fullsized":"2024-08-20T01:06:36.565025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740128+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:07:34.740128+0000","last_peered":"2024-08-20T01:07:34.740128+0000","last_clean":"2024-08-20T01:07:34.740128+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:07:34.740128+0000","last_undegraded":"2024-08-20T01:07:34.740128+0000","last_fullsized":"2024-08-20T01:07:34.740128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901002+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.901002+0000","last_peered":"2024-08-20T01:06:36.901002+0000","last_clean":"2024-08-20T01:06:36.901002+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.901002+0000","last_undegraded":"2024-08-20T01:06:36.901002+0000","last_fullsized":"2024-08-20T01:06:36.901002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620177+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620177+0000","last_peered":"2024-08-20T01:06:36.620177+0000","last_clean":"2024-08-20T01:06:36.620177+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620177+0000","last_undegraded":"2024-08-20T01:06:36.620177+0000","last_fullsized":"2024-08-20T01:06:36.620177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746528+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:07:34.746528+0000","last_peered":"2024-08-20T01:07:34.746528+0000","last_clean":"2024-08-20T01:07:34.746528+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:07:34.746528+0000","last_undegraded":"2024-08-20T01:07:34.746528+0000","last_fullsized":"2024-08-20T01:07:34.746528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":181,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:32.824837+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:08:32.824837+0000","last_peered":"2024-08-20T01:08:32.824837+0000","last_clean":"2024-08-20T01:08:32.824837+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:08:32.824837+0000","last_undegraded":"2024-08-20T01:08:32.824837+0000","last_fullsized":"2024-08-20T01:08:32.824837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564777+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564777+0000","last_peered":"2024-08-20T01:06:36.564777+0000","last_clean":"2024-08-20T01:06:36.564777+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564777+0000","last_undegraded":"2024-08-20T01:06:36.564777+0000","last_fullsized":"2024-08-20T01:06:36.564777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425144+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425144+0000","last_peered":"2024-08-20T01:06:37.425144+0000","last_clean":"2024-08-20T01:06:37.425144+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425144+0000","last_undegraded":"2024-08-20T01:06:37.425144+0000","last_fullsized":"2024-08-20T01:06:37.425144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731657+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:07:34.731657+0000","last_peered":"2024-08-20T01:07:34.731657+0000","last_clean":"2024-08-20T01:07:34.731657+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:07:34.731657+0000","last_undegraded":"2024-08-20T01:07:34.731657+0000","last_fullsized":"2024-08-20T01:07:34.731657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647535+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647535+0000","last_peered":"2024-08-20T01:06:36.647535+0000","last_clean":"2024-08-20T01:06:36.647535+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647535+0000","last_undegraded":"2024-08-20T01:06:36.647535+0000","last_fullsized":"2024-08-20T01:06:36.647535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338530+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338530+0000","last_peered":"2024-08-20T01:06:37.338530+0000","last_clean":"2024-08-20T01:06:37.338530+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338530+0000","last_undegraded":"2024-08-20T01:06:37.338530+0000","last_fullsized":"2024-08-20T01:06:37.338530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728742+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:07:34.728742+0000","last_peered":"2024-08-20T01:07:34.728742+0000","last_clean":"2024-08-20T01:07:34.728742+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:07:34.728742+0000","last_undegraded":"2024-08-20T01:07:34.728742+0000","last_fullsized":"2024-08-20T01:07:34.728742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566516+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566516+0000","last_peered":"2024-08-20T01:06:36.566516+0000","last_clean":"2024-08-20T01:06:36.566516+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566516+0000","last_undegraded":"2024-08-20T01:06:36.566516+0000","last_fullsized":"2024-08-20T01:06:36.566516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644361+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644361+0000","last_peered":"2024-08-20T01:06:36.644361+0000","last_clean":"2024-08-20T01:06:36.644361+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644361+0000","last_undegraded":"2024-08-20T01:06:36.644361+0000","last_fullsized":"2024-08-20T01:06:36.644361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732168+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:07:34.732168+0000","last_peered":"2024-08-20T01:07:34.732168+0000","last_clean":"2024-08-20T01:07:34.732168+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:07:34.732168+0000","last_undegraded":"2024-08-20T01:07:34.732168+0000","last_fullsized":"2024-08-20T01:07:34.732168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566548+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566548+0000","last_peered":"2024-08-20T01:06:36.566548+0000","last_clean":"2024-08-20T01:06:36.566548+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566548+0000","last_undegraded":"2024-08-20T01:06:36.566548+0000","last_fullsized":"2024-08-20T01:06:36.566548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":189,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:32.825374+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:08:32.825374+0000","last_peered":"2024-08-20T01:08:32.825374+0000","last_clean":"2024-08-20T01:08:32.825374+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:08:32.825374+0000","last_undegraded":"2024-08-20T01:08:32.825374+0000","last_fullsized":"2024-08-20T01:08:32.825374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1465,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732844+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:07:34.732844+0000","last_peered":"2024-08-20T01:07:34.732844+0000","last_clean":"2024-08-20T01:07:34.732844+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:07:34.732844+0000","last_undegraded":"2024-08-20T01:07:34.732844+0000","last_fullsized":"2024-08-20T01:07:34.732844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567404+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567404+0000","last_peered":"2024-08-20T01:06:36.567404+0000","last_clean":"2024-08-20T01:06:36.567404+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567404+0000","last_undegraded":"2024-08-20T01:06:36.567404+0000","last_fullsized":"2024-08-20T01:06:36.567404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732743+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:07:34.732743+0000","last_peered":"2024-08-20T01:07:34.732743+0000","last_clean":"2024-08-20T01:07:34.732743+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:07:34.732743+0000","last_undegraded":"2024-08-20T01:07:34.732743+0000","last_fullsized":"2024-08-20T01:07:34.732743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733589+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:07:34.733589+0000","last_peered":"2024-08-20T01:07:34.733589+0000","last_clean":"2024-08-20T01:07:34.733589+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:07:34.733589+0000","last_undegraded":"2024-08-20T01:07:34.733589+0000","last_fullsized":"2024-08-20T01:07:34.733589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739063+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:07:34.739063+0000","last_peered":"2024-08-20T01:07:34.739063+0000","last_clean":"2024-08-20T01:07:34.739063+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:07:34.739063+0000","last_undegraded":"2024-08-20T01:07:34.739063+0000","last_fullsized":"2024-08-20T01:07:34.739063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739518+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:07:34.739518+0000","last_peered":"2024-08-20T01:07:34.739518+0000","last_clean":"2024-08-20T01:07:34.739518+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:07:34.739518+0000","last_undegraded":"2024-08-20T01:07:34.739518+0000","last_fullsized":"2024-08-20T01:07:34.739518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740881+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:07:34.740881+0000","last_peered":"2024-08-20T01:07:34.740881+0000","last_clean":"2024-08-20T01:07:34.740881+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:07:34.740881+0000","last_undegraded":"2024-08-20T01:07:34.740881+0000","last_fullsized":"2024-08-20T01:07:34.740881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729504+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:07:34.729504+0000","last_peered":"2024-08-20T01:07:34.729504+0000","last_clean":"2024-08-20T01:07:34.729504+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:07:34.729504+0000","last_undegraded":"2024-08-20T01:07:34.729504+0000","last_fullsized":"2024-08-20T01:07:34.729504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725362+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:07:34.725362+0000","last_peered":"2024-08-20T01:07:34.725362+0000","last_clean":"2024-08-20T01:07:34.725362+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:07:34.725362+0000","last_undegraded":"2024-08-20T01:07:34.725362+0000","last_fullsized":"2024-08-20T01:07:34.725362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728179+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:07:34.728179+0000","last_peered":"2024-08-20T01:07:34.728179+0000","last_clean":"2024-08-20T01:07:34.728179+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:07:34.728179+0000","last_undegraded":"2024-08-20T01:07:34.728179+0000","last_fullsized":"2024-08-20T01:07:34.728179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732606+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:07:34.732606+0000","last_peered":"2024-08-20T01:07:34.732606+0000","last_clean":"2024-08-20T01:07:34.732606+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:07:34.732606+0000","last_undegraded":"2024-08-20T01:07:34.732606+0000","last_fullsized":"2024-08-20T01:07:34.732606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729298+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:07:34.729298+0000","last_peered":"2024-08-20T01:07:34.729298+0000","last_clean":"2024-08-20T01:07:34.729298+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:07:34.729298+0000","last_undegraded":"2024-08-20T01:07:34.729298+0000","last_fullsized":"2024-08-20T01:07:34.729298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747462+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:07:34.747462+0000","last_peered":"2024-08-20T01:07:34.747462+0000","last_clean":"2024-08-20T01:07:34.747462+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:07:34.747462+0000","last_undegraded":"2024-08-20T01:07:34.747462+0000","last_fullsized":"2024-08-20T01:07:34.747462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741436+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:07:34.741436+0000","last_peered":"2024-08-20T01:07:34.741436+0000","last_clean":"2024-08-20T01:07:34.741436+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:07:34.741436+0000","last_undegraded":"2024-08-20T01:07:34.741436+0000","last_fullsized":"2024-08-20T01:07:34.741436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731490+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:07:34.731490+0000","last_peered":"2024-08-20T01:07:34.731490+0000","last_clean":"2024-08-20T01:07:34.731490+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:07:34.731490+0000","last_undegraded":"2024-08-20T01:07:34.731490+0000","last_fullsized":"2024-08-20T01:07:34.731490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732616+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:07:34.732616+0000","last_peered":"2024-08-20T01:07:34.732616+0000","last_clean":"2024-08-20T01:07:34.732616+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:07:34.732616+0000","last_undegraded":"2024-08-20T01:07:34.732616+0000","last_fullsized":"2024-08-20T01:07:34.732616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.735563+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:07:34.735563+0000","last_peered":"2024-08-20T01:07:34.735563+0000","last_clean":"2024-08-20T01:07:34.735563+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:07:34.735563+0000","last_undegraded":"2024-08-20T01:07:34.735563+0000","last_fullsized":"2024-08-20T01:07:34.735563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742307+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:07:34.742307+0000","last_peered":"2024-08-20T01:07:34.742307+0000","last_clean":"2024-08-20T01:07:34.742307+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:07:34.742307+0000","last_undegraded":"2024-08-20T01:07:34.742307+0000","last_fullsized":"2024-08-20T01:07:34.742307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747020+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:07:34.747020+0000","last_peered":"2024-08-20T01:07:34.747020+0000","last_clean":"2024-08-20T01:07:34.747020+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:07:34.747020+0000","last_undegraded":"2024-08-20T01:07:34.747020+0000","last_fullsized":"2024-08-20T01:07:34.747020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740301+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:07:34.740301+0000","last_peered":"2024-08-20T01:07:34.740301+0000","last_clean":"2024-08-20T01:07:34.740301+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:07:34.740301+0000","last_undegraded":"2024-08-20T01:07:34.740301+0000","last_fullsized":"2024-08-20T01:07:34.740301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740343+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:07:34.740343+0000","last_peered":"2024-08-20T01:07:34.740343+0000","last_clean":"2024-08-20T01:07:34.740343+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:07:34.740343+0000","last_undegraded":"2024-08-20T01:07:34.740343+0000","last_fullsized":"2024-08-20T01:07:34.740343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738693+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:07:34.738693+0000","last_peered":"2024-08-20T01:07:34.738693+0000","last_clean":"2024-08-20T01:07:34.738693+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:07:34.738693+0000","last_undegraded":"2024-08-20T01:07:34.738693+0000","last_fullsized":"2024-08-20T01:07:34.738693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.736334+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:07:34.736334+0000","last_peered":"2024-08-20T01:07:34.736334+0000","last_clean":"2024-08-20T01:07:34.736334+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:07:34.736334+0000","last_undegraded":"2024-08-20T01:07:34.736334+0000","last_fullsized":"2024-08-20T01:07:34.736334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731015+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:07:34.731015+0000","last_peered":"2024-08-20T01:07:34.731015+0000","last_clean":"2024-08-20T01:07:34.731015+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:07:34.731015+0000","last_undegraded":"2024-08-20T01:07:34.731015+0000","last_fullsized":"2024-08-20T01:07:34.731015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740057+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:07:34.740057+0000","last_peered":"2024-08-20T01:07:34.740057+0000","last_clean":"2024-08-20T01:07:34.740057+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:07:34.740057+0000","last_undegraded":"2024-08-20T01:07:34.740057+0000","last_fullsized":"2024-08-20T01:07:34.740057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.744507+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:07:34.744507+0000","last_peered":"2024-08-20T01:07:34.744507+0000","last_clean":"2024-08-20T01:07:34.744507+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:07:34.744507+0000","last_undegraded":"2024-08-20T01:07:34.744507+0000","last_fullsized":"2024-08-20T01:07:34.744507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734496+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:07:34.734496+0000","last_peered":"2024-08-20T01:07:34.734496+0000","last_clean":"2024-08-20T01:07:34.734496+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:07:34.734496+0000","last_undegraded":"2024-08-20T01:07:34.734496+0000","last_fullsized":"2024-08-20T01:07:34.734496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730134+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:07:34.730134+0000","last_peered":"2024-08-20T01:07:34.730134+0000","last_clean":"2024-08-20T01:07:34.730134+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:07:34.730134+0000","last_undegraded":"2024-08-20T01:07:34.730134+0000","last_fullsized":"2024-08-20T01:07:34.730134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740683+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:07:34.740683+0000","last_peered":"2024-08-20T01:07:34.740683+0000","last_clean":"2024-08-20T01:07:34.740683+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:07:34.740683+0000","last_undegraded":"2024-08-20T01:07:34.740683+0000","last_fullsized":"2024-08-20T01:07:34.740683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734991+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:07:34.734991+0000","last_peered":"2024-08-20T01:07:34.734991+0000","last_clean":"2024-08-20T01:07:34.734991+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:07:34.734991+0000","last_undegraded":"2024-08-20T01:07:34.734991+0000","last_fullsized":"2024-08-20T01:07:34.734991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733118+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:07:34.733118+0000","last_peered":"2024-08-20T01:07:34.733118+0000","last_clean":"2024-08-20T01:07:34.733118+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:07:34.733118+0000","last_undegraded":"2024-08-20T01:07:34.733118+0000","last_fullsized":"2024-08-20T01:07:34.733118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747505+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:07:34.747505+0000","last_peered":"2024-08-20T01:07:34.747505+0000","last_clean":"2024-08-20T01:07:34.747505+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:07:34.747505+0000","last_undegraded":"2024-08-20T01:07:34.747505+0000","last_fullsized":"2024-08-20T01:07:34.747505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739436+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:07:34.739436+0000","last_peered":"2024-08-20T01:07:34.739436+0000","last_clean":"2024-08-20T01:07:34.739436+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:07:34.739436+0000","last_undegraded":"2024-08-20T01:07:34.739436+0000","last_fullsized":"2024-08-20T01:07:34.739436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742422+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:07:34.742422+0000","last_peered":"2024-08-20T01:07:34.742422+0000","last_clean":"2024-08-20T01:07:34.742422+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:07:34.742422+0000","last_undegraded":"2024-08-20T01:07:34.742422+0000","last_fullsized":"2024-08-20T01:07:34.742422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":860937,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1164,"num_read_kb":785,"num_write":406,"num_write_kb":2225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":647168,"data_stored":2185281,"data_compressed":92851,"data_compressed_allocated":122880,"data_compressed_original":1892352,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678068,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874297,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1171456,"data_stored":1563402,"data_compressed":80566,"data_compressed_allocated":106496,"data_compressed_original":1032192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":914728,"data_compressed":45926,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234051,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517224,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1113697,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":1503061,"data_compressed":80566,"data_compressed_allocated":106496,"data_compressed_original":1032192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":1299778,"data_compressed":64017,"data_compressed_allocated":86016,"data_compressed_original":835584,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378724,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":584187,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542247,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1036288,"data_stored":577680,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":396095,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":464846,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:08:39.000 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-08-20T01:08:39.000 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph pg dump --format=json 2024-08-20T01:08:39.372 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:39.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:39 smithi138 ceph-mon[18076]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:39 smithi138 ceph-mon[18076]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-08-20T01:08:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 -- 172.21.15.138:0/2598469714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 msgr2=0x7f67bc1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:39.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 --2- 172.21.15.138:0/2598469714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1116a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009ca0 tx=0x7f67b402a8b0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 -- 172.21.15.138:0/2598469714 shutdown_connections 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 --2- 172.21.15.138:0/2598469714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1116a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 --2- 172.21.15.138:0/2598469714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc10a520 0x7f67bc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 --2- 172.21.15.138:0/2598469714 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc104b10 0x7f67bc104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 -- 172.21.15.138:0/2598469714 >> 172.21.15.138:0/2598469714 conn(0x7f67bc100540 msgr2=0x7f67bc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 -- 172.21.15.138:0/2598469714 shutdown_connections 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.950+0000 7f67bb200640 1 -- 172.21.15.138:0/2598469714 wait complete. 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.951+0000 7f67bb200640 1 Processor -- start 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.951+0000 7f67bb200640 1 -- start start 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc10a520 0x7f67bc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:39.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc114b70 con 0x7f67bc104b10 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc1149f0 con 0x7f67bc10aea0 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67bb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67bc114cf0 con 0x7f67bc10a520 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36314/0 (socket says 172.21.15.138:36314) 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 -- 172.21.15.138:0/4139003966 learned_addr learned my addr 172.21.15.138:0/4139003966 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67ba800640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9400640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc10a520 0x7f67bc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 -- 172.21.15.138:0/4139003966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc10a520 msgr2=0x7f67bc19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc10a520 0x7f67bc19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 -- 172.21.15.138:0/4139003966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 msgr2=0x7f67bc1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:39.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67b9e00640 1 -- 172.21.15.138:0/4139003966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67bc1a0f80 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.952+0000 7f67ba800640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67b9e00640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac00cd80 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67bc1a1270 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67ac0108e0 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac018980 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.953+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67bc1a7b00 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.954+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67bc104ef0 con 0x7f67bc104b10 2024-08-20T01:08:39.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.956+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f67ac010430 con 0x7f67bc104b10 2024-08-20T01:08:39.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.958+0000 7f67b2a00640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 0x7f67a00b6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:39.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.958+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f67ac0e4a90 con 0x7f67bc104b10 2024-08-20T01:08:39.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.959+0000 7f67b9400640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 0x7f67a00b6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:39.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.960+0000 7f67b9400640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 0x7f67a00b6220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f67bc102e80 tx=0x7f67a8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:39.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:39.960+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f67ac0ab460 con 0x7f67bc104b10 2024-08-20T01:08:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:39 smithi167 ceph-mon[21965]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:39 smithi167 ceph-mon[21965]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-08-20T01:08:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:39 smithi142 ceph-mon[21799]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:39 smithi142 ceph-mon[21799]: pgmap v378: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 51 op/s 2024-08-20T01:08:40.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.144+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67bc1012d0 con 0x7f67a00b3d60 2024-08-20T01:08:40.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.150+0000 7f67b2a00640 1 -- 172.21.15.138:0/4139003966 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340861 (secure 0 0 0) 0x7f67bc1012d0 con 0x7f67a00b3d60 2024-08-20T01:08:40.152 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:40.153 INFO:teuthology.orchestra.run.smithi138.stderr:dumped all 2024-08-20T01:08:40.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 msgr2=0x7f67a00b6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:40.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 0x7f67a00b6220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f67bc102e80 tx=0x7f67a8005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:40.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 msgr2=0x7f67bc19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f67ac00e9f0 tx=0x7f67ac00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 shutdown_connections 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67a00b3d60 0x7f67a00b6220 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67bc10aea0 0x7f67bc1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67bc10a520 0x7f67bc19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 --2- 172.21.15.138:0/4139003966 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67bc104b10 0x7f67bc19aed0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.154+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 >> 172.21.15.138:0/4139003966 conn(0x7f67bc100540 msgr2=0x7f67bc100bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.155+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 shutdown_connections 2024-08-20T01:08:40.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:40.155+0000 7f67bb200640 1 -- 172.21.15.138:0/4139003966 wait complete. 2024-08-20T01:08:40.930 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:40 smithi138 ceph-mon[18076]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:40.930 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:40 smithi138 ceph-mon[18076]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-08-20T01:08:40.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:40 smithi167 ceph-mon[21965]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:40 smithi167 ceph-mon[21965]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-08-20T01:08:40.966 INFO:teuthology.orchestra.run.smithi138.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-08-20T01:08:38.524647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1451539,"num_objects":66,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1245,"num_read_kb":854,"num_write":531,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":893,"ondisk_log_size":893,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":353060,"kb_used_data":12112,"kb_used_omap":18,"kb_used_meta":340717,"kb_avail":1124588764,"statfs":{"total":1151940427776,"available":1151578894336,"internally_reserved":0,"allocated":12402688,"data_stored":10532393,"data_compressed":539298,"data_compressed_allocated":724992,"data_compressed_original":4939776,"omap_allocated":19142,"internal_metadata":348894522},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":791928,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":386,"num_read_kb":322,"num_write":229,"num_write_kb":2099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003486"},"pg_stats":[{"pgid":"2.1f","version":"86'9","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621129+0000","last_change":"2024-08-20T01:05:23.569825+0000","last_active":"2024-08-20T01:06:36.621129+0000","last_peered":"2024-08-20T01:06:36.621129+0000","last_clean":"2024-08-20T01:06:36.621129+0000","last_became_active":"2024-08-20T01:05:23.569438+0000","last_became_peered":"2024-08-20T01:05:23.569438+0000","last_unstale":"2024-08-20T01:06:36.621129+0000","last_undegraded":"2024-08-20T01:06:36.621129+0000","last_fullsized":"2024-08-20T01:06:36.621129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:01:06.845447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731638+0000","last_change":"2024-08-20T01:05:26.605863+0000","last_active":"2024-08-20T01:07:34.731638+0000","last_peered":"2024-08-20T01:07:34.731638+0000","last_clean":"2024-08-20T01:07:34.731638+0000","last_became_active":"2024-08-20T01:05:26.605130+0000","last_became_peered":"2024-08-20T01:05:26.605130+0000","last_unstale":"2024-08-20T01:07:34.731638+0000","last_undegraded":"2024-08-20T01:07:34.731638+0000","last_fullsized":"2024-08-20T01:07:34.731638+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:44.921317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.427003+0000","last_change":"2024-08-20T01:06:37.426479+0000","last_active":"2024-08-20T01:06:37.427003+0000","last_peered":"2024-08-20T01:06:37.427003+0000","last_clean":"2024-08-20T01:06:37.427003+0000","last_became_active":"2024-08-20T01:06:37.425924+0000","last_became_peered":"2024-08-20T01:06:37.425924+0000","last_unstale":"2024-08-20T01:06:37.427003+0000","last_undegraded":"2024-08-20T01:06:37.427003+0000","last_fullsized":"2024-08-20T01:06:37.427003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:03:44.398151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8,0],"acting":[2,3,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564918+0000","last_change":"2024-08-20T01:05:23.578535+0000","last_active":"2024-08-20T01:06:36.564918+0000","last_peered":"2024-08-20T01:06:36.564918+0000","last_clean":"2024-08-20T01:06:36.564918+0000","last_became_active":"2024-08-20T01:05:23.578162+0000","last_became_peered":"2024-08-20T01:05:23.578162+0000","last_unstale":"2024-08-20T01:06:36.564918+0000","last_undegraded":"2024-08-20T01:06:36.564918+0000","last_fullsized":"2024-08-20T01:06:36.564918+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:13:29.927317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747371+0000","last_change":"2024-08-20T01:05:26.613222+0000","last_active":"2024-08-20T01:07:34.747371+0000","last_peered":"2024-08-20T01:07:34.747371+0000","last_clean":"2024-08-20T01:07:34.747371+0000","last_became_active":"2024-08-20T01:05:26.612938+0000","last_became_peered":"2024-08-20T01:05:26.612938+0000","last_unstale":"2024-08-20T01:07:34.747371+0000","last_undegraded":"2024-08-20T01:07:34.747371+0000","last_fullsized":"2024-08-20T01:07:34.747371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:12:57.916735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564478+0000","last_change":"2024-08-20T01:05:35.967239+0000","last_active":"2024-08-20T01:06:36.564478+0000","last_peered":"2024-08-20T01:06:36.564478+0000","last_clean":"2024-08-20T01:06:36.564478+0000","last_became_active":"2024-08-20T01:05:35.966870+0000","last_became_peered":"2024-08-20T01:05:35.966870+0000","last_unstale":"2024-08-20T01:06:36.564478+0000","last_undegraded":"2024-08-20T01:06:36.564478+0000","last_fullsized":"2024-08-20T01:06:36.564478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:23:27.450120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566603+0000","last_change":"2024-08-20T01:05:23.573308+0000","last_active":"2024-08-20T01:06:36.566603+0000","last_peered":"2024-08-20T01:06:36.566603+0000","last_clean":"2024-08-20T01:06:36.566603+0000","last_became_active":"2024-08-20T01:05:23.572940+0000","last_became_peered":"2024-08-20T01:05:23.572940+0000","last_unstale":"2024-08-20T01:06:36.566603+0000","last_undegraded":"2024-08-20T01:06:36.566603+0000","last_fullsized":"2024-08-20T01:06:36.566603+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:48.426246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746216+0000","last_change":"2024-08-20T01:05:26.615821+0000","last_active":"2024-08-20T01:07:34.746216+0000","last_peered":"2024-08-20T01:07:34.746216+0000","last_clean":"2024-08-20T01:07:34.746216+0000","last_became_active":"2024-08-20T01:05:26.615468+0000","last_became_peered":"2024-08-20T01:05:26.615468+0000","last_unstale":"2024-08-20T01:07:34.746216+0000","last_undegraded":"2024-08-20T01:07:34.746216+0000","last_fullsized":"2024-08-20T01:07:34.746216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:44:12.138929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.649471+0000","last_change":"2024-08-20T01:06:36.649054+0000","last_active":"2024-08-20T01:06:36.649471+0000","last_peered":"2024-08-20T01:06:36.649471+0000","last_clean":"2024-08-20T01:06:36.649471+0000","last_became_active":"2024-08-20T01:06:36.648522+0000","last_became_peered":"2024-08-20T01:06:36.648522+0000","last_unstale":"2024-08-20T01:06:36.649471+0000","last_undegraded":"2024-08-20T01:06:36.649471+0000","last_fullsized":"2024-08-20T01:06:36.649471+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:14:48.200039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,10,5],"acting":[4,3,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567327+0000","last_change":"2024-08-20T01:05:23.570809+0000","last_active":"2024-08-20T01:06:36.567327+0000","last_peered":"2024-08-20T01:06:36.567327+0000","last_clean":"2024-08-20T01:06:36.567327+0000","last_became_active":"2024-08-20T01:05:23.570507+0000","last_became_peered":"2024-08-20T01:05:23.570507+0000","last_unstale":"2024-08-20T01:06:36.567327+0000","last_undegraded":"2024-08-20T01:06:36.567327+0000","last_fullsized":"2024-08-20T01:06:36.567327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:17:14.103206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730346+0000","last_change":"2024-08-20T01:05:26.612212+0000","last_active":"2024-08-20T01:07:34.730346+0000","last_peered":"2024-08-20T01:07:34.730346+0000","last_clean":"2024-08-20T01:07:34.730346+0000","last_became_active":"2024-08-20T01:05:26.611894+0000","last_became_peered":"2024-08-20T01:05:26.611894+0000","last_unstale":"2024-08-20T01:07:34.730346+0000","last_undegraded":"2024-08-20T01:07:34.730346+0000","last_fullsized":"2024-08-20T01:07:34.730346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:18.951247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564624+0000","last_change":"2024-08-20T01:05:35.962415+0000","last_active":"2024-08-20T01:06:36.564624+0000","last_peered":"2024-08-20T01:06:36.564624+0000","last_clean":"2024-08-20T01:06:36.564624+0000","last_became_active":"2024-08-20T01:05:35.961914+0000","last_became_peered":"2024-08-20T01:05:35.961914+0000","last_unstale":"2024-08-20T01:06:36.564624+0000","last_undegraded":"2024-08-20T01:06:36.564624+0000","last_fullsized":"2024-08-20T01:06:36.564624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:06:03.023955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733459+0000","last_change":"2024-08-20T01:05:26.611497+0000","last_active":"2024-08-20T01:07:34.733459+0000","last_peered":"2024-08-20T01:07:34.733459+0000","last_clean":"2024-08-20T01:07:34.733459+0000","last_became_active":"2024-08-20T01:05:26.611194+0000","last_became_peered":"2024-08-20T01:05:26.611194+0000","last_unstale":"2024-08-20T01:07:34.733459+0000","last_undegraded":"2024-08-20T01:07:34.733459+0000","last_fullsized":"2024-08-20T01:07:34.733459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:16.886263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":58,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:35.966353+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:35.958839+0000","last_became_peered":"2024-08-20T01:05:35.958839+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:00:23.568424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564869+0000","last_change":"2024-08-20T01:05:35.990322+0000","last_active":"2024-08-20T01:06:36.564869+0000","last_peered":"2024-08-20T01:06:36.564869+0000","last_clean":"2024-08-20T01:06:36.564869+0000","last_became_active":"2024-08-20T01:05:35.989880+0000","last_became_peered":"2024-08-20T01:05:35.989880+0000","last_unstale":"2024-08-20T01:06:36.564869+0000","last_undegraded":"2024-08-20T01:06:36.564869+0000","last_fullsized":"2024-08-20T01:06:36.564869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:21:42.078700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739679+0000","last_change":"2024-08-20T01:05:26.621990+0000","last_active":"2024-08-20T01:07:34.739679+0000","last_peered":"2024-08-20T01:07:34.739679+0000","last_clean":"2024-08-20T01:07:34.739679+0000","last_became_active":"2024-08-20T01:05:26.620774+0000","last_became_peered":"2024-08-20T01:05:26.620774+0000","last_unstale":"2024-08-20T01:07:34.739679+0000","last_undegraded":"2024-08-20T01:07:34.739679+0000","last_fullsized":"2024-08-20T01:07:34.739679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:07:15.919208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:36.101490+0000","last_change":"2024-08-20T01:05:23.570602+0000","last_active":"2024-08-20T01:07:36.101490+0000","last_peered":"2024-08-20T01:07:36.101490+0000","last_clean":"2024-08-20T01:07:36.101490+0000","last_became_active":"2024-08-20T01:05:23.570004+0000","last_became_peered":"2024-08-20T01:05:23.570004+0000","last_unstale":"2024-08-20T01:07:36.101490+0000","last_undegraded":"2024-08-20T01:07:36.101490+0000","last_fullsized":"2024-08-20T01:07:36.101490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:52:46.091265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564675+0000","last_change":"2024-08-20T01:05:35.961394+0000","last_active":"2024-08-20T01:06:36.564675+0000","last_peered":"2024-08-20T01:06:36.564675+0000","last_clean":"2024-08-20T01:06:36.564675+0000","last_became_active":"2024-08-20T01:05:35.960587+0000","last_became_peered":"2024-08-20T01:05:35.960587+0000","last_unstale":"2024-08-20T01:06:36.564675+0000","last_undegraded":"2024-08-20T01:06:36.564675+0000","last_fullsized":"2024-08-20T01:06:36.564675+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:32:20.597277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741181+0000","last_change":"2024-08-20T01:05:26.618516+0000","last_active":"2024-08-20T01:07:34.741181+0000","last_peered":"2024-08-20T01:07:34.741181+0000","last_clean":"2024-08-20T01:07:34.741181+0000","last_became_active":"2024-08-20T01:05:26.618104+0000","last_became_peered":"2024-08-20T01:05:26.618104+0000","last_unstale":"2024-08-20T01:07:34.741181+0000","last_undegraded":"2024-08-20T01:07:34.741181+0000","last_fullsized":"2024-08-20T01:07:34.741181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:10:41.508883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564578+0000","last_change":"2024-08-20T01:05:23.563634+0000","last_active":"2024-08-20T01:06:36.564578+0000","last_peered":"2024-08-20T01:06:36.564578+0000","last_clean":"2024-08-20T01:06:36.564578+0000","last_became_active":"2024-08-20T01:05:23.563274+0000","last_became_peered":"2024-08-20T01:05:23.563274+0000","last_unstale":"2024-08-20T01:06:36.564578+0000","last_undegraded":"2024-08-20T01:06:36.564578+0000","last_fullsized":"2024-08-20T01:06:36.564578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:08.948096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620950+0000","last_change":"2024-08-20T01:05:35.971117+0000","last_active":"2024-08-20T01:06:36.620950+0000","last_peered":"2024-08-20T01:06:36.620950+0000","last_clean":"2024-08-20T01:06:36.620950+0000","last_became_active":"2024-08-20T01:05:35.970587+0000","last_became_peered":"2024-08-20T01:05:35.970587+0000","last_unstale":"2024-08-20T01:06:36.620950+0000","last_undegraded":"2024-08-20T01:06:36.620950+0000","last_fullsized":"2024-08-20T01:06:36.620950+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:20:00.297238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727978+0000","last_change":"2024-08-20T01:05:26.615993+0000","last_active":"2024-08-20T01:07:34.727978+0000","last_peered":"2024-08-20T01:07:34.727978+0000","last_clean":"2024-08-20T01:07:34.727978+0000","last_became_active":"2024-08-20T01:05:26.615671+0000","last_became_peered":"2024-08-20T01:05:26.615671+0000","last_unstale":"2024-08-20T01:07:34.727978+0000","last_undegraded":"2024-08-20T01:07:34.727978+0000","last_fullsized":"2024-08-20T01:07:34.727978+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:24:41.407661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642277+0000","last_change":"2024-08-20T01:05:35.968932+0000","last_active":"2024-08-20T01:06:36.642277+0000","last_peered":"2024-08-20T01:06:36.642277+0000","last_clean":"2024-08-20T01:06:36.642277+0000","last_became_active":"2024-08-20T01:05:35.968584+0000","last_became_peered":"2024-08-20T01:05:35.968584+0000","last_unstale":"2024-08-20T01:06:36.642277+0000","last_undegraded":"2024-08-20T01:06:36.642277+0000","last_fullsized":"2024-08-20T01:06:36.642277+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:54:05.593708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565004+0000","last_change":"2024-08-20T01:05:35.991726+0000","last_active":"2024-08-20T01:06:36.565004+0000","last_peered":"2024-08-20T01:06:36.565004+0000","last_clean":"2024-08-20T01:06:36.565004+0000","last_became_active":"2024-08-20T01:05:35.991326+0000","last_became_peered":"2024-08-20T01:05:35.991326+0000","last_unstale":"2024-08-20T01:06:36.565004+0000","last_undegraded":"2024-08-20T01:06:36.565004+0000","last_fullsized":"2024-08-20T01:06:36.565004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:28:13.310546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738711+0000","last_change":"2024-08-20T01:05:26.615815+0000","last_active":"2024-08-20T01:07:34.738711+0000","last_peered":"2024-08-20T01:07:34.738711+0000","last_clean":"2024-08-20T01:07:34.738711+0000","last_became_active":"2024-08-20T01:05:26.615191+0000","last_became_peered":"2024-08-20T01:05:26.615191+0000","last_unstale":"2024-08-20T01:07:34.738711+0000","last_undegraded":"2024-08-20T01:07:34.738711+0000","last_fullsized":"2024-08-20T01:07:34.738711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:43:50.345210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565514+0000","last_change":"2024-08-20T01:05:23.563144+0000","last_active":"2024-08-20T01:06:36.565514+0000","last_peered":"2024-08-20T01:06:36.565514+0000","last_clean":"2024-08-20T01:06:36.565514+0000","last_became_active":"2024-08-20T01:05:23.562807+0000","last_became_peered":"2024-08-20T01:05:23.562807+0000","last_unstale":"2024-08-20T01:06:36.565514+0000","last_undegraded":"2024-08-20T01:06:36.565514+0000","last_fullsized":"2024-08-20T01:06:36.565514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:46:52.804347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567004+0000","last_change":"2024-08-20T01:05:35.962921+0000","last_active":"2024-08-20T01:06:36.567004+0000","last_peered":"2024-08-20T01:06:36.567004+0000","last_clean":"2024-08-20T01:06:36.567004+0000","last_became_active":"2024-08-20T01:05:35.962529+0000","last_became_peered":"2024-08-20T01:05:35.962529+0000","last_unstale":"2024-08-20T01:06:36.567004+0000","last_undegraded":"2024-08-20T01:06:36.567004+0000","last_fullsized":"2024-08-20T01:06:36.567004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:06:09.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727759+0000","last_change":"2024-08-20T01:05:26.610334+0000","last_active":"2024-08-20T01:07:34.727759+0000","last_peered":"2024-08-20T01:07:34.727759+0000","last_clean":"2024-08-20T01:07:34.727759+0000","last_became_active":"2024-08-20T01:05:26.610052+0000","last_became_peered":"2024-08-20T01:05:26.610052+0000","last_unstale":"2024-08-20T01:07:34.727759+0000","last_undegraded":"2024-08-20T01:07:34.727759+0000","last_fullsized":"2024-08-20T01:07:34.727759+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:51:45.084506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'95","reported_seq":223,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:34.030555+0000","last_change":"2024-08-20T01:05:23.570052+0000","last_active":"2024-08-20T01:08:34.030555+0000","last_peered":"2024-08-20T01:08:34.030555+0000","last_clean":"2024-08-20T01:08:34.030555+0000","last_became_active":"2024-08-20T01:05:23.569479+0000","last_became_peered":"2024-08-20T01:05:23.569479+0000","last_unstale":"2024-08-20T01:08:34.030555+0000","last_undegraded":"2024-08-20T01:08:34.030555+0000","last_fullsized":"2024-08-20T01:08:34.030555+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:29:48.431937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":175,"num_write_kb":1222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564152+0000","last_change":"2024-08-20T01:05:35.990031+0000","last_active":"2024-08-20T01:06:36.564152+0000","last_peered":"2024-08-20T01:06:36.564152+0000","last_clean":"2024-08-20T01:06:36.564152+0000","last_became_active":"2024-08-20T01:05:35.989643+0000","last_became_peered":"2024-08-20T01:05:35.989643+0000","last_unstale":"2024-08-20T01:06:36.564152+0000","last_undegraded":"2024-08-20T01:06:36.564152+0000","last_fullsized":"2024-08-20T01:06:36.564152+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:31:48.342273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729501+0000","last_change":"2024-08-20T01:05:26.610095+0000","last_active":"2024-08-20T01:07:34.729501+0000","last_peered":"2024-08-20T01:07:34.729501+0000","last_clean":"2024-08-20T01:07:34.729501+0000","last_became_active":"2024-08-20T01:05:26.607952+0000","last_became_peered":"2024-08-20T01:05:26.607952+0000","last_unstale":"2024-08-20T01:07:34.729501+0000","last_undegraded":"2024-08-20T01:07:34.729501+0000","last_fullsized":"2024-08-20T01:07:34.729501+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:46:40.699020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566711+0000","last_change":"2024-08-20T01:05:23.577960+0000","last_active":"2024-08-20T01:06:36.566711+0000","last_peered":"2024-08-20T01:06:36.566711+0000","last_clean":"2024-08-20T01:06:36.566711+0000","last_became_active":"2024-08-20T01:05:23.577343+0000","last_became_peered":"2024-08-20T01:05:23.577343+0000","last_unstale":"2024-08-20T01:06:36.566711+0000","last_undegraded":"2024-08-20T01:06:36.566711+0000","last_fullsized":"2024-08-20T01:06:36.566711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:46:33.504360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338363+0000","last_change":"2024-08-20T01:05:35.964214+0000","last_active":"2024-08-20T01:06:37.338363+0000","last_peered":"2024-08-20T01:06:37.338363+0000","last_clean":"2024-08-20T01:06:37.338363+0000","last_became_active":"2024-08-20T01:05:35.963807+0000","last_became_peered":"2024-08-20T01:05:35.963807+0000","last_unstale":"2024-08-20T01:06:37.338363+0000","last_undegraded":"2024-08-20T01:06:37.338363+0000","last_fullsized":"2024-08-20T01:06:37.338363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:11:44.482625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732734+0000","last_change":"2024-08-20T01:05:26.599043+0000","last_active":"2024-08-20T01:07:34.732734+0000","last_peered":"2024-08-20T01:07:34.732734+0000","last_clean":"2024-08-20T01:07:34.732734+0000","last_became_active":"2024-08-20T01:05:26.598682+0000","last_became_peered":"2024-08-20T01:05:26.598682+0000","last_unstale":"2024-08-20T01:07:34.732734+0000","last_undegraded":"2024-08-20T01:07:34.732734+0000","last_fullsized":"2024-08-20T01:07:34.732734+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:43:25.627908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620842+0000","last_change":"2024-08-20T01:05:23.580006+0000","last_active":"2024-08-20T01:06:36.620842+0000","last_peered":"2024-08-20T01:06:36.620842+0000","last_clean":"2024-08-20T01:06:36.620842+0000","last_became_active":"2024-08-20T01:05:23.577635+0000","last_became_peered":"2024-08-20T01:05:23.577635+0000","last_unstale":"2024-08-20T01:06:36.620842+0000","last_undegraded":"2024-08-20T01:06:36.620842+0000","last_fullsized":"2024-08-20T01:06:36.620842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:27:07.551852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566765+0000","last_change":"2024-08-20T01:05:35.990478+0000","last_active":"2024-08-20T01:06:36.566765+0000","last_peered":"2024-08-20T01:06:36.566765+0000","last_clean":"2024-08-20T01:06:36.566765+0000","last_became_active":"2024-08-20T01:05:35.989998+0000","last_became_peered":"2024-08-20T01:05:35.989998+0000","last_unstale":"2024-08-20T01:06:36.566765+0000","last_undegraded":"2024-08-20T01:06:36.566765+0000","last_fullsized":"2024-08-20T01:06:36.566765+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:27:03.069154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740243+0000","last_change":"2024-08-20T01:05:26.622792+0000","last_active":"2024-08-20T01:07:34.740243+0000","last_peered":"2024-08-20T01:07:34.740243+0000","last_clean":"2024-08-20T01:07:34.740243+0000","last_became_active":"2024-08-20T01:05:26.622230+0000","last_became_peered":"2024-08-20T01:05:26.622230+0000","last_unstale":"2024-08-20T01:07:34.740243+0000","last_undegraded":"2024-08-20T01:07:34.740243+0000","last_fullsized":"2024-08-20T01:07:34.740243+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:52:39.146830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900778+0000","last_change":"2024-08-20T01:05:23.568228+0000","last_active":"2024-08-20T01:06:36.900778+0000","last_peered":"2024-08-20T01:06:36.900778+0000","last_clean":"2024-08-20T01:06:36.900778+0000","last_became_active":"2024-08-20T01:05:23.567802+0000","last_became_peered":"2024-08-20T01:05:23.567802+0000","last_unstale":"2024-08-20T01:06:36.900778+0000","last_undegraded":"2024-08-20T01:06:36.900778+0000","last_fullsized":"2024-08-20T01:06:36.900778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:03.792773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566915+0000","last_change":"2024-08-20T01:05:35.968341+0000","last_active":"2024-08-20T01:06:36.566915+0000","last_peered":"2024-08-20T01:06:36.566915+0000","last_clean":"2024-08-20T01:06:36.566915+0000","last_became_active":"2024-08-20T01:05:35.967379+0000","last_became_peered":"2024-08-20T01:05:35.967379+0000","last_unstale":"2024-08-20T01:06:36.566915+0000","last_undegraded":"2024-08-20T01:06:36.566915+0000","last_fullsized":"2024-08-20T01:06:36.566915+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:50:23.441080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731513+0000","last_change":"2024-08-20T01:05:26.609290+0000","last_active":"2024-08-20T01:07:34.731513+0000","last_peered":"2024-08-20T01:07:34.731513+0000","last_clean":"2024-08-20T01:07:34.731513+0000","last_became_active":"2024-08-20T01:05:26.608593+0000","last_became_peered":"2024-08-20T01:05:26.608593+0000","last_unstale":"2024-08-20T01:07:34.731513+0000","last_undegraded":"2024-08-20T01:07:34.731513+0000","last_fullsized":"2024-08-20T01:07:34.731513+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:06.697800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":181,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.825138+0000","last_change":"2024-08-20T01:05:23.577456+0000","last_active":"2024-08-20T01:08:33.825138+0000","last_peered":"2024-08-20T01:08:33.825138+0000","last_clean":"2024-08-20T01:08:33.825138+0000","last_became_active":"2024-08-20T01:05:23.577070+0000","last_became_peered":"2024-08-20T01:05:23.577070+0000","last_unstale":"2024-08-20T01:08:33.825138+0000","last_undegraded":"2024-08-20T01:08:33.825138+0000","last_fullsized":"2024-08-20T01:08:33.825138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:16:13.603325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":135,"num_read_kb":135,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567516+0000","last_change":"2024-08-20T01:05:35.991918+0000","last_active":"2024-08-20T01:06:36.567516+0000","last_peered":"2024-08-20T01:06:36.567516+0000","last_clean":"2024-08-20T01:06:36.567516+0000","last_became_active":"2024-08-20T01:05:35.991174+0000","last_became_peered":"2024-08-20T01:05:35.991174+0000","last_unstale":"2024-08-20T01:06:36.567516+0000","last_undegraded":"2024-08-20T01:06:36.567516+0000","last_fullsized":"2024-08-20T01:06:36.567516+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:51:38.487728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.727983+0000","last_change":"2024-08-20T01:05:26.605931+0000","last_active":"2024-08-20T01:07:34.727983+0000","last_peered":"2024-08-20T01:07:34.727983+0000","last_clean":"2024-08-20T01:07:34.727983+0000","last_became_active":"2024-08-20T01:05:26.605580+0000","last_became_peered":"2024-08-20T01:05:26.605580+0000","last_unstale":"2024-08-20T01:07:34.727983+0000","last_undegraded":"2024-08-20T01:07:34.727983+0000","last_fullsized":"2024-08-20T01:07:34.727983+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:41.083980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900807+0000","last_change":"2024-08-20T01:05:23.571973+0000","last_active":"2024-08-20T01:06:36.900807+0000","last_peered":"2024-08-20T01:06:36.900807+0000","last_clean":"2024-08-20T01:06:36.900807+0000","last_became_active":"2024-08-20T01:05:23.571615+0000","last_became_peered":"2024-08-20T01:05:23.571615+0000","last_unstale":"2024-08-20T01:06:36.900807+0000","last_undegraded":"2024-08-20T01:06:36.900807+0000","last_fullsized":"2024-08-20T01:06:36.900807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:25:59.351374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647669+0000","last_change":"2024-08-20T01:05:35.989955+0000","last_active":"2024-08-20T01:06:36.647669+0000","last_peered":"2024-08-20T01:06:36.647669+0000","last_clean":"2024-08-20T01:06:36.647669+0000","last_became_active":"2024-08-20T01:05:35.989565+0000","last_became_peered":"2024-08-20T01:05:35.989565+0000","last_unstale":"2024-08-20T01:06:36.647669+0000","last_undegraded":"2024-08-20T01:06:36.647669+0000","last_fullsized":"2024-08-20T01:06:36.647669+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:40:07.918860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747444+0000","last_change":"2024-08-20T01:05:26.607378+0000","last_active":"2024-08-20T01:07:34.747444+0000","last_peered":"2024-08-20T01:07:34.747444+0000","last_clean":"2024-08-20T01:07:34.747444+0000","last_became_active":"2024-08-20T01:05:26.607041+0000","last_became_peered":"2024-08-20T01:05:26.607041+0000","last_unstale":"2024-08-20T01:07:34.747444+0000","last_undegraded":"2024-08-20T01:07:34.747444+0000","last_fullsized":"2024-08-20T01:07:34.747444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:16:38.744029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565335+0000","last_change":"2024-08-20T01:05:23.563538+0000","last_active":"2024-08-20T01:06:36.565335+0000","last_peered":"2024-08-20T01:06:36.565335+0000","last_clean":"2024-08-20T01:06:36.565335+0000","last_became_active":"2024-08-20T01:05:23.563180+0000","last_became_peered":"2024-08-20T01:05:23.563180+0000","last_unstale":"2024-08-20T01:06:36.565335+0000","last_undegraded":"2024-08-20T01:06:36.565335+0000","last_fullsized":"2024-08-20T01:06:36.565335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:19:53.090008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900724+0000","last_change":"2024-08-20T01:04:06.709259+0000","last_active":"2024-08-20T01:06:36.900724+0000","last_peered":"2024-08-20T01:06:36.900724+0000","last_clean":"2024-08-20T01:06:36.900724+0000","last_became_active":"2024-08-20T01:04:06.704067+0000","last_became_peered":"2024-08-20T01:04:06.704067+0000","last_unstale":"2024-08-20T01:06:36.900724+0000","last_undegraded":"2024-08-20T01:06:36.900724+0000","last_fullsized":"2024-08-20T01:06:36.900724+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:00:31.072119+0000","last_clean_scrub_stamp":"2024-08-20T01:00:31.072119+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:27:50.192072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"86'1","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.900855+0000","last_change":"2024-08-20T01:05:35.962698+0000","last_active":"2024-08-20T01:06:36.900855+0000","last_peered":"2024-08-20T01:06:36.900855+0000","last_clean":"2024-08-20T01:06:36.900855+0000","last_became_active":"2024-08-20T01:05:35.961547+0000","last_became_peered":"2024-08-20T01:05:35.961547+0000","last_unstale":"2024-08-20T01:06:36.900855+0000","last_undegraded":"2024-08-20T01:06:36.900855+0000","last_fullsized":"2024-08-20T01:06:36.900855+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:14:39.916541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732158+0000","last_change":"2024-08-20T01:05:26.615968+0000","last_active":"2024-08-20T01:07:34.732158+0000","last_peered":"2024-08-20T01:07:34.732158+0000","last_clean":"2024-08-20T01:07:34.732158+0000","last_became_active":"2024-08-20T01:05:26.615632+0000","last_became_peered":"2024-08-20T01:05:26.615632+0000","last_unstale":"2024-08-20T01:07:34.732158+0000","last_undegraded":"2024-08-20T01:07:34.732158+0000","last_fullsized":"2024-08-20T01:07:34.732158+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:39:40.463331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567812+0000","last_change":"2024-08-20T01:05:23.576798+0000","last_active":"2024-08-20T01:06:36.567812+0000","last_peered":"2024-08-20T01:06:36.567812+0000","last_clean":"2024-08-20T01:06:36.567812+0000","last_became_active":"2024-08-20T01:05:23.576496+0000","last_became_peered":"2024-08-20T01:05:23.576496+0000","last_unstale":"2024-08-20T01:06:36.567812+0000","last_undegraded":"2024-08-20T01:06:36.567812+0000","last_fullsized":"2024-08-20T01:06:36.567812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:33:50.392509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.642230+0000","last_change":"2024-08-20T01:05:35.968152+0000","last_active":"2024-08-20T01:06:36.642230+0000","last_peered":"2024-08-20T01:06:36.642230+0000","last_clean":"2024-08-20T01:06:36.642230+0000","last_became_active":"2024-08-20T01:05:35.967718+0000","last_became_peered":"2024-08-20T01:05:35.967718+0000","last_unstale":"2024-08-20T01:06:36.642230+0000","last_undegraded":"2024-08-20T01:06:36.642230+0000","last_fullsized":"2024-08-20T01:06:36.642230+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:20:50.190204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731242+0000","last_change":"2024-08-20T01:05:26.607408+0000","last_active":"2024-08-20T01:07:34.731242+0000","last_peered":"2024-08-20T01:07:34.731242+0000","last_clean":"2024-08-20T01:07:34.731242+0000","last_became_active":"2024-08-20T01:05:26.607067+0000","last_became_peered":"2024-08-20T01:05:26.607067+0000","last_unstale":"2024-08-20T01:07:34.731242+0000","last_undegraded":"2024-08-20T01:07:34.731242+0000","last_fullsized":"2024-08-20T01:07:34.731242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:34:44.526174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'60","reported_seq":132,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:34.030121+0000","last_change":"2024-08-20T01:05:23.569786+0000","last_active":"2024-08-20T01:08:34.030121+0000","last_peered":"2024-08-20T01:08:34.030121+0000","last_clean":"2024-08-20T01:08:34.030121+0000","last_became_active":"2024-08-20T01:05:23.568788+0000","last_became_peered":"2024-08-20T01:05:23.568788+0000","last_unstale":"2024-08-20T01:08:34.030121+0000","last_undegraded":"2024-08-20T01:08:34.030121+0000","last_fullsized":"2024-08-20T01:08:34.030121+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:02:53.271186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393846,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":32,"num_write":71,"num_write_kb":873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425109+0000","last_change":"2024-08-20T01:05:35.972922+0000","last_active":"2024-08-20T01:06:37.425109+0000","last_peered":"2024-08-20T01:06:37.425109+0000","last_clean":"2024-08-20T01:06:37.425109+0000","last_became_active":"2024-08-20T01:05:35.972450+0000","last_became_peered":"2024-08-20T01:05:35.972450+0000","last_unstale":"2024-08-20T01:06:37.425109+0000","last_undegraded":"2024-08-20T01:06:37.425109+0000","last_fullsized":"2024-08-20T01:06:37.425109+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:13:11.882379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731342+0000","last_change":"2024-08-20T01:05:26.605846+0000","last_active":"2024-08-20T01:07:34.731342+0000","last_peered":"2024-08-20T01:07:34.731342+0000","last_clean":"2024-08-20T01:07:34.731342+0000","last_became_active":"2024-08-20T01:05:26.605517+0000","last_became_peered":"2024-08-20T01:05:26.605517+0000","last_unstale":"2024-08-20T01:07:34.731342+0000","last_undegraded":"2024-08-20T01:07:34.731342+0000","last_fullsized":"2024-08-20T01:07:34.731342+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:05:46.969059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567578+0000","last_change":"2024-08-20T01:05:23.568902+0000","last_active":"2024-08-20T01:06:36.567578+0000","last_peered":"2024-08-20T01:06:36.567578+0000","last_clean":"2024-08-20T01:06:36.567578+0000","last_became_active":"2024-08-20T01:05:23.568502+0000","last_became_peered":"2024-08-20T01:05:23.568502+0000","last_unstale":"2024-08-20T01:06:36.567578+0000","last_undegraded":"2024-08-20T01:06:36.567578+0000","last_fullsized":"2024-08-20T01:06:36.567578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:01.519948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.646941+0000","last_change":"2024-08-20T01:05:35.962706+0000","last_active":"2024-08-20T01:06:36.646941+0000","last_peered":"2024-08-20T01:06:36.646941+0000","last_clean":"2024-08-20T01:06:36.646941+0000","last_became_active":"2024-08-20T01:05:35.962263+0000","last_became_peered":"2024-08-20T01:05:35.962263+0000","last_unstale":"2024-08-20T01:06:36.646941+0000","last_undegraded":"2024-08-20T01:06:36.646941+0000","last_fullsized":"2024-08-20T01:06:36.646941+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:57:33.863231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739075+0000","last_change":"2024-08-20T01:05:26.613295+0000","last_active":"2024-08-20T01:07:34.739075+0000","last_peered":"2024-08-20T01:07:34.739075+0000","last_clean":"2024-08-20T01:07:34.739075+0000","last_became_active":"2024-08-20T01:05:26.612998+0000","last_became_peered":"2024-08-20T01:05:26.612998+0000","last_unstale":"2024-08-20T01:07:34.739075+0000","last_undegraded":"2024-08-20T01:07:34.739075+0000","last_fullsized":"2024-08-20T01:07:34.739075+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:28:50.999435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566920+0000","last_change":"2024-08-20T01:05:23.573992+0000","last_active":"2024-08-20T01:06:36.566920+0000","last_peered":"2024-08-20T01:06:36.566920+0000","last_clean":"2024-08-20T01:06:36.566920+0000","last_became_active":"2024-08-20T01:05:23.573389+0000","last_became_peered":"2024-08-20T01:05:23.573389+0000","last_unstale":"2024-08-20T01:06:36.566920+0000","last_undegraded":"2024-08-20T01:06:36.566920+0000","last_fullsized":"2024-08-20T01:06:36.566920+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:40:11.163554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.426955+0000","last_change":"2024-08-20T01:06:37.426893+0000","last_active":"2024-08-20T01:06:37.426955+0000","last_peered":"2024-08-20T01:06:37.426955+0000","last_clean":"2024-08-20T01:06:37.426955+0000","last_became_active":"2024-08-20T01:06:37.426234+0000","last_became_peered":"2024-08-20T01:06:37.426234+0000","last_unstale":"2024-08-20T01:06:37.426955+0000","last_undegraded":"2024-08-20T01:06:37.426955+0000","last_fullsized":"2024-08-20T01:06:37.426955+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:25.279532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731827+0000","last_change":"2024-08-20T01:05:35.958529+0000","last_active":"2024-08-20T01:07:34.731827+0000","last_peered":"2024-08-20T01:07:34.731827+0000","last_clean":"2024-08-20T01:07:34.731827+0000","last_became_active":"2024-08-20T01:05:35.958116+0000","last_became_peered":"2024-08-20T01:05:35.958116+0000","last_unstale":"2024-08-20T01:07:34.731827+0000","last_undegraded":"2024-08-20T01:07:34.731827+0000","last_fullsized":"2024-08-20T01:07:34.731827+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:07:59.183763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338193+0000","last_change":"2024-08-20T01:05:35.978476+0000","last_active":"2024-08-20T01:06:37.338193+0000","last_peered":"2024-08-20T01:06:37.338193+0000","last_clean":"2024-08-20T01:06:37.338193+0000","last_became_active":"2024-08-20T01:05:35.968085+0000","last_became_peered":"2024-08-20T01:05:35.968085+0000","last_unstale":"2024-08-20T01:06:37.338193+0000","last_undegraded":"2024-08-20T01:06:37.338193+0000","last_fullsized":"2024-08-20T01:06:37.338193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:52:16.250692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.340579+0000","last_change":"2024-08-20T01:06:37.340063+0000","last_active":"2024-08-20T01:06:37.340579+0000","last_peered":"2024-08-20T01:06:37.340579+0000","last_clean":"2024-08-20T01:06:37.340579+0000","last_became_active":"2024-08-20T01:06:37.339637+0000","last_became_peered":"2024-08-20T01:06:37.339637+0000","last_unstale":"2024-08-20T01:06:37.340579+0000","last_undegraded":"2024-08-20T01:06:37.340579+0000","last_fullsized":"2024-08-20T01:06:37.340579+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:12:35.118623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.724142+0000","last_change":"2024-08-20T01:05:26.608079+0000","last_active":"2024-08-20T01:07:34.724142+0000","last_peered":"2024-08-20T01:07:34.724142+0000","last_clean":"2024-08-20T01:07:34.724142+0000","last_became_active":"2024-08-20T01:05:26.607791+0000","last_became_peered":"2024-08-20T01:05:26.607791+0000","last_unstale":"2024-08-20T01:07:34.724142+0000","last_undegraded":"2024-08-20T01:07:34.724142+0000","last_fullsized":"2024-08-20T01:07:34.724142+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:31:53.041659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":182,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.825690+0000","last_change":"2024-08-20T01:05:23.564266+0000","last_active":"2024-08-20T01:08:33.825690+0000","last_peered":"2024-08-20T01:08:33.825690+0000","last_clean":"2024-08-20T01:08:33.825690+0000","last_became_active":"2024-08-20T01:05:23.563861+0000","last_became_peered":"2024-08-20T01:05:23.563861+0000","last_unstale":"2024-08-20T01:08:33.825690+0000","last_undegraded":"2024-08-20T01:08:33.825690+0000","last_fullsized":"2024-08-20T01:08:33.825690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:23:44.003449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":138,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.621011+0000","last_change":"2024-08-20T01:05:35.965729+0000","last_active":"2024-08-20T01:06:36.621011+0000","last_peered":"2024-08-20T01:06:36.621011+0000","last_clean":"2024-08-20T01:06:36.621011+0000","last_became_active":"2024-08-20T01:05:35.965364+0000","last_became_peered":"2024-08-20T01:05:35.965364+0000","last_unstale":"2024-08-20T01:06:36.621011+0000","last_undegraded":"2024-08-20T01:06:36.621011+0000","last_fullsized":"2024-08-20T01:06:36.621011+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:08:59.993221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739960+0000","last_change":"2024-08-20T01:05:26.613184+0000","last_active":"2024-08-20T01:07:34.739960+0000","last_peered":"2024-08-20T01:07:34.739960+0000","last_clean":"2024-08-20T01:07:34.739960+0000","last_became_active":"2024-08-20T01:05:26.612885+0000","last_became_peered":"2024-08-20T01:05:26.612885+0000","last_unstale":"2024-08-20T01:07:34.739960+0000","last_undegraded":"2024-08-20T01:07:34.739960+0000","last_fullsized":"2024-08-20T01:07:34.739960+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:22:20.119401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647459+0000","last_change":"2024-08-20T01:05:23.569574+0000","last_active":"2024-08-20T01:06:36.647459+0000","last_peered":"2024-08-20T01:06:36.647459+0000","last_clean":"2024-08-20T01:06:36.647459+0000","last_became_active":"2024-08-20T01:05:23.568973+0000","last_became_peered":"2024-08-20T01:05:23.568973+0000","last_unstale":"2024-08-20T01:06:36.647459+0000","last_undegraded":"2024-08-20T01:06:36.647459+0000","last_fullsized":"2024-08-20T01:06:36.647459+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:11:32.362905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338356+0000","last_change":"2024-08-20T01:05:35.968025+0000","last_active":"2024-08-20T01:06:37.338356+0000","last_peered":"2024-08-20T01:06:37.338356+0000","last_clean":"2024-08-20T01:06:37.338356+0000","last_became_active":"2024-08-20T01:05:35.967652+0000","last_became_peered":"2024-08-20T01:05:35.967652+0000","last_unstale":"2024-08-20T01:06:37.338356+0000","last_undegraded":"2024-08-20T01:06:37.338356+0000","last_fullsized":"2024-08-20T01:06:37.338356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:15:07.726048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725099+0000","last_change":"2024-08-20T01:05:26.607298+0000","last_active":"2024-08-20T01:07:34.725099+0000","last_peered":"2024-08-20T01:07:34.725099+0000","last_clean":"2024-08-20T01:07:34.725099+0000","last_became_active":"2024-08-20T01:05:26.606939+0000","last_became_peered":"2024-08-20T01:05:26.606939+0000","last_unstale":"2024-08-20T01:07:34.725099+0000","last_undegraded":"2024-08-20T01:07:34.725099+0000","last_fullsized":"2024-08-20T01:07:34.725099+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:51:10.053368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566749+0000","last_change":"2024-08-20T01:05:23.570642+0000","last_active":"2024-08-20T01:06:36.566749+0000","last_peered":"2024-08-20T01:06:36.566749+0000","last_clean":"2024-08-20T01:06:36.566749+0000","last_became_active":"2024-08-20T01:05:23.570294+0000","last_became_peered":"2024-08-20T01:05:23.570294+0000","last_unstale":"2024-08-20T01:06:36.566749+0000","last_undegraded":"2024-08-20T01:06:36.566749+0000","last_fullsized":"2024-08-20T01:06:36.566749+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:12:25.998803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647424+0000","last_change":"2024-08-20T01:05:35.969965+0000","last_active":"2024-08-20T01:06:36.647424+0000","last_peered":"2024-08-20T01:06:36.647424+0000","last_clean":"2024-08-20T01:06:36.647424+0000","last_became_active":"2024-08-20T01:05:35.969511+0000","last_became_peered":"2024-08-20T01:05:35.969511+0000","last_unstale":"2024-08-20T01:06:36.647424+0000","last_undegraded":"2024-08-20T01:06:36.647424+0000","last_fullsized":"2024-08-20T01:06:36.647424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:56:22.155575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.719431+0000","last_change":"2024-08-20T01:05:26.608527+0000","last_active":"2024-08-20T01:07:34.719431+0000","last_peered":"2024-08-20T01:07:34.719431+0000","last_clean":"2024-08-20T01:07:34.719431+0000","last_became_active":"2024-08-20T01:05:26.608213+0000","last_became_peered":"2024-08-20T01:05:26.608213+0000","last_unstale":"2024-08-20T01:07:34.719431+0000","last_undegraded":"2024-08-20T01:07:34.719431+0000","last_fullsized":"2024-08-20T01:07:34.719431+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:59:36.484132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'41","reported_seq":364,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:33.826628+0000","last_change":"2024-08-20T01:05:23.568199+0000","last_active":"2024-08-20T01:08:33.826628+0000","last_peered":"2024-08-20T01:08:33.826628+0000","last_clean":"2024-08-20T01:08:33.826628+0000","last_became_active":"2024-08-20T01:05:23.567894+0000","last_became_peered":"2024-08-20T01:05:23.567894+0000","last_unstale":"2024-08-20T01:08:33.826628+0000","last_undegraded":"2024-08-20T01:08:33.826628+0000","last_fullsized":"2024-08-20T01:08:33.826628+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:46:24.051618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69492,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":281,"num_read_kb":205,"num_write":42,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620266+0000","last_change":"2024-08-20T01:05:35.964698+0000","last_active":"2024-08-20T01:06:36.620266+0000","last_peered":"2024-08-20T01:06:36.620266+0000","last_clean":"2024-08-20T01:06:36.620266+0000","last_became_active":"2024-08-20T01:05:35.964087+0000","last_became_peered":"2024-08-20T01:05:35.964087+0000","last_unstale":"2024-08-20T01:06:36.620266+0000","last_undegraded":"2024-08-20T01:06:36.620266+0000","last_fullsized":"2024-08-20T01:06:36.620266+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:59:13.070499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747475+0000","last_change":"2024-08-20T01:05:26.620951+0000","last_active":"2024-08-20T01:07:34.747475+0000","last_peered":"2024-08-20T01:07:34.747475+0000","last_clean":"2024-08-20T01:07:34.747475+0000","last_became_active":"2024-08-20T01:05:26.620186+0000","last_became_peered":"2024-08-20T01:05:26.620186+0000","last_unstale":"2024-08-20T01:07:34.747475+0000","last_undegraded":"2024-08-20T01:07:34.747475+0000","last_fullsized":"2024-08-20T01:07:34.747475+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:22:15.124246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647695+0000","last_change":"2024-08-20T01:05:23.577554+0000","last_active":"2024-08-20T01:06:36.647695+0000","last_peered":"2024-08-20T01:06:36.647695+0000","last_clean":"2024-08-20T01:06:36.647695+0000","last_became_active":"2024-08-20T01:05:23.577215+0000","last_became_peered":"2024-08-20T01:05:23.577215+0000","last_unstale":"2024-08-20T01:06:36.647695+0000","last_undegraded":"2024-08-20T01:06:36.647695+0000","last_fullsized":"2024-08-20T01:06:36.647695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:05:01.566604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.565025+0000","last_change":"2024-08-20T01:05:35.968836+0000","last_active":"2024-08-20T01:06:36.565025+0000","last_peered":"2024-08-20T01:06:36.565025+0000","last_clean":"2024-08-20T01:06:36.565025+0000","last_became_active":"2024-08-20T01:05:35.968341+0000","last_became_peered":"2024-08-20T01:05:35.968341+0000","last_unstale":"2024-08-20T01:06:36.565025+0000","last_undegraded":"2024-08-20T01:06:36.565025+0000","last_fullsized":"2024-08-20T01:06:36.565025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:33.284565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740128+0000","last_change":"2024-08-20T01:05:26.619802+0000","last_active":"2024-08-20T01:07:34.740128+0000","last_peered":"2024-08-20T01:07:34.740128+0000","last_clean":"2024-08-20T01:07:34.740128+0000","last_became_active":"2024-08-20T01:05:26.616585+0000","last_became_peered":"2024-08-20T01:05:26.616585+0000","last_unstale":"2024-08-20T01:07:34.740128+0000","last_undegraded":"2024-08-20T01:07:34.740128+0000","last_fullsized":"2024-08-20T01:07:34.740128+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:01:12.904844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.901002+0000","last_change":"2024-08-20T01:05:23.567445+0000","last_active":"2024-08-20T01:06:36.901002+0000","last_peered":"2024-08-20T01:06:36.901002+0000","last_clean":"2024-08-20T01:06:36.901002+0000","last_became_active":"2024-08-20T01:05:23.566975+0000","last_became_peered":"2024-08-20T01:05:23.566975+0000","last_unstale":"2024-08-20T01:06:36.901002+0000","last_undegraded":"2024-08-20T01:06:36.901002+0000","last_fullsized":"2024-08-20T01:06:36.901002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:36:48.858325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.620177+0000","last_change":"2024-08-20T01:05:35.972530+0000","last_active":"2024-08-20T01:06:36.620177+0000","last_peered":"2024-08-20T01:06:36.620177+0000","last_clean":"2024-08-20T01:06:36.620177+0000","last_became_active":"2024-08-20T01:05:35.972165+0000","last_became_peered":"2024-08-20T01:05:35.972165+0000","last_unstale":"2024-08-20T01:06:36.620177+0000","last_undegraded":"2024-08-20T01:06:36.620177+0000","last_fullsized":"2024-08-20T01:06:36.620177+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:21.134164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.746528+0000","last_change":"2024-08-20T01:05:26.616451+0000","last_active":"2024-08-20T01:07:34.746528+0000","last_peered":"2024-08-20T01:07:34.746528+0000","last_clean":"2024-08-20T01:07:34.746528+0000","last_became_active":"2024-08-20T01:05:26.616151+0000","last_became_peered":"2024-08-20T01:05:26.616151+0000","last_unstale":"2024-08-20T01:07:34.746528+0000","last_undegraded":"2024-08-20T01:07:34.746528+0000","last_fullsized":"2024-08-20T01:07:34.746528+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:07:44.257927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'5","reported_seq":181,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:32.824837+0000","last_change":"2024-08-20T01:05:23.572377+0000","last_active":"2024-08-20T01:08:32.824837+0000","last_peered":"2024-08-20T01:08:32.824837+0000","last_clean":"2024-08-20T01:08:32.824837+0000","last_became_active":"2024-08-20T01:05:23.571753+0000","last_became_peered":"2024-08-20T01:05:23.571753+0000","last_unstale":"2024-08-20T01:08:32.824837+0000","last_undegraded":"2024-08-20T01:08:32.824837+0000","last_fullsized":"2024-08-20T01:08:32.824837+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:33:28.556838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":134,"num_read_kb":134,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.564777+0000","last_change":"2024-08-20T01:05:35.963959+0000","last_active":"2024-08-20T01:06:36.564777+0000","last_peered":"2024-08-20T01:06:36.564777+0000","last_clean":"2024-08-20T01:06:36.564777+0000","last_became_active":"2024-08-20T01:05:35.960735+0000","last_became_peered":"2024-08-20T01:05:35.960735+0000","last_unstale":"2024-08-20T01:06:36.564777+0000","last_undegraded":"2024-08-20T01:06:36.564777+0000","last_fullsized":"2024-08-20T01:06:36.564777+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T12:52:51.808704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.425144+0000","last_change":"2024-08-20T01:05:35.974589+0000","last_active":"2024-08-20T01:06:37.425144+0000","last_peered":"2024-08-20T01:06:37.425144+0000","last_clean":"2024-08-20T01:06:37.425144+0000","last_became_active":"2024-08-20T01:05:35.974205+0000","last_became_peered":"2024-08-20T01:05:35.974205+0000","last_unstale":"2024-08-20T01:06:37.425144+0000","last_undegraded":"2024-08-20T01:06:37.425144+0000","last_fullsized":"2024-08-20T01:06:37.425144+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:45:48.467599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731657+0000","last_change":"2024-08-20T01:05:26.606654+0000","last_active":"2024-08-20T01:07:34.731657+0000","last_peered":"2024-08-20T01:07:34.731657+0000","last_clean":"2024-08-20T01:07:34.731657+0000","last_became_active":"2024-08-20T01:05:26.606314+0000","last_became_peered":"2024-08-20T01:05:26.606314+0000","last_unstale":"2024-08-20T01:07:34.731657+0000","last_undegraded":"2024-08-20T01:07:34.731657+0000","last_fullsized":"2024-08-20T01:07:34.731657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:54:05.976602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.647535+0000","last_change":"2024-08-20T01:05:23.570319+0000","last_active":"2024-08-20T01:06:36.647535+0000","last_peered":"2024-08-20T01:06:36.647535+0000","last_clean":"2024-08-20T01:06:36.647535+0000","last_became_active":"2024-08-20T01:05:23.569597+0000","last_became_peered":"2024-08-20T01:05:23.569597+0000","last_unstale":"2024-08-20T01:06:36.647535+0000","last_undegraded":"2024-08-20T01:06:36.647535+0000","last_fullsized":"2024-08-20T01:06:36.647535+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:40:41.908327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:37.338530+0000","last_change":"2024-08-20T01:05:35.965826+0000","last_active":"2024-08-20T01:06:37.338530+0000","last_peered":"2024-08-20T01:06:37.338530+0000","last_clean":"2024-08-20T01:06:37.338530+0000","last_became_active":"2024-08-20T01:05:35.965438+0000","last_became_peered":"2024-08-20T01:05:35.965438+0000","last_unstale":"2024-08-20T01:06:37.338530+0000","last_undegraded":"2024-08-20T01:06:37.338530+0000","last_fullsized":"2024-08-20T01:06:37.338530+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:30:44.430490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728742+0000","last_change":"2024-08-20T01:05:26.604712+0000","last_active":"2024-08-20T01:07:34.728742+0000","last_peered":"2024-08-20T01:07:34.728742+0000","last_clean":"2024-08-20T01:07:34.728742+0000","last_became_active":"2024-08-20T01:05:26.604383+0000","last_became_peered":"2024-08-20T01:05:26.604383+0000","last_unstale":"2024-08-20T01:07:34.728742+0000","last_undegraded":"2024-08-20T01:07:34.728742+0000","last_fullsized":"2024-08-20T01:07:34.728742+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:28:52.564635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566516+0000","last_change":"2024-08-20T01:05:23.567007+0000","last_active":"2024-08-20T01:06:36.566516+0000","last_peered":"2024-08-20T01:06:36.566516+0000","last_clean":"2024-08-20T01:06:36.566516+0000","last_became_active":"2024-08-20T01:05:23.566488+0000","last_became_peered":"2024-08-20T01:05:23.566488+0000","last_unstale":"2024-08-20T01:06:36.566516+0000","last_undegraded":"2024-08-20T01:06:36.566516+0000","last_fullsized":"2024-08-20T01:06:36.566516+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:30:47.924840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.644361+0000","last_change":"2024-08-20T01:05:23.570441+0000","last_active":"2024-08-20T01:06:36.644361+0000","last_peered":"2024-08-20T01:06:36.644361+0000","last_clean":"2024-08-20T01:06:36.644361+0000","last_became_active":"2024-08-20T01:05:23.570070+0000","last_became_peered":"2024-08-20T01:05:23.570070+0000","last_unstale":"2024-08-20T01:06:36.644361+0000","last_undegraded":"2024-08-20T01:06:36.644361+0000","last_fullsized":"2024-08-20T01:06:36.644361+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:43:48.703869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732168+0000","last_change":"2024-08-20T01:05:26.620101+0000","last_active":"2024-08-20T01:07:34.732168+0000","last_peered":"2024-08-20T01:07:34.732168+0000","last_clean":"2024-08-20T01:07:34.732168+0000","last_became_active":"2024-08-20T01:05:26.619858+0000","last_became_peered":"2024-08-20T01:05:26.619858+0000","last_unstale":"2024-08-20T01:07:34.732168+0000","last_undegraded":"2024-08-20T01:07:34.732168+0000","last_fullsized":"2024-08-20T01:07:34.732168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T04:03:43.142995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.566548+0000","last_change":"2024-08-20T01:05:35.973400+0000","last_active":"2024-08-20T01:06:36.566548+0000","last_peered":"2024-08-20T01:06:36.566548+0000","last_clean":"2024-08-20T01:06:36.566548+0000","last_became_active":"2024-08-20T01:05:35.973042+0000","last_became_peered":"2024-08-20T01:05:35.973042+0000","last_unstale":"2024-08-20T01:06:36.566548+0000","last_undegraded":"2024-08-20T01:06:36.566548+0000","last_fullsized":"2024-08-20T01:06:36.566548+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:20.408544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'8","reported_seq":189,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:08:32.825374+0000","last_change":"2024-08-20T01:05:23.571679+0000","last_active":"2024-08-20T01:08:32.825374+0000","last_peered":"2024-08-20T01:08:32.825374+0000","last_clean":"2024-08-20T01:08:32.825374+0000","last_became_active":"2024-08-20T01:05:23.571297+0000","last_became_peered":"2024-08-20T01:05:23.571297+0000","last_unstale":"2024-08-20T01:08:32.825374+0000","last_undegraded":"2024-08-20T01:08:32.825374+0000","last_fullsized":"2024-08-20T01:08:32.825374+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:22.543757+0000","last_clean_scrub_stamp":"2024-08-20T01:05:22.543757+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:10:04.150457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1465,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":139,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"88'7","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732844+0000","last_change":"2024-08-20T01:05:35.953602+0000","last_active":"2024-08-20T01:07:34.732844+0000","last_peered":"2024-08-20T01:07:34.732844+0000","last_clean":"2024-08-20T01:07:34.732844+0000","last_became_active":"2024-08-20T01:05:35.953084+0000","last_became_peered":"2024-08-20T01:05:35.953084+0000","last_unstale":"2024-08-20T01:07:34.732844+0000","last_undegraded":"2024-08-20T01:07:34.732844+0000","last_fullsized":"2024-08-20T01:07:34.732844+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:38:39.928215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,6],"acting":[8,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:06:36.567404+0000","last_change":"2024-08-20T01:05:35.965677+0000","last_active":"2024-08-20T01:06:36.567404+0000","last_peered":"2024-08-20T01:06:36.567404+0000","last_clean":"2024-08-20T01:06:36.567404+0000","last_became_active":"2024-08-20T01:05:35.964246+0000","last_became_peered":"2024-08-20T01:05:35.964246+0000","last_unstale":"2024-08-20T01:06:36.567404+0000","last_undegraded":"2024-08-20T01:06:36.567404+0000","last_fullsized":"2024-08-20T01:06:36.567404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:34.937557+0000","last_clean_scrub_stamp":"2024-08-20T01:05:34.937557+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:29:33.102951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732743+0000","last_change":"2024-08-20T01:05:26.606864+0000","last_active":"2024-08-20T01:07:34.732743+0000","last_peered":"2024-08-20T01:07:34.732743+0000","last_clean":"2024-08-20T01:07:34.732743+0000","last_became_active":"2024-08-20T01:05:26.606561+0000","last_became_peered":"2024-08-20T01:05:26.606561+0000","last_unstale":"2024-08-20T01:07:34.732743+0000","last_undegraded":"2024-08-20T01:07:34.732743+0000","last_fullsized":"2024-08-20T01:07:34.732743+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:26:13.037401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"88'12","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733589+0000","last_change":"2024-08-20T01:05:26.612799+0000","last_active":"2024-08-20T01:07:34.733589+0000","last_peered":"2024-08-20T01:07:34.733589+0000","last_clean":"2024-08-20T01:07:34.733589+0000","last_became_active":"2024-08-20T01:05:26.612503+0000","last_became_peered":"2024-08-20T01:05:26.612503+0000","last_unstale":"2024-08-20T01:07:34.733589+0000","last_undegraded":"2024-08-20T01:07:34.733589+0000","last_fullsized":"2024-08-20T01:07:34.733589+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:22:53.272755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739063+0000","last_change":"2024-08-20T01:05:26.622720+0000","last_active":"2024-08-20T01:07:34.739063+0000","last_peered":"2024-08-20T01:07:34.739063+0000","last_clean":"2024-08-20T01:07:34.739063+0000","last_became_active":"2024-08-20T01:05:26.622378+0000","last_became_peered":"2024-08-20T01:05:26.622378+0000","last_unstale":"2024-08-20T01:07:34.739063+0000","last_undegraded":"2024-08-20T01:07:34.739063+0000","last_fullsized":"2024-08-20T01:07:34.739063+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:48:56.550426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739518+0000","last_change":"2024-08-20T01:05:26.616994+0000","last_active":"2024-08-20T01:07:34.739518+0000","last_peered":"2024-08-20T01:07:34.739518+0000","last_clean":"2024-08-20T01:07:34.739518+0000","last_became_active":"2024-08-20T01:05:26.616630+0000","last_became_peered":"2024-08-20T01:05:26.616630+0000","last_unstale":"2024-08-20T01:07:34.739518+0000","last_undegraded":"2024-08-20T01:07:34.739518+0000","last_fullsized":"2024-08-20T01:07:34.739518+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:47:55.084614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740881+0000","last_change":"2024-08-20T01:05:26.612793+0000","last_active":"2024-08-20T01:07:34.740881+0000","last_peered":"2024-08-20T01:07:34.740881+0000","last_clean":"2024-08-20T01:07:34.740881+0000","last_became_active":"2024-08-20T01:05:26.612478+0000","last_became_peered":"2024-08-20T01:05:26.612478+0000","last_unstale":"2024-08-20T01:07:34.740881+0000","last_undegraded":"2024-08-20T01:07:34.740881+0000","last_fullsized":"2024-08-20T01:07:34.740881+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:31.035251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729504+0000","last_change":"2024-08-20T01:05:26.603937+0000","last_active":"2024-08-20T01:07:34.729504+0000","last_peered":"2024-08-20T01:07:34.729504+0000","last_clean":"2024-08-20T01:07:34.729504+0000","last_became_active":"2024-08-20T01:05:26.603588+0000","last_became_peered":"2024-08-20T01:05:26.603588+0000","last_unstale":"2024-08-20T01:07:34.729504+0000","last_undegraded":"2024-08-20T01:07:34.729504+0000","last_fullsized":"2024-08-20T01:07:34.729504+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:14:53.737452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"88'15","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.725362+0000","last_change":"2024-08-20T01:05:26.610311+0000","last_active":"2024-08-20T01:07:34.725362+0000","last_peered":"2024-08-20T01:07:34.725362+0000","last_clean":"2024-08-20T01:07:34.725362+0000","last_became_active":"2024-08-20T01:05:26.609984+0000","last_became_peered":"2024-08-20T01:05:26.609984+0000","last_unstale":"2024-08-20T01:07:34.725362+0000","last_undegraded":"2024-08-20T01:07:34.725362+0000","last_fullsized":"2024-08-20T01:07:34.725362+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:02:54.973710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"88'2","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.728179+0000","last_change":"2024-08-20T01:05:26.613835+0000","last_active":"2024-08-20T01:07:34.728179+0000","last_peered":"2024-08-20T01:07:34.728179+0000","last_clean":"2024-08-20T01:07:34.728179+0000","last_became_active":"2024-08-20T01:05:26.613225+0000","last_became_peered":"2024-08-20T01:05:26.613225+0000","last_unstale":"2024-08-20T01:07:34.728179+0000","last_undegraded":"2024-08-20T01:07:34.728179+0000","last_fullsized":"2024-08-20T01:07:34.728179+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:48:23.876532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732606+0000","last_change":"2024-08-20T01:05:26.608793+0000","last_active":"2024-08-20T01:07:34.732606+0000","last_peered":"2024-08-20T01:07:34.732606+0000","last_clean":"2024-08-20T01:07:34.732606+0000","last_became_active":"2024-08-20T01:05:26.608193+0000","last_became_peered":"2024-08-20T01:05:26.608193+0000","last_unstale":"2024-08-20T01:07:34.732606+0000","last_undegraded":"2024-08-20T01:07:34.732606+0000","last_fullsized":"2024-08-20T01:07:34.732606+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:19:56.661805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.729298+0000","last_change":"2024-08-20T01:05:26.607853+0000","last_active":"2024-08-20T01:07:34.729298+0000","last_peered":"2024-08-20T01:07:34.729298+0000","last_clean":"2024-08-20T01:07:34.729298+0000","last_became_active":"2024-08-20T01:05:26.607549+0000","last_became_peered":"2024-08-20T01:05:26.607549+0000","last_unstale":"2024-08-20T01:07:34.729298+0000","last_undegraded":"2024-08-20T01:07:34.729298+0000","last_fullsized":"2024-08-20T01:07:34.729298+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:37:49.063807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747462+0000","last_change":"2024-08-20T01:05:26.611737+0000","last_active":"2024-08-20T01:07:34.747462+0000","last_peered":"2024-08-20T01:07:34.747462+0000","last_clean":"2024-08-20T01:07:34.747462+0000","last_became_active":"2024-08-20T01:05:26.611453+0000","last_became_peered":"2024-08-20T01:05:26.611453+0000","last_unstale":"2024-08-20T01:07:34.747462+0000","last_undegraded":"2024-08-20T01:07:34.747462+0000","last_fullsized":"2024-08-20T01:07:34.747462+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:37:55.636304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.741436+0000","last_change":"2024-08-20T01:05:26.622876+0000","last_active":"2024-08-20T01:07:34.741436+0000","last_peered":"2024-08-20T01:07:34.741436+0000","last_clean":"2024-08-20T01:07:34.741436+0000","last_became_active":"2024-08-20T01:05:26.622543+0000","last_became_peered":"2024-08-20T01:05:26.622543+0000","last_unstale":"2024-08-20T01:07:34.741436+0000","last_undegraded":"2024-08-20T01:07:34.741436+0000","last_fullsized":"2024-08-20T01:07:34.741436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:26:45.864442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"88'8","reported_seq":51,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731490+0000","last_change":"2024-08-20T01:05:35.965410+0000","last_active":"2024-08-20T01:07:34.731490+0000","last_peered":"2024-08-20T01:07:34.731490+0000","last_clean":"2024-08-20T01:07:34.731490+0000","last_became_active":"2024-08-20T01:05:35.964793+0000","last_became_peered":"2024-08-20T01:05:35.964793+0000","last_unstale":"2024-08-20T01:07:34.731490+0000","last_undegraded":"2024-08-20T01:07:34.731490+0000","last_fullsized":"2024-08-20T01:07:34.731490+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:47:31.451462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.732616+0000","last_change":"2024-08-20T01:05:26.613069+0000","last_active":"2024-08-20T01:07:34.732616+0000","last_peered":"2024-08-20T01:07:34.732616+0000","last_clean":"2024-08-20T01:07:34.732616+0000","last_became_active":"2024-08-20T01:05:26.612079+0000","last_became_peered":"2024-08-20T01:05:26.612079+0000","last_unstale":"2024-08-20T01:07:34.732616+0000","last_undegraded":"2024-08-20T01:07:34.732616+0000","last_fullsized":"2024-08-20T01:07:34.732616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:27:35.049846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.735563+0000","last_change":"2024-08-20T01:05:26.613735+0000","last_active":"2024-08-20T01:07:34.735563+0000","last_peered":"2024-08-20T01:07:34.735563+0000","last_clean":"2024-08-20T01:07:34.735563+0000","last_became_active":"2024-08-20T01:05:26.613405+0000","last_became_peered":"2024-08-20T01:05:26.613405+0000","last_unstale":"2024-08-20T01:07:34.735563+0000","last_undegraded":"2024-08-20T01:07:34.735563+0000","last_fullsized":"2024-08-20T01:07:34.735563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:34:15.923974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742307+0000","last_change":"2024-08-20T01:05:26.620545+0000","last_active":"2024-08-20T01:07:34.742307+0000","last_peered":"2024-08-20T01:07:34.742307+0000","last_clean":"2024-08-20T01:07:34.742307+0000","last_became_active":"2024-08-20T01:05:26.620322+0000","last_became_peered":"2024-08-20T01:05:26.620322+0000","last_unstale":"2024-08-20T01:07:34.742307+0000","last_undegraded":"2024-08-20T01:07:34.742307+0000","last_fullsized":"2024-08-20T01:07:34.742307+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:45:56.065671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"88'9","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747020+0000","last_change":"2024-08-20T01:05:26.620420+0000","last_active":"2024-08-20T01:07:34.747020+0000","last_peered":"2024-08-20T01:07:34.747020+0000","last_clean":"2024-08-20T01:07:34.747020+0000","last_became_active":"2024-08-20T01:05:26.619585+0000","last_became_peered":"2024-08-20T01:05:26.619585+0000","last_unstale":"2024-08-20T01:07:34.747020+0000","last_undegraded":"2024-08-20T01:07:34.747020+0000","last_fullsized":"2024-08-20T01:07:34.747020+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T03:18:50.980268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740301+0000","last_change":"2024-08-20T01:05:26.619830+0000","last_active":"2024-08-20T01:07:34.740301+0000","last_peered":"2024-08-20T01:07:34.740301+0000","last_clean":"2024-08-20T01:07:34.740301+0000","last_became_active":"2024-08-20T01:05:26.619440+0000","last_became_peered":"2024-08-20T01:05:26.619440+0000","last_unstale":"2024-08-20T01:07:34.740301+0000","last_undegraded":"2024-08-20T01:07:34.740301+0000","last_fullsized":"2024-08-20T01:07:34.740301+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:07:21.216898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740343+0000","last_change":"2024-08-20T01:05:26.616152+0000","last_active":"2024-08-20T01:07:34.740343+0000","last_peered":"2024-08-20T01:07:34.740343+0000","last_clean":"2024-08-20T01:07:34.740343+0000","last_became_active":"2024-08-20T01:05:26.615826+0000","last_became_peered":"2024-08-20T01:05:26.615826+0000","last_unstale":"2024-08-20T01:07:34.740343+0000","last_undegraded":"2024-08-20T01:07:34.740343+0000","last_fullsized":"2024-08-20T01:07:34.740343+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:37:29.714657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"88'10","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.738693+0000","last_change":"2024-08-20T01:05:26.609060+0000","last_active":"2024-08-20T01:07:34.738693+0000","last_peered":"2024-08-20T01:07:34.738693+0000","last_clean":"2024-08-20T01:07:34.738693+0000","last_became_active":"2024-08-20T01:05:26.608271+0000","last_became_peered":"2024-08-20T01:05:26.608271+0000","last_unstale":"2024-08-20T01:07:34.738693+0000","last_undegraded":"2024-08-20T01:07:34.738693+0000","last_fullsized":"2024-08-20T01:07:34.738693+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:38:01.898862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.736334+0000","last_change":"2024-08-20T01:05:26.609094+0000","last_active":"2024-08-20T01:07:34.736334+0000","last_peered":"2024-08-20T01:07:34.736334+0000","last_clean":"2024-08-20T01:07:34.736334+0000","last_became_active":"2024-08-20T01:05:26.608776+0000","last_became_peered":"2024-08-20T01:05:26.608776+0000","last_unstale":"2024-08-20T01:07:34.736334+0000","last_undegraded":"2024-08-20T01:07:34.736334+0000","last_fullsized":"2024-08-20T01:07:34.736334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:48:01.143850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.731015+0000","last_change":"2024-08-20T01:05:26.616622+0000","last_active":"2024-08-20T01:07:34.731015+0000","last_peered":"2024-08-20T01:07:34.731015+0000","last_clean":"2024-08-20T01:07:34.731015+0000","last_became_active":"2024-08-20T01:05:26.616329+0000","last_became_peered":"2024-08-20T01:05:26.616329+0000","last_unstale":"2024-08-20T01:07:34.731015+0000","last_undegraded":"2024-08-20T01:07:34.731015+0000","last_fullsized":"2024-08-20T01:07:34.731015+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T07:45:39.702274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"88'16","reported_seq":52,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740057+0000","last_change":"2024-08-20T01:05:26.609835+0000","last_active":"2024-08-20T01:07:34.740057+0000","last_peered":"2024-08-20T01:07:34.740057+0000","last_clean":"2024-08-20T01:07:34.740057+0000","last_became_active":"2024-08-20T01:05:26.609405+0000","last_became_peered":"2024-08-20T01:05:26.609405+0000","last_unstale":"2024-08-20T01:07:34.740057+0000","last_undegraded":"2024-08-20T01:07:34.740057+0000","last_fullsized":"2024-08-20T01:07:34.740057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T08:47:04.708963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"88'11","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.744507+0000","last_change":"2024-08-20T01:05:26.616790+0000","last_active":"2024-08-20T01:07:34.744507+0000","last_peered":"2024-08-20T01:07:34.744507+0000","last_clean":"2024-08-20T01:07:34.744507+0000","last_became_active":"2024-08-20T01:05:26.616527+0000","last_became_peered":"2024-08-20T01:05:26.616527+0000","last_unstale":"2024-08-20T01:07:34.744507+0000","last_undegraded":"2024-08-20T01:07:34.744507+0000","last_fullsized":"2024-08-20T01:07:34.744507+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T01:51:06.651909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"88'9","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734496+0000","last_change":"2024-08-20T01:05:35.990306+0000","last_active":"2024-08-20T01:07:34.734496+0000","last_peered":"2024-08-20T01:07:34.734496+0000","last_clean":"2024-08-20T01:07:34.734496+0000","last_became_active":"2024-08-20T01:05:35.989928+0000","last_became_peered":"2024-08-20T01:05:35.989928+0000","last_unstale":"2024-08-20T01:07:34.734496+0000","last_undegraded":"2024-08-20T01:07:34.734496+0000","last_fullsized":"2024-08-20T01:07:34.734496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:20:52.095314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.39","version":"88'5","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.730134+0000","last_change":"2024-08-20T01:05:26.618052+0000","last_active":"2024-08-20T01:07:34.730134+0000","last_peered":"2024-08-20T01:07:34.730134+0000","last_clean":"2024-08-20T01:07:34.730134+0000","last_became_active":"2024-08-20T01:05:26.617649+0000","last_became_peered":"2024-08-20T01:05:26.617649+0000","last_unstale":"2024-08-20T01:07:34.730134+0000","last_undegraded":"2024-08-20T01:07:34.730134+0000","last_fullsized":"2024-08-20T01:07:34.730134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:39:07.646381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"88'6","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.740683+0000","last_change":"2024-08-20T01:05:26.607992+0000","last_active":"2024-08-20T01:07:34.740683+0000","last_peered":"2024-08-20T01:07:34.740683+0000","last_clean":"2024-08-20T01:07:34.740683+0000","last_became_active":"2024-08-20T01:05:26.607643+0000","last_became_peered":"2024-08-20T01:05:26.607643+0000","last_unstale":"2024-08-20T01:07:34.740683+0000","last_undegraded":"2024-08-20T01:07:34.740683+0000","last_fullsized":"2024-08-20T01:07:34.740683+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T09:38:05.337264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.734991+0000","last_change":"2024-08-20T01:05:26.610101+0000","last_active":"2024-08-20T01:07:34.734991+0000","last_peered":"2024-08-20T01:07:34.734991+0000","last_clean":"2024-08-20T01:07:34.734991+0000","last_became_active":"2024-08-20T01:05:26.609820+0000","last_became_peered":"2024-08-20T01:05:26.609820+0000","last_unstale":"2024-08-20T01:07:34.734991+0000","last_undegraded":"2024-08-20T01:07:34.734991+0000","last_fullsized":"2024-08-20T01:07:34.734991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T06:27:50.826168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"88'8","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.733118+0000","last_change":"2024-08-20T01:05:35.968498+0000","last_active":"2024-08-20T01:07:34.733118+0000","last_peered":"2024-08-20T01:07:34.733118+0000","last_clean":"2024-08-20T01:07:34.733118+0000","last_became_active":"2024-08-20T01:05:35.968073+0000","last_became_peered":"2024-08-20T01:05:35.968073+0000","last_unstale":"2024-08-20T01:07:34.733118+0000","last_undegraded":"2024-08-20T01:07:34.733118+0000","last_fullsized":"2024-08-20T01:07:34.733118+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T11:49:47.377308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3d","version":"88'7","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.747505+0000","last_change":"2024-08-20T01:05:26.609288+0000","last_active":"2024-08-20T01:07:34.747505+0000","last_peered":"2024-08-20T01:07:34.747505+0000","last_clean":"2024-08-20T01:07:34.747505+0000","last_became_active":"2024-08-20T01:05:26.609009+0000","last_became_peered":"2024-08-20T01:05:26.609009+0000","last_unstale":"2024-08-20T01:07:34.747505+0000","last_undegraded":"2024-08-20T01:07:34.747505+0000","last_fullsized":"2024-08-20T01:07:34.747505+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T10:57:08.432521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"88'4","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.739436+0000","last_change":"2024-08-20T01:05:26.617212+0000","last_active":"2024-08-20T01:07:34.739436+0000","last_peered":"2024-08-20T01:07:34.739436+0000","last_clean":"2024-08-20T01:07:34.739436+0000","last_became_active":"2024-08-20T01:05:26.615055+0000","last_became_peered":"2024-08-20T01:05:26.615055+0000","last_unstale":"2024-08-20T01:07:34.739436+0000","last_undegraded":"2024-08-20T01:07:34.739436+0000","last_fullsized":"2024-08-20T01:07:34.739436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T02:40:49.134084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"88'8","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-08-20T01:07:34.742422+0000","last_change":"2024-08-20T01:05:26.614290+0000","last_active":"2024-08-20T01:07:34.742422+0000","last_peered":"2024-08-20T01:07:34.742422+0000","last_clean":"2024-08-20T01:07:34.742422+0000","last_became_active":"2024-08-20T01:05:26.613695+0000","last_became_peered":"2024-08-20T01:05:26.613695+0000","last_unstale":"2024-08-20T01:07:34.742422+0000","last_undegraded":"2024-08-20T01:07:34.742422+0000","last_fullsized":"2024-08-20T01:07:34.742422+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-08-20T01:05:25.569995+0000","last_clean_scrub_stamp":"2024-08-20T01:05:25.569995+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-08-21T05:52:38.216323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":860937,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1164,"num_read_kb":785,"num_write":406,"num_write_kb":2225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675840,"data_stored":2582811,"data_compressed":112140,"data_compressed_allocated":147456,"data_compressed_original":2285568,"omap_allocated":0,"internal_metadata":0},"log_size":299,"ondisk_log_size":299,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247620,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678068,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":978944,"data_stored":517292,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874297,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32336,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712816,"statfs":{"total":95995035648,"available":95961923584,"internally_reserved":0,"allocated":1171456,"data_stored":1563402,"data_compressed":80566,"data_compressed_allocated":106496,"data_compressed_original":1032192,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037822,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1015808,"data_stored":914728,"data_compressed":45926,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234052,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":910440,"data_compressed":45926,"data_compressed_allocated":61440,"data_compressed_original":466944,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430280,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1036288,"data_stored":1113697,"data_compressed":62475,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593807,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":512592,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790036,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28104,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717048,"statfs":{"total":95995035648,"available":95966257152,"internally_reserved":0,"allocated":1097728,"data_stored":1503061,"data_compressed":80566,"data_compressed_allocated":106496,"data_compressed_original":1032192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953561,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":517242,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1056768,"data_stored":1299778,"data_compressed":64017,"data_compressed_allocated":86016,"data_compressed_original":835584,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378724,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1064960,"data_stored":584187,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542248,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1040384,"data_stored":581994,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705773,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":513980,"data_compressed":26637,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34215,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35838,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":787276,"data_compressed":37380,"data_compressed_allocated":49152,"data_compressed_original":761856,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":396095,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":393842,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":464846,"data_compressed":18091,"data_compressed_allocated":24576,"data_compressed_original":368640,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-08-20T01:08:40.971 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:08:40.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:40 smithi142 ceph-mon[21799]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-08-20T01:08:40.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:40 smithi142 ceph-mon[21799]: pgmap v379: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 175 KiB/s wr, 54 op/s 2024-08-20T01:08:41.341 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:41.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:41 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:08:41.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:41 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 -- 172.21.15.138:0/3072624206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c10a6b0 msgr2=0x7f3a8c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 --2- 172.21.15.138:0/3072624206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c10aaf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009900 tx=0x7f3a7402a400 comp rx=0 tx=0).stop 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 -- 172.21.15.138:0/3072624206 shutdown_connections 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 --2- 172.21.15.138:0/3072624206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 0x7f3a8c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 --2- 172.21.15.138:0/3072624206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c10aaf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 --2- 172.21.15.138:0/3072624206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:41.976 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 -- 172.21.15.138:0/3072624206 >> 172.21.15.138:0/3072624206 conn(0x7f3a8c100730 msgr2=0x7f3a8c102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 -- 172.21.15.138:0/3072624206 shutdown_connections 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.962+0000 7f3a91400640 1 -- 172.21.15.138:0/3072624206 wait complete. 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.963+0000 7f3a91400640 1 Processor -- start 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.963+0000 7f3a91400640 1 -- start start 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 0x7f3a8c1a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c114d40 con 0x7f3a8c10b030 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c114bc0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a91400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8c114ec0 con 0x7f3a8c10a6b0 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a83e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 0x7f3a8c1a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:41.977 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43192/0 (socket says 172.21.15.138:43192) 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 -- 172.21.15.138:0/4015044723 learned_addr learned my addr 172.21.15.138:0/4015044723 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 -- 172.21.15.138:0/4015044723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c10a6b0 msgr2=0x7f3a8c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 -- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 msgr2=0x7f3a8c1a0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 0x7f3a8c1a0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.964+0000 7f3a8a800640 1 -- 172.21.15.138:0/4015044723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a8c1a12e0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a8a800640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9f0 tx=0x7f3a7c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00cd80 con 0x7f3a8c104ca0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a8c1a15d0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a7c0054d0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c010690 con 0x7f3a8c104ca0 2024-08-20T01:08:41.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.965+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a8c1a7c90 con 0x7f3a8c104ca0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.966+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a8c105080 con 0x7f3a8c104ca0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.968+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f3a7c004e90 con 0x7f3a8c104ca0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.969+0000 7f3a83400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 0x7f3a700b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.970+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3a7c0e46d0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.970+0000 7f3a83e00640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 0x7f3a700b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.970+0000 7f3a83e00640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 0x7f3a700b61f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005ab0 tx=0x7f3a74005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:41.979 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:41.971+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f3a7c0ab6f0 con 0x7f3a8c104ca0 2024-08-20T01:08:41.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:41 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:08:42.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.204+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a8c19c360 con 0x7f3a8c104ca0 2024-08-20T01:08:42.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.205+0000 7f3a83400640 1 -- 172.21.15.138:0/4015044723 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f3a7c0aaff0 con 0x7f3a8c104ca0 2024-08-20T01:08:42.204 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:42.204 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"btime":"2024-08-20T01:08:32:577416+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:32.416368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:08:42.204 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 25 2024-08-20T01:08:42.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 msgr2=0x7f3a700b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 0x7f3a700b61f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a74005ab0 tx=0x7f3a74005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 msgr2=0x7f3a8c19b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9f0 tx=0x7f3a7c00edf0 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 shutdown_connections 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a700b3d30 0x7f3a700b61f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3a8c10b030 0x7f3a8c1a0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3a8c10a6b0 0x7f3a8c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 --2- 172.21.15.138:0/4015044723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3a8c104ca0 0x7f3a8c19b230 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:42.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.208+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 >> 172.21.15.138:0/4015044723 conn(0x7f3a8c100730 msgr2=0x7f3a8c102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:42.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.209+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 shutdown_connections 2024-08-20T01:08:42.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:42.209+0000 7f3a91400640 1 -- 172.21.15.138:0/4015044723 wait complete. 2024-08-20T01:08:43.020 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/4015044723' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:42 smithi167 ceph-mon[21965]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/4015044723' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:42 smithi142 ceph-mon[21799]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:43.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/4015044723' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:08:43.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:42 smithi138 ceph-mon[18076]: pgmap v380: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:43.388 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:44.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.022+0000 7f8765000640 1 -- 172.21.15.138:0/1792539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8760108e40 msgr2=0x7f876010f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:44.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.022+0000 7f8765000640 1 --2- 172.21.15.138:0/1792539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8760108e40 0x7f876010f640 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8758009900 tx=0x7f875802a410 comp rx=0 tx=0).stop 2024-08-20T01:08:44.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 -- 172.21.15.138:0/1792539536 shutdown_connections 2024-08-20T01:08:44.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 --2- 172.21.15.138:0/1792539536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8760108e40 0x7f876010f640 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 --2- 172.21.15.138:0/1792539536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87601084c0 0x7f8760108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 --2- 172.21.15.138:0/1792539536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760102ab0 0x7f8760102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 -- 172.21.15.138:0/1792539536 >> 172.21.15.138:0/1792539536 conn(0x7f87600fe500 msgr2=0x7f8760100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 -- 172.21.15.138:0/1792539536 shutdown_connections 2024-08-20T01:08:44.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.023+0000 7f8765000640 1 -- 172.21.15.138:0/1792539536 wait complete. 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 Processor -- start 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 -- start start 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 0x7f8760199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760108e40 0x7f876019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8760112b50 con 0x7f87601084c0 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87601129d0 con 0x7f8760102ab0 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.024+0000 7f8765000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8760112cd0 con 0x7f8760108e40 2024-08-20T01:08:44.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 0x7f8760199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43208/0 (socket says 172.21.15.138:43208) 2024-08-20T01:08:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 -- 172.21.15.138:0/719314304 learned_addr learned my addr 172.21.15.138:0/719314304 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875e800640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760108e40 0x7f876019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:44.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 -- 172.21.15.138:0/719314304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760108e40 msgr2=0x7f876019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760108e40 0x7f876019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 -- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 msgr2=0x7f8760199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 0x7f8760199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 -- 172.21.15.138:0/719314304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876019efd0 con 0x7f8760102ab0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875d400640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 0x7f8760199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.025+0000 7f875de00640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f875000eb90 tx=0x7f875000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.026+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750004d20 con 0x7f8760102ab0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.026+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876019f2c0 con 0x7f8760102ab0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.026+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f875000ee90 con 0x7f8760102ab0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.026+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750010690 con 0x7f8760102ab0 2024-08-20T01:08:44.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.026+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87601a5980 con 0x7f8760102ab0 2024-08-20T01:08:44.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.027+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8760102e90 con 0x7f8760102ab0 2024-08-20T01:08:44.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.029+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f8750010830 con 0x7f8760102ab0 2024-08-20T01:08:44.028 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.030+0000 7f8756a00640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 0x7f87440b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:44.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.031+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f87500e4a80 con 0x7f8760102ab0 2024-08-20T01:08:44.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.031+0000 7f875d400640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 0x7f87440b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:44.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.031+0000 7f875d400640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 0x7f87440b6170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8760107fa0 tx=0x7f874c00b410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:44.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.032+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8750055e20 con 0x7f8760102ab0 2024-08-20T01:08:44.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.252+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87601a5d10 con 0x7f8760102ab0 2024-08-20T01:08:44.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.253+0000 7f8756a00640 1 -- 172.21.15.138:0/719314304 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7f87500abaa0 con 0x7f8760102ab0 2024-08-20T01:08:44.252 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:44.252 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"btime":"2024-08-20T01:08:32:577416+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:32.416368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:08:44.252 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 25 2024-08-20T01:08:44.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 msgr2=0x7f87440b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:44.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 0x7f87440b6170 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8760107fa0 tx=0x7f874c00b410 comp rx=0 tx=0).stop 2024-08-20T01:08:44.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 msgr2=0x7f8760198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:44.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f875000eb90 tx=0x7f875000c6a0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 shutdown_connections 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87440b3cb0 0x7f87440b6170 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8760108e40 0x7f876019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601084c0 0x7f8760199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 --2- 172.21.15.138:0/719314304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8760102ab0 0x7f8760198f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 >> 172.21.15.138:0/719314304 conn(0x7f87600fe500 msgr2=0x7f87601008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 shutdown_connections 2024-08-20T01:08:44.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:44.256+0000 7f8765000640 1 -- 172.21.15.138:0/719314304 wait complete. 2024-08-20T01:08:44.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:44 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/719314304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:44.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:44 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/719314304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:44 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/719314304' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:45.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:08:45.433 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:45.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:45 smithi167 ceph-mon[21965]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:45.722 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:45 smithi138 ceph-mon[18076]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:45.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:45 smithi142 ceph-mon[21799]: pgmap v381: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-08-20T01:08:46.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 -- 172.21.15.138:0/2304100264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f822410a6b0 msgr2=0x7f822410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 --2- 172.21.15.138:0/2304100264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f822410a6b0 0x7f822410aaf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8210009900 tx=0x7f821002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 -- 172.21.15.138:0/2304100264 shutdown_connections 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 --2- 172.21.15.138:0/2304100264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f8224111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 --2- 172.21.15.138:0/2304100264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f822410a6b0 0x7f822410aaf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 --2- 172.21.15.138:0/2304100264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8224104ca0 0x7f8224105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.033+0000 7f8228e00640 1 -- 172.21.15.138:0/2304100264 >> 172.21.15.138:0/2304100264 conn(0x7f82241006f0 msgr2=0x7f8224102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.034+0000 7f8228e00640 1 -- 172.21.15.138:0/2304100264 shutdown_connections 2024-08-20T01:08:46.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.034+0000 7f8228e00640 1 -- 172.21.15.138:0/2304100264 wait complete. 2024-08-20T01:08:46.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.035+0000 7f8228e00640 1 Processor -- start 2024-08-20T01:08:46.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.035+0000 7f8228e00640 1 -- start start 2024-08-20T01:08:46.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:46.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f822410a6b0 0x7f822419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f82241a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8224114d40 con 0x7f8224104ca0 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8224114bc0 con 0x7f822410b030 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8228e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8224114ec0 con 0x7f822410a6b0 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8221e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8221e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:36352/0 (socket says 172.21.15.138:36352) 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8221e00640 1 -- 172.21.15.138:0/1703658695 learned_addr learned my addr 172.21.15.138:0/1703658695 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8222800640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f82241a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:46.036 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.036+0000 7f8221400640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f822410a6b0 0x7f822419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 -- 172.21.15.138:0/1703658695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f822410a6b0 msgr2=0x7f822419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f822410a6b0 0x7f822419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 -- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 msgr2=0x7f822419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 -- 172.21.15.138:0/1703658695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82241a1220 con 0x7f822410b030 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8221e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:46.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8222800640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f82241a0a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f821c009fd0 tx=0x7f821c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c00ce20 con 0x7f822410b030 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82241a1510 con 0x7f822410b030 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821c005020 con 0x7f822410b030 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.037+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821c010690 con 0x7f822410b030 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.038+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82241a7bd0 con 0x7f822410b030 2024-08-20T01:08:46.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.039+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8224105080 con 0x7f822410b030 2024-08-20T01:08:46.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.041+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f821c0054d0 con 0x7f822410b030 2024-08-20T01:08:46.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.042+0000 7f821aa00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 0x7f82080b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:46.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.043+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f821c0e4ea0 con 0x7f822410b030 2024-08-20T01:08:46.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.043+0000 7f8221e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 0x7f82080b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:46.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.043+0000 7f8221e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 0x7f82080b61f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8214011500 tx=0x7f8214005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:46.043 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.044+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f821c0abec0 con 0x7f822410b030 2024-08-20T01:08:46.226 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.227+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f822419c2a0 con 0x7f822410b030 2024-08-20T01:08:46.228 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.229+0000 7f821aa00640 1 -- 172.21.15.138:0/1703658695 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+25233 (secure 0 0 0) 0x7f821c0ab7c0 con 0x7f822410b030 2024-08-20T01:08:46.229 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:46.229 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":88,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:06:36.560026+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 msgr2=0x7f82080b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 0x7f82080b61f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8214011500 tx=0x7f8214005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 msgr2=0x7f82241a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f82241a0a50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f821c009fd0 tx=0x7f821c00ef10 comp rx=0 tx=0).stop 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 shutdown_connections 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82080b3d30 0x7f82080b61f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.231 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f822410b030 0x7f82241a0a50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f822410a6b0 0x7f822419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 --2- 172.21.15.138:0/1703658695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8224104ca0 0x7f822419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:46.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.231+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 >> 172.21.15.138:0/1703658695 conn(0x7f82241006f0 msgr2=0x7f8224102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:46.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.232+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 shutdown_connections 2024-08-20T01:08:46.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:46.232+0000 7f8228e00640 1 -- 172.21.15.138:0/1703658695 wait complete. 2024-08-20T01:08:46.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:46 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1703658695' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:46.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:46 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1703658695' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:46.756 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:46 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1703658695' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:08:47.111 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-08-20T01:08:47.480 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:47.676 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:47 smithi138 ceph-mon[18076]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-08-20T01:08:47.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:47 smithi167 ceph-mon[21965]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-08-20T01:08:47.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:47 smithi142 ceph-mon[21799]: pgmap v382: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 73 KiB/s wr, 17 op/s 2024-08-20T01:08:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 -- 172.21.15.138:0/1290017493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d4101d90 msgr2=0x7f77d410eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 --2- 172.21.15.138:0/1290017493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d4101d90 0x7f77d410eae0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002a480 comp rx=0 tx=0).stop 2024-08-20T01:08:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 -- 172.21.15.138:0/1290017493 shutdown_connections 2024-08-20T01:08:47.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 --2- 172.21.15.138:0/1290017493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d410f020 0x7f77d4111410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 --2- 172.21.15.138:0/1290017493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d4101d90 0x7f77d410eae0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 --2- 172.21.15.138:0/1290017493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101470 0x7f77d4101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 -- 172.21.15.138:0/1290017493 >> 172.21.15.138:0/1290017493 conn(0x7f77d40fd300 msgr2=0x7f77d40ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 -- 172.21.15.138:0/1290017493 shutdown_connections 2024-08-20T01:08:47.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.792+0000 7f77da200640 1 -- 172.21.15.138:0/1290017493 wait complete. 2024-08-20T01:08:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 Processor -- start 2024-08-20T01:08:47.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 -- start start 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4101470 0x7f77d419b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 0x7f77d419b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4114c20 con 0x7f77d410f020 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4114aa0 con 0x7f77d4101d90 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.793+0000 7f77da200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77d4114da0 con 0x7f77d4101470 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43826/0 (socket says 172.21.15.138:43826) 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 -- 172.21.15.138:0/4132367094 learned_addr learned my addr 172.21.15.138:0/4132367094 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d2a00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 0x7f77d419b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3400640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4101470 0x7f77d419b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 -- 172.21.15.138:0/4132367094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4101470 msgr2=0x7f77d419b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4101470 0x7f77d419b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 -- 172.21.15.138:0/4132367094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 msgr2=0x7f77d419b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 0x7f77d419b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 -- 172.21.15.138:0/4132367094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d41a11d0 con 0x7f77d410f020 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d2a00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 0x7f77d419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.794+0000 7f77d3e00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e970 tx=0x7f77cc00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.795+0000 7f77cbe00640 1 -- 172.21.15.138:0/4132367094 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00ccc0 con 0x7f77d410f020 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.795+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d41a14c0 con 0x7f77d410f020 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.795+0000 7f77cbe00640 1 -- 172.21.15.138:0/4132367094 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77cc00ce60 con 0x7f77d410f020 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.795+0000 7f77cbe00640 1 -- 172.21.15.138:0/4132367094 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc010430 con 0x7f77d410f020 2024-08-20T01:08:47.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.795+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a7b80 con 0x7f77d410f020 2024-08-20T01:08:47.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.799+0000 7f77dd0da740 1 Processor -- start 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.799+0000 7f77cbe00640 1 -- 172.21.15.138:0/4132367094 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f77cc004ff0 con 0x7f77d410f020 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 -- start start 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb77d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb865d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585fdaed210 con 0x5585fdb841e0 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585fdadc710 con 0x5585fdb78250 2024-08-20T01:08:47.799 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77dd0da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585fdae0750 con 0x5585fdb77930 2024-08-20T01:08:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51072/0 (socket says 172.21.15.138:51072) 2024-08-20T01:08:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb865d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77d3400640 1 -- 172.21.15.138:0/2644493203 learned_addr learned my addr 172.21.15.138:0/2644493203 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.800+0000 7f77d2a00640 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb77d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 -- 172.21.15.138:0/2644493203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 msgr2=0x5585fdb77d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb77d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 -- 172.21.15.138:0/2644493203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 msgr2=0x5585fdb865d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb865d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 -- 172.21.15.138:0/2644493203 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585fdaa8de0 con 0x5585fdb78250 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d2a00640 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb77d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77d3400640 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77c40039f0 tx=0x7f77c4003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa7ae59f4b2e353 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77cbe00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 0x7f77b80b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77bfe00640 1 -- 172.21.15.138:0/2644493203 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004b20 con 0x5585fdb78250 2024-08-20T01:08:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77cbe00640 1 -- 172.21.15.138:0/4132367094 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f77cc0e4710 con 0x7f77d410f020 2024-08-20T01:08:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.801+0000 7f77bfe00640 1 -- 172.21.15.138:0/2644493203 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c4004cc0 con 0x5585fdb78250 2024-08-20T01:08:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.802+0000 7f77bfe00640 1 -- 172.21.15.138:0/2644493203 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c40126b0 con 0x5585fdb78250 2024-08-20T01:08:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.802+0000 7f77d2a00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 0x7f77b80b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.802+0000 7f77dd0da740 1 -- 172.21.15.138:0/2644493203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 msgr2=0x5585fdb83ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.802+0000 7f77d2a00640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 0x7f77b80b6170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0040040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.802+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f77c40039f0 tx=0x7f77c4003eb0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 -- 172.21.15.138:0/2644493203 shutdown_connections 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb865d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb83ca0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2644493203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb77d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 -- 172.21.15.138:0/2644493203 >> 172.21.15.138:0/2644493203 conn(0x5585fdb75760 msgr2=0x5585fdb76900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 -- 172.21.15.138:0/2644493203 shutdown_connections 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 -- 172.21.15.138:0/2644493203 wait complete. 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 -- start start 2024-08-20T01:08:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.803+0000 7f77dd0da740 1 Processor -- start 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb77930 0x5585fdb839d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb80e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585fda15ab0 con 0x5585fdb77930 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585fda83140 con 0x5585fdb78250 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585fdb86ec0 con 0x5585fdb841e0 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77dd0da740 10 client.0 _ll_register_callbacks cb 0x5585fda965d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54510/0 (socket says 172.21.15.138:54510) 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77d3e00640 1 -- 172.21.15.138:0/2971541563 learned_addr learned my addr 172.21.15.138:0/2971541563 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77d3400640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb77930 0x5585fdb839d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.804+0000 7f77d2a00640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb80e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 -- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 msgr2=0x5585fdb80e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb80e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 -- 172.21.15.138:0/2971541563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb77930 msgr2=0x5585fdb839d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb77930 0x5585fdb839d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 -- 172.21.15.138:0/2971541563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585fda84170 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77d3e00640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0a8300 tx=0x7f77cc0e20d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=458900eeb5159e5d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0af450 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c4004cc0 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f77cc0e4370 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77cc0e4570 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77dd0da740 10 client.25444 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.805+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585fd83f790 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c4004cc0 con 0x5585fdb841e0 2024-08-20T01:08:47.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f77cc070080 con 0x5585fdb841e0 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 10 client.25444 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 20 client.25444 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdb93e50 0x5585fdb96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 4 client.25444 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585fdaef160 con 0x5585fdb93e50 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.806+0000 7f77d3400640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdb93e50 0x5585fdb96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.807+0000 7f77d3400640 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdb93e50 0x5585fdb96240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8621de621c0b52 server_cookie=85a7c78f7da0729f in_seq=0 out_seq=0 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.807+0000 7f77bcc00640 10 client.25444 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77bcc00640 1 -- 172.21.15.138:0/2971541563 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585fdaef160 con 0x5585fdb93e50 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77bcc00640 10 client.25444 handle_command_reply: tid=0 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 client.25444 shutdown 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 msgr2=0x5585fdb81350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0a8300 tx=0x7f77cc0e20d0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 shutdown_connections 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdb93e50 0x5585fdb96240 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb841e0 0x5585fdb81350 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb78250 0x5585fdb80e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77bcc00640 0 client.25444 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 --2- 172.21.15.138:0/2971541563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb77930 0x5585fdb839d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 >> 172.21.15.138:0/2971541563 conn(0x5585fdb75760 msgr2=0x5585fdb7b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 shutdown_connections 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 1 -- 172.21.15.138:0/2971541563 wait complete. 2024-08-20T01:08:47.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.808+0000 7f77dd0da740 20 client.25444 trim_cache size 0 max 16384 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.823+0000 7f77dd0da740 1 Processor -- start 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.823+0000 7f77dd0da740 1 -- start start 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 0x5585fdb80680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb9c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41b4530 con 0x5585fdb841e0 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41b1e00 con 0x5585fdb77930 2024-08-20T01:08:47.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77dd0da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41afc50 con 0x5585fdb78250 2024-08-20T01:08:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54518/0 (socket says 172.21.15.138:54518) 2024-08-20T01:08:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 -- 172.21.15.138:0/1989177780 learned_addr learned my addr 172.21.15.138:0/1989177780 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d3e00640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb9c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d3400640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 0x5585fdb80680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 -- 172.21.15.138:0/1989177780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 msgr2=0x5585fdb80680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 0x5585fdb80680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 -- 172.21.15.138:0/1989177780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 msgr2=0x5585fdb9c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb9c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d2a00640 1 -- 172.21.15.138:0/1989177780 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585fd8b8620 con 0x5585fdb78250 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.824+0000 7f77d3400640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 0x5585fdb80680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.825+0000 7f77d2a00640 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2db569530b569d2d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.825+0000 7f77bf400640 1 -- 172.21.15.138:0/1989177780 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0035040 con 0x5585fdb78250 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.825+0000 7f77bf400640 1 -- 172.21.15.138:0/1989177780 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c004f070 con 0x5585fdb78250 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.825+0000 7f77bf400640 1 -- 172.21.15.138:0/1989177780 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0053910 con 0x5585fdb78250 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- 172.21.15.138:0/1989177780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 msgr2=0x5585fdb9bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c0009260 comp rx=0 tx=0).stop 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- 172.21.15.138:0/1989177780 shutdown_connections 2024-08-20T01:08:47.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb841e0 0x5585fdb9c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb78250 0x5585fdb9bb50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1989177780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb77930 0x5585fdb80680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- 172.21.15.138:0/1989177780 >> 172.21.15.138:0/1989177780 conn(0x5585fdb75760 msgr2=0x5585fdb82c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- 172.21.15.138:0/1989177780 shutdown_connections 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- 172.21.15.138:0/1989177780 wait complete. 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.826+0000 7f77dd0da740 1 -- start start 2024-08-20T01:08:47.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 Processor -- start 2024-08-20T01:08:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb8c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 0x5585fdb89b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41b02c0 con 0x5585fdb78250 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41b1730 con 0x5585fdb841e0 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41b0990 con 0x5585fdb77930 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77dd0da740 10 client.0 _ll_register_callbacks cb 0x7f77cc0a6f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 0x5585fdb89b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.827+0000 7f77d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb8c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51104/0 (socket says 172.21.15.138:51104) 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 -- 172.21.15.138:0/1950850455 learned_addr learned my addr 172.21.15.138:0/1950850455 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 0x5585fdb89b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43856/0 (socket says 172.21.15.138:43856) 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 -- 172.21.15.138:0/1950850455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 msgr2=0x5585fdb8c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb8c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 -- 172.21.15.138:0/1950850455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 msgr2=0x5585fdb89b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 0x5585fdb89b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 -- 172.21.15.138:0/1950850455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585fdb8a710 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77d3e00640 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c6a0 tx=0x7f77cc0b3ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f23f0eac4ac361d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77bcc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0055f0 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c004f070 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.828+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f77cc0a7680 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f77cc090050 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 10 client.25472 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5585fdbfea10 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c004f070 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f77cc0a9d60 con 0x5585fdb841e0 2024-08-20T01:08:47.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 10 client.25472 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 10 client.25472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 20 client.25472 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdbff190 0x5585fdc015b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 4 client.25472 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.829+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585fdaeac00 con 0x5585fdbff190 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.830+0000 7f77d3400640 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdbff190 0x5585fdc015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.830+0000 7f77d3400640 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdbff190 0x5585fdc015b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5908179c606d4aa9 server_cookie=f4618f9e10b52a7b in_seq=0 out_seq=0 2024-08-20T01:08:47.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.830+0000 7f77bcc00640 10 client.25472 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77bcc00640 1 -- 172.21.15.138:0/1950850455 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585fdaeac00 con 0x5585fdbff190 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77bcc00640 10 client.25472 handle_command_reply: tid=0 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 client.25472 shutdown 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 msgr2=0x5585fdb8a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c6a0 tx=0x7f77cc0b3ee0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 shutdown_connections 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585fdbff190 0x5585fdc015b0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585fdb841e0 0x5585fdb8a070 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77bcc00640 0 client.25472 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585fdb78250 0x5585fdb89b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 --2- 172.21.15.138:0/1950850455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585fdb77930 0x5585fdb8c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.842+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 >> 172.21.15.138:0/1950850455 conn(0x5585fdb75760 msgr2=0x5585fdb8efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 shutdown_connections 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77dd0da740 1 -- 172.21.15.138:0/1950850455 wait complete. 2024-08-20T01:08:47.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77dd0da740 20 client.25472 trim_cache size 0 max 16384 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 msgr2=0x7f77b80b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 0x7f77b80b6170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f77c0000c00 tx=0x7f77c0040040 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 msgr2=0x7f77d41a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.843+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e970 tx=0x7f77cc00ed70 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 shutdown_connections 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77b80b3cb0 0x7f77b80b6170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77d410f020 0x7f77d41a0a00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77d4101d90 0x7f77d419b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 --2- 172.21.15.138:0/4132367094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f77d4101470 0x7f77d419b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 >> 172.21.15.138:0/4132367094 conn(0x7f77d40fd300 msgr2=0x7f77d4110360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 shutdown_connections 2024-08-20T01:08:47.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:08:47.844+0000 7f77da200640 1 -- 172.21.15.138:0/4132367094 wait complete. 2024-08-20T01:08:48.925 INFO:teuthology.orchestra.run.smithi138.stdout:{"run_dir":"/var/run/ceph"} 2024-08-20T01:08:48.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-08-20T01:08:48.926 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 7244 2024-08-20T01:08:48.927 DEBUG:tasks.quiescer:Yielding 2024-08-20T01:08:48.927 INFO:teuthology.run_tasks:Running task workunit... 2024-08-20T01:08:48.953 INFO:tasks.workunit:Pulling workunits from ref ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T01:08:48.953 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-08-20T01:08:48.953 INFO:tasks.workunit:timeout=3h 2024-08-20T01:08:48.954 INFO:tasks.workunit:cleanup=True 2024-08-20T01:08:48.954 DEBUG:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:48.955 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:08:48.995 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:48.995 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-08-20T01:08:48.995 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-08-20 01:06:32.416061637 +0000 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-08-20 01:06:32.416061637 +0000 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-08-20 01:08:31.508288295 +0000 2024-08-20T01:08:48.996 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-08-20 01:06:32.416061637 +0000 2024-08-20T01:08:48.996 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-08-20T01:08:48.996 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-08-20T01:08:49.100 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ef4275410d8483698016fe484b2a3c664080b4c1 2024-08-20T01:08:49.139 INFO:tasks.workunit.client.0.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-08-20T01:08:49.347 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:49 smithi138 ceph-mon[18076]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T01:08:49.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:49 smithi167 ceph-mon[21965]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 -- 172.21.15.138:0/4069079153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 msgr2=0x7fd7b0111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/4069079153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b0111820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8009ca0 tx=0x7fd7a802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 -- 172.21.15.138:0/4069079153 shutdown_connections 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/4069079153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b0111820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/4069079153 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 0x7fd7b010aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/4069079153 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b0105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 -- 172.21.15.138:0/4069079153 >> 172.21.15.138:0/4069079153 conn(0x7fd7b01006c0 msgr2=0x7fd7b0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 -- 172.21.15.138:0/4069079153 shutdown_connections 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.975+0000 7fd7b5c00640 1 -- 172.21.15.138:0/4069079153 wait complete. 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 Processor -- start 2024-08-20T01:08:49.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 -- start start 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 0x7fd7b019b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b01a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0114cf0 con 0x7fd7b010b020 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0114b70 con 0x7fd7b010a6a0 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.976+0000 7fd7b5c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0114e70 con 0x7fd7b0104c90 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54530/0 (socket says 172.21.15.138:54530) 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 -- 172.21.15.138:0/2340166561 learned_addr learned my addr 172.21.15.138:0/2340166561 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7af400640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b01a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7a6000640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 0x7fd7b019b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:49.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 -- 172.21.15.138:0/2340166561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 msgr2=0x7fd7b019b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 0x7fd7b019b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 -- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 msgr2=0x7fd7b01a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b01a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 -- 172.21.15.138:0/2340166561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b01a11b0 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.977+0000 7fd7aea00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000e9f0 tx=0x7fd7a000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.978+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a000cd80 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.978+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b01a14a0 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.978+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a00108e0 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.978+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0018980 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.978+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b01a7d30 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.979+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b0105070 con 0x7fd7b0104c90 2024-08-20T01:08:49.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.981+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fd7a0010430 con 0x7fd7b0104c90 2024-08-20T01:08:49.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.983+0000 7fd7a7e00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 0x7fd7940b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:49.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.983+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fd7a00e4a90 con 0x7fd7b0104c90 2024-08-20T01:08:49.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.983+0000 7fd7a6000640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 0x7fd7940b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:49.984 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.984+0000 7fd7a6000640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 0x7fd7940b6170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0106e10 tx=0x7fd798005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:49.985 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:49.984+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd7a00ab460 con 0x7fd7b0104c90 2024-08-20T01:08:49.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:49 smithi142 ceph-mon[21799]: pgmap v383: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-08-20T01:08:50.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.204+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7b019c230 con 0x7fd7b0104c90 2024-08-20T01:08:50.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.205+0000 7fd7a7e00640 1 -- 172.21.15.138:0/2340166561 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11932 (secure 0 0 0) 0x7fd7a00af9b0 con 0x7fd7b0104c90 2024-08-20T01:08:50.205 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:08:50.205 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":25,"btime":"2024-08-20T01:08:32:577416+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:32.416368+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:08:50.205 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 25 2024-08-20T01:08:50.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 msgr2=0x7fd7940b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:50.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 0x7fd7940b6170 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0106e10 tx=0x7fd798005f90 comp rx=0 tx=0).stop 2024-08-20T01:08:50.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 msgr2=0x7fd7b019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:50.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7a000e9f0 tx=0x7fd7a000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:50.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 shutdown_connections 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd7940b3cb0 0x7fd7940b6170 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd7b010b020 0x7fd7b01a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd7b010a6a0 0x7fd7b019b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 --2- 172.21.15.138:0/2340166561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd7b0104c90 0x7fd7b019b100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 >> 172.21.15.138:0/2340166561 conn(0x7fd7b01006c0 msgr2=0x7fd7b0100e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 shutdown_connections 2024-08-20T01:08:50.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:50.208+0000 7fd7b5c00640 1 -- 172.21.15.138:0/2340166561 wait complete. 2024-08-20T01:08:50.911 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2340166561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:50.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-08-20T01:05:27.648119+0000', 'modified': '2024-08-20T01:08:32.416368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24743, 'mds_1': 24706, 'mds_2': 24704}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6835/2224495590', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 2224495590}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 2224495590}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24679': {'gid': 24679, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6837/2270224611', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6836', 'nonce': 2270224611}, {'type': 'v1', 'addr': '172.21.15.167:6837', 'nonce': 2270224611}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24694': {'gid': 24694, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.167:6839/2113916191', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.167:6838', 'nonce': 2113916191}, {'type': 'v1', 'addr': '172.21.15.167:6839', 'nonce': 2113916191}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24704': {'gid': 24704, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 21, 'addr': '172.21.15.142:6837/1103910039', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 1103910039}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 1103910039}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24706': {'gid': 24706, 'name': 'g', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 19, 'addr': '172.21.15.138:6839/1714719372', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.138:6838', 'nonce': 1714719372}, {'type': 'v1', 'addr': '172.21.15.138:6839', 'nonce': 1714719372}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24743': {'gid': 24743, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.142:6839/80285616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 80285616}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 80285616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24743, 'qdb_cluster': [24706, 24743, 24704]} 2024-08-20T01:08:50.950 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-08-20T01:08:50.950 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.h status 2024-08-20T01:08:50.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:50 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2340166561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:50 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2340166561' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:08:51.361 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:51.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:51 smithi138 ceph-mon[18076]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T01:08:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:51 smithi167 ceph-mon[21965]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T01:08:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:51 smithi142 ceph-mon[21799]: pgmap v384: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-08-20T01:08:52.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 -- 172.21.15.138:0/3894248869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10a520 msgr2=0x7f423c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 --2- 172.21.15.138:0/3894248869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10a520 0x7f423c10a960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4228009900 tx=0x7f422802a410 comp rx=0 tx=0).stop 2024-08-20T01:08:52.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 -- 172.21.15.138:0/3894248869 shutdown_connections 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 --2- 172.21.15.138:0/3894248869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10aea0 0x7f423c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 --2- 172.21.15.138:0/3894248869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10a520 0x7f423c10a960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 --2- 172.21.15.138:0/3894248869 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 -- 172.21.15.138:0/3894248869 >> 172.21.15.138:0/3894248869 conn(0x7f423c100540 msgr2=0x7f423c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 -- 172.21.15.138:0/3894248869 shutdown_connections 2024-08-20T01:08:52.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.017+0000 7f423b200640 1 -- 172.21.15.138:0/3894248869 wait complete. 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.018+0000 7f423b200640 1 Processor -- start 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.018+0000 7f423b200640 1 -- start start 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.018+0000 7f423b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.018+0000 7f423b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 0x7f423c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.018+0000 7f423b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10aea0 0x7f423c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f423b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114b60 con 0x7f423c10aea0 2024-08-20T01:08:52.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f423b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f423c1149e0 con 0x7f423c10a520 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f423b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f423c114ce0 con 0x7f423c104b10 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f423a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10aea0 0x7f423c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 0x7f423c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 0x7f423c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51132/0 (socket says 172.21.15.138:51132) 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239400640 1 -- 172.21.15.138:0/2745995049 learned_addr learned my addr 172.21.15.138:0/2745995049 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:52.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54560/0 (socket says 172.21.15.138:54560) 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 -- 172.21.15.138:0/2745995049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 msgr2=0x7f423c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 0x7f423c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 -- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10aea0 msgr2=0x7f423c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10aea0 0x7f423c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.019+0000 7f4239e00640 1 -- 172.21.15.138:0/2745995049 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f423c1a0fe0 con 0x7f423c104b10 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f4239e00640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f422c00e9f0 tx=0x7f422c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f4232a00640 1 -- 172.21.15.138:0/2745995049 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c00ccc0 con 0x7f423c104b10 2024-08-20T01:08:52.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f4232a00640 1 -- 172.21.15.138:0/2745995049 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c005020 con 0x7f423c104b10 2024-08-20T01:08:52.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f423c1a12d0 con 0x7f423c104b10 2024-08-20T01:08:52.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f4232a00640 1 -- 172.21.15.138:0/2745995049 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c004a00 con 0x7f423c104b10 2024-08-20T01:08:52.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.020+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f423c1a7990 con 0x7f423c104b10 2024-08-20T01:08:52.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.023+0000 7f4243640740 1 Processor -- start 2024-08-20T01:08:52.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.023+0000 7f4243640740 1 -- start start 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b752a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607b7494840 con 0x5607b752aa20 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607b7483d40 con 0x5607b751e170 2024-08-20T01:08:52.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4243640740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607b7487d80 con 0x5607b751ea90 2024-08-20T01:08:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4232a00640 1 -- 172.21.15.138:0/2745995049 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f422c004ba0 con 0x7f423c104b10 2024-08-20T01:08:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4239400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4239400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43902/0 (socket says 172.21.15.138:43902) 2024-08-20T01:08:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4239400640 1 -- 172.21.15.138:0/3595670004 learned_addr learned my addr 172.21.15.138:0/3595670004 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:52.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f423a800640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4239e00640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b752a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.024+0000 7f4239400640 1 -- 172.21.15.138:0/3595670004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 msgr2=0x5607b752a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.025 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4239400640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b752a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4239400640 1 -- 172.21.15.138:0/3595670004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 msgr2=0x5607b751e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4239400640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4239400640 1 -- 172.21.15.138:0/3595670004 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b742a1c0 con 0x5607b752aa20 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f423a800640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4239400640 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4228009a20 tx=0x7f4228031b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7e7b0ae987894b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4227000640 1 -- 172.21.15.138:0/3595670004 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228035040 con 0x5607b752aa20 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4227000640 1 -- 172.21.15.138:0/3595670004 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4228031760 con 0x5607b752aa20 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.025+0000 7f4227000640 1 -- 172.21.15.138:0/3595670004 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42280047a0 con 0x5607b752aa20 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4232a00640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 0x7f42200b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 -- 172.21.15.138:0/3595670004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 msgr2=0x5607b752ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4228009a20 tx=0x7f4228031b10 comp rx=0 tx=0).stop 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4232a00640 1 -- 172.21.15.138:0/2745995049 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f422c0e41c0 con 0x7f423c104b10 2024-08-20T01:08:52.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 -- 172.21.15.138:0/3595670004 shutdown_connections 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b752aa20 0x5607b752ce10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b752a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 --2- 172.21.15.138:0/3595670004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 -- 172.21.15.138:0/3595670004 >> 172.21.15.138:0/3595670004 conn(0x5607b751bfa0 msgr2=0x5607b751d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f423a800640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 0x7f42200b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 -- 172.21.15.138:0/3595670004 shutdown_connections 2024-08-20T01:08:52.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.026+0000 7f4243640740 1 -- 172.21.15.138:0/3595670004 wait complete. 2024-08-20T01:08:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f4243640740 1 -- start start 2024-08-20T01:08:52.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f423a800640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 0x7f42200b62a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4234005fd0 tx=0x7f4234009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f4243640740 1 Processor -- start 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f4243640740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f4243640740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7527930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.027+0000 7f4243640740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 0x5607b75369a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4243640740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607b72e65f0 con 0x5607b751e170 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4243640740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607b741f170 con 0x5607b751ea90 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4243640740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607b752d670 con 0x5607b7528020 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4243640740 10 client.0 _ll_register_callbacks cb 0x5607b747e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43916/0 (socket says 172.21.15.138:43916) 2024-08-20T01:08:52.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 -- 172.21.15.138:0/327722447 learned_addr learned my addr 172.21.15.138:0/327722447 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239400640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7527930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f423a800640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 0x5607b75369a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 -- 172.21.15.138:0/327722447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 msgr2=0x5607b75369a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 0x5607b75369a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 -- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 msgr2=0x5607b7527930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7527930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 -- 172.21.15.138:0/327722447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b71e3540 con 0x5607b751e170 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f423a800640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 0x5607b75369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f4239e00640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c6a0 tx=0x7f422c0aa4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b719b83316bacba server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.028+0000 7f421b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c01c030 con 0x5607b751e170 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f422c0920b0 con 0x5607b751e170 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4228031760 con 0x5607b751e170 2024-08-20T01:08:52.031 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f422c0ab1e0 con 0x5607b751e170 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f4243640740 10 client.15459 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607b7539530 con 0x5607b751e170 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4228031760 con 0x5607b751e170 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f422c0e18f0 con 0x5607b751e170 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f4243640740 10 client.15459 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.029+0000 7f4243640740 10 client.15459 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4243640740 20 client.15459 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b753bab0 0x5607b753cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4243640740 4 client.15459 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607b7496790 con 0x5607b753bab0 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4239400640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b753bab0 0x5607b753cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f4239400640 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b753bab0 0x5607b753cf20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e71713239b7a71f0 server_cookie=166cbec6210ecfe3 in_seq=0 out_seq=0 2024-08-20T01:08:52.032 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.030+0000 7f421b400640 10 client.15459 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:52.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f421b400640 1 -- 172.21.15.138:0/327722447 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607b7496790 con 0x5607b753bab0 2024-08-20T01:08:52.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f421b400640 10 client.15459 handle_command_reply: tid=0 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 client.15459 shutdown 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 msgr2=0x5607b752a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f422c00c6a0 tx=0x7f422c0aa4f0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 shutdown_connections 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b753bab0 0x5607b753cf20 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b7528020 0x5607b75369a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7527930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f421b400640 0 client.15459 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 --2- 172.21.15.138:0/327722447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b751e170 0x5607b752a4c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.031+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 >> 172.21.15.138:0/327722447 conn(0x5607b751bfa0 msgr2=0x5607b7521b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.032+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 shutdown_connections 2024-08-20T01:08:52.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.032+0000 7f4243640740 1 -- 172.21.15.138:0/327722447 wait complete. 2024-08-20T01:08:52.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.032+0000 7f4243640740 20 client.15459 trim_cache size 0 max 16384 2024-08-20T01:08:52.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 Processor -- start 2024-08-20T01:08:52.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 -- start start 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b7526e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607b753b260 con 0x5607b7528020 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607b753b930 con 0x5607b751e170 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.050+0000 7f4243640740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607b753c000 con 0x5607b751ea90 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f423a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f423a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43920/0 (socket says 172.21.15.138:43920) 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f423a800640 1 -- 172.21.15.138:0/2910982386 learned_addr learned my addr 172.21.15.138:0/2910982386 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239400640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b7526e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 -- 172.21.15.138:0/2910982386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 msgr2=0x5607b7526e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b7526e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 -- 172.21.15.138:0/2910982386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 msgr2=0x5607b7536910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 -- 172.21.15.138:0/2910982386 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607b751ca20 con 0x5607b751e170 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f423a800640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.051+0000 7f4239e00640 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751d700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f422c004770 tx=0x7f422c005700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdcb14b2e84ef853 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4226600640 1 -- 172.21.15.138:0/2910982386 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c01c070 con 0x5607b751e170 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4226600640 1 -- 172.21.15.138:0/2910982386 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c0c1050 con 0x5607b751e170 2024-08-20T01:08:52.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4226600640 1 -- 172.21.15.138:0/2910982386 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f422c0e18f0 con 0x5607b751e170 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 -- 172.21.15.138:0/2910982386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 msgr2=0x5607b751d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751d700 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f422c004770 tx=0x7f422c005700 comp rx=0 tx=0).stop 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 -- 172.21.15.138:0/2910982386 shutdown_connections 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b7536910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751ea90 0x5607b7526e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 --2- 172.21.15.138:0/2910982386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751e170 0x5607b751d700 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 -- 172.21.15.138:0/2910982386 >> 172.21.15.138:0/2910982386 conn(0x5607b751bfa0 msgr2=0x5607b751cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.052+0000 7f4243640740 1 -- 172.21.15.138:0/2910982386 shutdown_connections 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.053+0000 7f4243640740 1 -- 172.21.15.138:0/2910982386 wait complete. 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.053+0000 7f4243640740 1 -- start start 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.053+0000 7f4243640740 1 Processor -- start 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7541890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b75a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607b753cda0 con 0x5607b7528020 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607b753c6d0 con 0x5607b751ea90 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607b753d470 con 0x5607b751e170 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4243640740 10 client.0 _ll_register_callbacks cb 0x7f422c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:52.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f423a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b75a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54596/0 (socket says 172.21.15.138:54596) 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7541890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239e00640 1 -- 172.21.15.138:0/785845956 learned_addr learned my addr 172.21.15.138:0/785845956 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 -- 172.21.15.138:0/785845956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 msgr2=0x5607b7541350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 -- 172.21.15.138:0/785845956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 msgr2=0x5607b75a51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b75a51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239400640 1 -- 172.21.15.138:0/785845956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607b7542180 con 0x5607b751ea90 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.054+0000 7f4239e00640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f4239400640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7541890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5607b751e0d0 tx=0x7f42280053c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca1754085b0700 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f421b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4228035040 con 0x5607b751ea90 2024-08-20T01:08:52.057 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f422c0c1050 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4228004040 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4228060050 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f4243640740 10 client.25496 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607b75a6830 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.055+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f422c0c1050 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4228003ca0 con 0x5607b751ea90 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 10 client.25496 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 10 client.25496 resolve_mds: resolved name 'h' to daemon mds.h 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 20 client.25496 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b75aaf50 0x5607b75a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 4 client.25496 mds_command: new command op to 24704 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 --> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] -- command(tid 0: {"prefix": "status"}) -- 0x5607b7492230 con 0x5607b75aaf50 2024-08-20T01:08:52.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.056+0000 7f4239e00640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b75aaf50 0x5607b75a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:52.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4239e00640 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b75aaf50 0x5607b75a9c60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ffac5185158ab97c server_cookie=b33251130c0aed7e in_seq=0 out_seq=0 2024-08-20T01:08:52.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f421b400640 10 client.25496 ms_handle_connect on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f421b400640 1 -- 172.21.15.138:0/785845956 <== mds.2 v2:172.21.15.142:6836/1103910039 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5607b7492230 con 0x5607b75aaf50 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f421b400640 10 client.25496 handle_command_reply: tid=0 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 client.25496 shutdown 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 msgr2=0x5607b7541890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7541890 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x5607b751e0d0 tx=0x7f42280053c0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 shutdown_connections 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:6836/1103910039,v1:172.21.15.142:6837/1103910039] conn(0x5607b75aaf50 0x5607b75a9c60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5607b7528020 0x5607b75a51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5607b751ea90 0x5607b7541890 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f421b400640 0 client.25496 ms_handle_reset on v2:172.21.15.142:6836/1103910039 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 --2- 172.21.15.138:0/785845956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5607b751e170 0x5607b7541350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 >> 172.21.15.138:0/785845956 conn(0x5607b751bfa0 msgr2=0x5607b7528f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 shutdown_connections 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 1 -- 172.21.15.138:0/785845956 wait complete. 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.057+0000 7f4243640740 20 client.25496 trim_cache size 0 max 16384 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 2, 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24704, 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 152.456764271, 2024-08-20T01:08:52.061 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 25, 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 88, 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 230.92825242500001 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 msgr2=0x7f42200b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 0x7f42200b62a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4234005fd0 tx=0x7f4234009500 comp rx=0 tx=0).stop 2024-08-20T01:08:52.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 msgr2=0x7f423c19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f422c00e9f0 tx=0x7f422c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 shutdown_connections 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f42200b3de0 0x7f42200b62a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f423c10aea0 0x7f423c1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f423c10a520 0x7f423c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 --2- 172.21.15.138:0/2745995049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f423c104b10 0x7f423c19af30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 >> 172.21.15.138:0/2745995049 conn(0x7f423c100540 msgr2=0x7f423c102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 shutdown_connections 2024-08-20T01:08:52.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:52.058+0000 7f423b200640 1 -- 172.21.15.138:0/2745995049 wait complete. 2024-08-20T01:08:52.936 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.g status 2024-08-20T01:08:53.306 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:53.788 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:53 smithi138 ceph-mon[18076]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-08-20T01:08:53.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.902+0000 7f4e92e00640 1 -- 172.21.15.138:0/2584982275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 msgr2=0x7f4e8c102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.902 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.902+0000 7f4e92e00640 1 --2- 172.21.15.138:0/2584982275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c102cf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009900 tx=0x7f4e8002a410 comp rx=0 tx=0).stop 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 -- 172.21.15.138:0/2584982275 shutdown_connections 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 --2- 172.21.15.138:0/2584982275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c10f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 --2- 172.21.15.138:0/2584982275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108320 0x7f4e8c108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 --2- 172.21.15.138:0/2584982275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c102cf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 -- 172.21.15.138:0/2584982275 >> 172.21.15.138:0/2584982275 conn(0x7f4e8c0fe3c0 msgr2=0x7f4e8c1007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 -- 172.21.15.138:0/2584982275 shutdown_connections 2024-08-20T01:08:53.903 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.903+0000 7f4e92e00640 1 -- 172.21.15.138:0/2584982275 wait complete. 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.904+0000 7f4e92e00640 1 Processor -- start 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.904+0000 7f4e92e00640 1 -- start start 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108320 0x7f4e8c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c1129b0 con 0x7f4e8c102910 2024-08-20T01:08:53.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c112830 con 0x7f4e8c108ca0 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e8c112b30 con 0x7f4e8c108320 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51174/0 (socket says 172.21.15.138:51174) 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92400640 1 -- 172.21.15.138:0/3131388590 learned_addr learned my addr 172.21.15.138:0/3131388590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:53.905 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91000640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108320 0x7f4e8c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 -- 172.21.15.138:0/3131388590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108320 msgr2=0x7f4e8c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108320 0x7f4e8c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 -- 172.21.15.138:0/3131388590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 msgr2=0x7f4e8c19e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e91a00640 1 -- 172.21.15.138:0/3131388590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8c19edf0 con 0x7f4e8c102910 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.905+0000 7f4e92400640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e91a00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c198d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002aa00 tx=0x7f4e80003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e8a000640 1 -- 172.21.15.138:0/3131388590 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8003f070 con 0x7f4e8c102910 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e8a000640 1 -- 172.21.15.138:0/3131388590 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80031b50 con 0x7f4e8c102910 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e8a000640 1 -- 172.21.15.138:0/3131388590 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80035030 con 0x7f4e8c102910 2024-08-20T01:08:53.906 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e8c19f080 con 0x7f4e8c102910 2024-08-20T01:08:53.907 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.906+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e8c1a57a0 con 0x7f4e8c102910 2024-08-20T01:08:53.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.910+0000 7f4e96e82740 1 Processor -- start 2024-08-20T01:08:53.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.910+0000 7f4e96e82740 1 -- start start 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e8a000640 1 -- 172.21.15.138:0/3131388590 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f4e80042070 con 0x7f4e8c102910 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920897960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555920809840 con 0x555920897ea0 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559207f8d40 con 0x5559208a3e30 2024-08-20T01:08:53.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e96e82740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559207fcd80 con 0x555920897580 2024-08-20T01:08:53.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43944/0 (socket says 172.21.15.138:43944) 2024-08-20T01:08:53.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e91a00640 1 -- 172.21.15.138:0/2385131497 learned_addr learned my addr 172.21.15.138:0/2385131497 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:53.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e91000640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.911+0000 7f4e92400640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920897960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 -- 172.21.15.138:0/2385131497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 msgr2=0x555920897960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920897960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 -- 172.21.15.138:0/2385131497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 msgr2=0x5559208a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 -- 172.21.15.138:0/2385131497 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592079f1c0 con 0x555920897ea0 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91000640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:53.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.912+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e8010c5a0 tx=0x7f4e8010c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60550b335d7a43a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e8a000640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 0x7f4e740b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e7ea00640 1 -- 172.21.15.138:0/2385131497 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e801060d0 con 0x555920897ea0 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e7ea00640 1 -- 172.21.15.138:0/2385131497 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80111030 con 0x555920897ea0 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e7ea00640 1 -- 172.21.15.138:0/2385131497 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800d9670 con 0x555920897ea0 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e8a000640 1 -- 172.21.15.138:0/3131388590 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4e80104460 con 0x7f4e8c102910 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.913+0000 7f4e92400640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 0x7f4e740b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 -- 172.21.15.138:0/2385131497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 msgr2=0x5559208a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4e8010c5a0 tx=0x7f4e8010c690 comp rx=0 tx=0).stop 2024-08-20T01:08:53.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e92400640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 0x7f4e740b62a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005c40 tx=0x7f4e84009760 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 -- 172.21.15.138:0/2385131497 shutdown_connections 2024-08-20T01:08:53.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897ea0 0x5559208a38f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2385131497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920897960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 -- 172.21.15.138:0/2385131497 >> 172.21.15.138:0/2385131497 conn(0x5559208953b0 msgr2=0x555920896550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.914+0000 7f4e96e82740 1 -- 172.21.15.138:0/2385131497 shutdown_connections 2024-08-20T01:08:53.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.915+0000 7f4e96e82740 1 -- 172.21.15.138:0/2385131497 wait complete. 2024-08-20T01:08:53.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.915+0000 7f4e96e82740 1 -- start start 2024-08-20T01:08:53.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.915+0000 7f4e96e82740 1 Processor -- start 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897ea0 0x5559208a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55592065b5f0 con 0x555920897580 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555920794170 con 0x5559208a3e30 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559208a6870 con 0x555920897ea0 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e96e82740 10 client.0 _ll_register_callbacks cb 0x5559207f38b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43958/0 (socket says 172.21.15.138:43958) 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e91a00640 1 -- 172.21.15.138:0/2006022669 learned_addr learned my addr 172.21.15.138:0/2006022669 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e91000640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897ea0 0x5559208a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e92400640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.916+0000 7f4e91a00640 1 -- 172.21.15.138:0/2006022669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897ea0 msgr2=0x5559208a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897ea0 0x5559208a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e91a00640 1 -- 172.21.15.138:0/2006022669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 msgr2=0x5559208a0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e91a00640 1 -- 172.21.15.138:0/2006022669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555920558540 con 0x555920897580 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e92400640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e91a00640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002a930 tx=0x7f4e8002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c784293fcd52c7ff server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e73400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:53.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800d87b0 con 0x555920897580 2024-08-20T01:08:53.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4e80104150 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.917+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80111030 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e80049020 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 10 client.15477 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559208b2000 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80111030 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4e8004c060 con 0x555920897580 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 10 client.15477 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 10 client.15477 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 20 client.15477 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5559208b4430 0x5559208b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 4 client.15477 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.918+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55592080b790 con 0x5559208b4430 2024-08-20T01:08:53.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.919+0000 7f4e91000640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5559208b4430 0x5559208b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.919+0000 7f4e91000640 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5559208b4430 0x5559208b58a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd7c6bf071361493 server_cookie=f300891a0453bae9 in_seq=0 out_seq=0 2024-08-20T01:08:53.920 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.919+0000 7f4e73400640 10 client.15477 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.920+0000 7f4e73400640 1 -- 172.21.15.138:0/2006022669 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55592080b790 con 0x5559208b4430 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.920+0000 7f4e73400640 10 client.15477 handle_command_reply: tid=0 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.920+0000 7f4e96e82740 1 client.15477 shutdown 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.920+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 msgr2=0x5559208a35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.920+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002a930 tx=0x7f4e8002a410 comp rx=0 tx=0).stop 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 shutdown_connections 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5559208a3e30 0x5559208a0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x5559208b4430 0x5559208b58a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897ea0 0x5559208a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 --2- 172.21.15.138:0/2006022669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555920897580 0x5559208a35b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 >> 172.21.15.138:0/2006022669 conn(0x5559208953b0 msgr2=0x55592089af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e73400640 0 client.15477 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:53.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 shutdown_connections 2024-08-20T01:08:53.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 1 -- 172.21.15.138:0/2006022669 wait complete. 2024-08-20T01:08:53.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.921+0000 7f4e96e82740 20 client.15477 trim_cache size 0 max 16384 2024-08-20T01:08:53.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 Processor -- start 2024-08-20T01:08:53.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 -- start start 2024-08-20T01:08:53.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920896b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x55592089dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5559208b3be0 con 0x5559208a3e30 2024-08-20T01:08:53.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559208b42b0 con 0x555920897ea0 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e96e82740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559208b4980 con 0x555920897580 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43964/0 (socket says 172.21.15.138:43964) 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e92400640 1 -- 172.21.15.138:0/1254629263 learned_addr learned my addr 172.21.15.138:0/1254629263 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.940+0000 7f4e91a00640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920896b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91000640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x55592089dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 -- 172.21.15.138:0/1254629263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 msgr2=0x55592089dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x55592089dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 -- 172.21.15.138:0/1254629263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 msgr2=0x55592089e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 -- 172.21.15.138:0/1254629263 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555920895e30 con 0x555920897580 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e92400640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:53.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e91a00640 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920896b60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cab30 tx=0x7f4e800ca610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4baf2abf8383ae16 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.941+0000 7f4e7e000640 1 -- 172.21.15.138:0/1254629263 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800d89b0 con 0x555920897580 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e7e000640 1 -- 172.21.15.138:0/1254629263 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80111070 con 0x555920897580 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e7e000640 1 -- 172.21.15.138:0/1254629263 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800d8bf0 con 0x555920897580 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 -- 172.21.15.138:0/1254629263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 msgr2=0x555920896b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920896b60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cab30 tx=0x7f4e800ca610 comp rx=0 tx=0).stop 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 -- 172.21.15.138:0/1254629263 shutdown_connections 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x55592089e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x55592089dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 --2- 172.21.15.138:0/1254629263 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x555920896b60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.942+0000 7f4e96e82740 1 -- 172.21.15.138:0/1254629263 >> 172.21.15.138:0/1254629263 conn(0x5559208953b0 msgr2=0x555920896450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.943+0000 7f4e96e82740 1 -- 172.21.15.138:0/1254629263 shutdown_connections 2024-08-20T01:08:53.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.943+0000 7f4e96e82740 1 -- 172.21.15.138:0/1254629263 wait complete. 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.943+0000 7f4e96e82740 1 -- start start 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.943+0000 7f4e96e82740 1 Processor -- start 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x5559208b7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x5559208b7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x5559208bc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5559208b5720 con 0x5559208a3e30 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5559208b5050 con 0x555920897ea0 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5559208b5df0 con 0x555920897580 2024-08-20T01:08:53.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e96e82740 10 client.0 _ll_register_callbacks cb 0x7f4e780097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x5559208b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x5559208bc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x5559208b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x5559208b7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54650/0 (socket says 172.21.15.138:54650) 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e92400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x5559208bc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43966/0 (socket says 172.21.15.138:43966) 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91a00640 1 -- 172.21.15.138:0/365828671 learned_addr learned my addr 172.21.15.138:0/365828671 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 -- 172.21.15.138:0/365828671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 msgr2=0x5559208b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x5559208b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 -- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 msgr2=0x5559208bc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x5559208bc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.944+0000 7f4e91000640 1 -- 172.21.15.138:0/365828671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559208bcd20 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.945+0000 7f4e91000640 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x5559208b7610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5559208b6a20 tx=0x7f4e78004100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c2f3d3031f8949e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.945+0000 7f4e73400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.945+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e78003cc0 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.945+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80111070 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.945+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4e78005020 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e7802f050 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e96e82740 10 client.25514 fetch_fsmap learned FSMap version 25 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559208be1e0 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80111070 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f4e78020410 con 0x555920897ea0 2024-08-20T01:08:53.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.946+0000 7f4e96e82740 10 client.25514 fetch_fsmap finished waiting for FSMap version 25 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e96e82740 10 client.25514 resolve_mds: resolved name 'g' to daemon mds.g 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e96e82740 20 client.25514 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x555920922f00 0x555920921c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e96e82740 4 client.25514 mds_command: new command op to 24706 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 --> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] -- command(tid 0: {"prefix": "status"}) -- 0x555920807230 con 0x555920922f00 2024-08-20T01:08:53.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e91a00640 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x555920922f00 0x555920921c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e91a00640 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x555920922f00 0x555920921c10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=865d8d14b63e9acb server_cookie=a4b6ff70419edc85 in_seq=0 out_seq=0 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.947+0000 7f4e73400640 10 client.25514 ms_handle_connect on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e73400640 1 -- 172.21.15.138:0/365828671 <== mds.1 v2:172.21.15.138:6838/1714719372 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555920807230 con 0x555920922f00 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e73400640 10 client.25514 handle_command_reply: tid=0 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 client.25514 shutdown 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 msgr2=0x5559208b7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x5559208b7610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x5559208b6a20 tx=0x7f4e78004100 comp rx=0 tx=0).stop 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 shutdown_connections 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5559208a3e30 0x5559208bc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555920897ea0 0x5559208b7610 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.138:6838/1714719372,v1:172.21.15.138:6839/1714719372] conn(0x555920922f00 0x555920921c10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 --2- 172.21.15.138:0/365828671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555920897580 0x5559208b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e73400640 0 client.25514 ms_handle_reset on v2:172.21.15.138:6838/1714719372 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 >> 172.21.15.138:0/365828671 conn(0x5559208953b0 msgr2=0x55592088f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 shutdown_connections 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.948+0000 7f4e96e82740 1 -- 172.21.15.138:0/365828671 wait complete. 2024-08-20T01:08:53.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e96e82740 20 client.25514 trim_cache size 0 max 16384 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 1, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24706, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 155.35690927499999, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 25, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 88, 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 226.419505305 2024-08-20T01:08:53.951 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:53.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 msgr2=0x7f4e740b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 0x7f4e740b62a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4e84005c40 tx=0x7f4e84009760 comp rx=0 tx=0).stop 2024-08-20T01:08:53.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 msgr2=0x7f4e8c198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:53.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c198d40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002aa00 tx=0x7f4e80003b90 comp rx=0 tx=0).stop 2024-08-20T01:08:53.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.949+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 shutdown_connections 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e740b3de0 0x7f4e740b62a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e8c108ca0 0x7f4e8c19e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e8c108320 0x7f4e8c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 --2- 172.21.15.138:0/3131388590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e8c102910 0x7f4e8c198d40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 >> 172.21.15.138:0/3131388590 conn(0x7f4e8c0fe3c0 msgr2=0x7f4e8c0ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 shutdown_connections 2024-08-20T01:08:53.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:53.950+0000 7f4e92e00640 1 -- 172.21.15.138:0/3131388590 wait complete. 2024-08-20T01:08:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:53 smithi167 ceph-mon[21965]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-08-20T01:08:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:53 smithi142 ceph-mon[21799]: pgmap v385: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-08-20T01:08:54.885 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph tell mds.b status 2024-08-20T01:08:55.282 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:08:55.676 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:55 smithi138 ceph-mon[18076]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-08-20T01:08:55.676 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:55 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:55.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 -- 172.21.15.138:0/1832306148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 msgr2=0x7fe03c105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 --2- 172.21.15.138:0/1832306148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c105090 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe02c009900 tx=0x7fe02c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 -- 172.21.15.138:0/1832306148 shutdown_connections 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 --2- 172.21.15.138:0/1832306148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 --2- 172.21.15.138:0/1832306148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 0x7fe03c10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 --2- 172.21.15.138:0/1832306148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c105090 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 -- 172.21.15.138:0/1832306148 >> 172.21.15.138:0/1832306148 conn(0x7fe03c100720 msgr2=0x7fe03c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 -- 172.21.15.138:0/1832306148 shutdown_connections 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.901+0000 7fe042800640 1 -- 172.21.15.138:0/1832306148 wait complete. 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.902+0000 7fe042800640 1 Processor -- start 2024-08-20T01:08:55.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 -- start start 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 0x7fe03c19b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c114d10 con 0x7fe03c104cb0 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c114b90 con 0x7fe03c10b040 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe042800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe03c114e90 con 0x7fe03c10a6c0 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe03b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe03b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c19b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43986/0 (socket says 172.21.15.138:43986) 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe03b400640 1 -- 172.21.15.138:0/16796972 learned_addr learned my addr 172.21.15.138:0/16796972 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.903+0000 7fe03be00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.912 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03aa00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 0x7fe03c19b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 -- 172.21.15.138:0/16796972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 msgr2=0x7fe03c19b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 0x7fe03c19b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 -- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 msgr2=0x7fe03c19b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c19b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 -- 172.21.15.138:0/16796972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe03c1a1230 con 0x7fe03c10b040 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.904+0000 7fe03be00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c1a0a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe03400da10 tx=0x7fe03400ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.905+0000 7fe033e00640 1 -- 172.21.15.138:0/16796972 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03400bd70 con 0x7fe03c10b040 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.905+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe03c1a1520 con 0x7fe03c10b040 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.905+0000 7fe033e00640 1 -- 172.21.15.138:0/16796972 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe034003e20 con 0x7fe03c10b040 2024-08-20T01:08:55.913 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.905+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe03c1a7be0 con 0x7fe03c10b040 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.905+0000 7fe033e00640 1 -- 172.21.15.138:0/16796972 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0340106c0 con 0x7fe03c10b040 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.909+0000 7fe04546c740 1 Processor -- start 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe033e00640 1 -- 172.21.15.138:0/16796972 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7fe034003970 con 0x7fe03c10b040 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 -- start start 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919044280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919046bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e918fae840 con 0x55e919037f10 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e918f9dd40 con 0x55e919038830 2024-08-20T01:08:55.914 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.910+0000 7fe04546c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e918fa1d80 con 0x55e9190447c0 2024-08-20T01:08:55.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43992/0 (socket says 172.21.15.138:43992) 2024-08-20T01:08:55.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 -- 172.21.15.138:0/362727567 learned_addr learned my addr 172.21.15.138:0/362727567 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:55.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03b400640 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919044280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.915 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919046bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 -- 172.21.15.138:0/362727567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 msgr2=0x55e919046bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919046bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 -- 172.21.15.138:0/362727567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 msgr2=0x55e919044280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919044280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.911+0000 7fe03aa00640 1 -- 172.21.15.138:0/362727567 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e918f441c0 con 0x55e919037f10 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe03aa00640 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe028013820 tx=0x7fe028009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e8e0fc2a7452c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe033e00640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 0x7fe0200b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe033e00640 1 -- 172.21.15.138:0/16796972 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe0340e3d50 con 0x7fe03c10b040 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe030c00640 1 -- 172.21.15.138:0/362727567 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028013da0 con 0x55e919037f10 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe030c00640 1 -- 172.21.15.138:0/362727567 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0280117e0 con 0x55e919037f10 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe030c00640 1 -- 172.21.15.138:0/362727567 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028003850 con 0x55e919037f10 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.912+0000 7fe03b400640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 0x7fe0200b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 -- 172.21.15.138:0/362727567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 msgr2=0x55e9190382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe028013820 tx=0x7fe028009570 comp rx=0 tx=0).stop 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe03b400640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 0x7fe0200b61f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e91903b090 tx=0x7fe02c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 -- 172.21.15.138:0/362727567 shutdown_connections 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919046bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919044280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 --2- 172.21.15.138:0/362727567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919037f10 0x55e9190382f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 -- 172.21.15.138:0/362727567 >> 172.21.15.138:0/362727567 conn(0x55e919035d40 msgr2=0x55e919036ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 -- 172.21.15.138:0/362727567 shutdown_connections 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.913+0000 7fe04546c740 1 -- 172.21.15.138:0/362727567 wait complete. 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.914+0000 7fe04546c740 1 -- start start 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.914+0000 7fe04546c740 1 Processor -- start 2024-08-20T01:08:55.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e919043fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919041960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e918e005f0 con 0x55e919038830 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e918f39170 con 0x55e919037f10 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e919047410 con 0x55e9190447c0 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe04546c740 10 client.0 _ll_register_callbacks cb 0x55e918f988b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44004/0 (socket says 172.21.15.138:44004) 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe03b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e919043fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe03aa00640 1 -- 172.21.15.138:0/4032659539 learned_addr learned my addr 172.21.15.138:0/4032659539 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:55.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.915+0000 7fe03be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919041960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 -- 172.21.15.138:0/4032659539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 msgr2=0x55e919041960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919041960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 -- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 msgr2=0x55e919043fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e919043fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 -- 172.21.15.138:0/4032659539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e918cfd540 con 0x55e919038830 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03b400640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e919043fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe03aa00640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0280052a0 tx=0x7fe0280052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb10854fa7019843 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe025600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02802d060 con 0x55e919038830 2024-08-20T01:08:55.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0280117e0 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.916+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe028004600 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe028034050 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe04546c740 10 client.15495 fetch_fsmap learned FSMap version 26 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e919052cf0 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0280117e0 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe028020560 con 0x55e919038830 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe04546c740 10 client.15495 fetch_fsmap finished waiting for FSMap version 26 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe04546c740 10 client.15495 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.917+0000 7fe04546c740 20 client.15495 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:55.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.918+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e919055150 0x55e9190565c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.918+0000 7fe04546c740 4 client.15495 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.918+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e918fb0790 con 0x55e919055150 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.918+0000 7fe03b400640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e919055150 0x55e9190565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.919+0000 7fe03b400640 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e919055150 0x55e9190565c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=355a28d68ef64b87 server_cookie=69bcf4cd41e55ef6 in_seq=0 out_seq=0 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.919+0000 7fe025600640 10 client.15495 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.919+0000 7fe025600640 1 -- 172.21.15.138:0/4032659539 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e918fb0790 con 0x55e919055150 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.919+0000 7fe025600640 10 client.15495 handle_command_reply: tid=0 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 client.15495 shutdown 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 msgr2=0x55e919041420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe0280052a0 tx=0x7fe0280052d0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 shutdown_connections 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e919055150 0x55e9190565c0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919041960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919041420 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 --2- 172.21.15.138:0/4032659539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e919043fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 >> 172.21.15.138:0/4032659539 conn(0x55e919035d40 msgr2=0x55e91903b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe025600640 0 client.15495 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 shutdown_connections 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 1 -- 172.21.15.138:0/4032659539 wait complete. 2024-08-20T01:08:55.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.920+0000 7fe04546c740 20 client.15495 trim_cache size 0 max 16384 2024-08-20T01:08:55.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.941+0000 7fe04546c740 1 Processor -- start 2024-08-20T01:08:55.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.941+0000 7fe04546c740 1 -- start start 2024-08-20T01:08:55.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919041710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e919054900 con 0x55e9190447c0 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e919054fd0 con 0x55e919038830 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe04546c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e9190556a0 con 0x55e919037f10 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54694/0 (socket says 172.21.15.138:54694) 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03b400640 1 -- 172.21.15.138:0/2676941181 learned_addr learned my addr 172.21.15.138:0/2676941181 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:55.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03be00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44016/0 (socket says 172.21.15.138:44016) 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.942+0000 7fe03aa00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919041710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 -- 172.21.15.138:0/2676941181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 msgr2=0x55e9190411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 -- 172.21.15.138:0/2676941181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 msgr2=0x55e919036c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 -- 172.21.15.138:0/2676941181 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e919035b70 con 0x55e919038830 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03b400640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:55.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03be00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:08:55.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe03aa00640 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919041710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe028015010 tx=0x7fe028009570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e39c083aa47b78 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe027e00640 1 -- 172.21.15.138:0/2676941181 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02802d070 con 0x55e919038830 2024-08-20T01:08:55.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe027e00640 1 -- 172.21.15.138:0/2676941181 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028017040 con 0x55e919038830 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.943+0000 7fe027e00640 1 -- 172.21.15.138:0/2676941181 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe028020da0 con 0x55e919038830 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 -- 172.21.15.138:0/2676941181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 msgr2=0x55e919041710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919041710 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe028015010 tx=0x7fe028009570 comp rx=0 tx=0).stop 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 -- 172.21.15.138:0/2676941181 shutdown_connections 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e9190447c0 0x55e919036c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919038830 0x55e919041710 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 --2- 172.21.15.138:0/2676941181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e919037f10 0x55e9190411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 -- 172.21.15.138:0/2676941181 >> 172.21.15.138:0/2676941181 conn(0x55e919035d40 msgr2=0x55e919054150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.944+0000 7fe04546c740 1 -- 172.21.15.138:0/2676941181 shutdown_connections 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.945+0000 7fe04546c740 1 -- 172.21.15.138:0/2676941181 wait complete. 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.945+0000 7fe04546c740 1 -- start start 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.945+0000 7fe04546c740 1 Processor -- start 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e91905f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919059690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919059e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e919056440 con 0x55e919038830 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e919055d70 con 0x55e919037f10 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e919056b10 con 0x55e9190447c0 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe04546c740 10 client.0 _ll_register_callbacks cb 0x7fe02c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919059690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe03b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e91905f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe03aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919059690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44022/0 (socket says 172.21.15.138:44022) 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.946+0000 7fe03aa00640 1 -- 172.21.15.138:0/3268073219 learned_addr learned my addr 172.21.15.138:0/3268073219 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03be00640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919059e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 -- 172.21.15.138:0/3268073219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 msgr2=0x55e919059e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919059e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 -- 172.21.15.138:0/3268073219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 msgr2=0x55e919059690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:08:55.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919059690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 -- 172.21.15.138:0/3268073219 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9190367c0 con 0x55e919037f10 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe03b400640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e91905f180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005ec0 tx=0x7fe02c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1cc71e923636bd9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe025600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe02c035040 con 0x55e919037f10 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.947+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe028017040 con 0x55e919037f10 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe02c054680 con 0x55e919037f10 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe02c054880 con 0x55e919037f10 2024-08-20T01:08:55.950 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 10 client.25532 fetch_fsmap learned FSMap version 26 2024-08-20T01:08:55.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55e9190be9c0 con 0x55e919037f10 2024-08-20T01:08:55.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe028017040 con 0x55e919037f10 2024-08-20T01:08:55.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 26) ==== 8358+0+0 (secure 0 0 0) 0x7fe02c05a030 con 0x55e919037f10 2024-08-20T01:08:55.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 10 client.25532 fetch_fsmap finished waiting for FSMap version 26 2024-08-20T01:08:55.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 10 client.25532 resolve_mds: resolved name 'b' to daemon mds.b 2024-08-20T01:08:55.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 20 client.25532 populate_metadata read hostname 'smithi138' 2024-08-20T01:08:55.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.948+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e9190c3600 0x55e9190c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:08:55.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.949+0000 7fe04546c740 4 client.25532 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-08-20T01:08:55.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.949+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "status"}) -- 0x55e918fac230 con 0x55e9190c3600 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.949+0000 7fe03aa00640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e9190c3600 0x55e9190c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.949+0000 7fe03aa00640 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e9190c3600 0x55e9190c22e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5323af83cb22a409 server_cookie=35e0e06da5d9a2b7 in_seq=0 out_seq=0 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.949+0000 7fe025600640 10 client.25532 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe025600640 1 -- 172.21.15.138:0/3268073219 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e918fac230 con 0x55e9190c3600 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe025600640 10 client.25532 handle_command_reply: tid=0 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 client.25532 shutdown 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 msgr2=0x55e91905f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e91905f180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe02c005ec0 tx=0x7fe02c009260 comp rx=0 tx=0).stop 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 shutdown_connections 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e9190c3600 0x55e9190c22e0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e9190447c0 0x55e919059e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e919038830 0x55e919059690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe025600640 0 client.25532 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 --2- 172.21.15.138:0/3268073219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e919037f10 0x55e91905f180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 >> 172.21.15.138:0/3268073219 conn(0x55e919035d40 msgr2=0x55e9190411d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 shutdown_connections 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 1 -- 172.21.15.138:0/3268073219 wait complete. 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.950+0000 7fe04546c740 20 client.25532 trim_cache size 0 max 16384 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stdout: "cluster_fsid": "d876cd00-5e8e-11ef-bccf-c7b262605968", 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stdout: "whoami": 0, 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24743, 2024-08-20T01:08:55.959 INFO:teuthology.orchestra.run.smithi138.stdout: "want_state": "up:active", 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "state": "up:active", 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs", 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "rank_uptime": 207.34531730399999, 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "mdsmap_epoch": 26, 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch": 88, 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_epoch_barrier": 88, 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout: "uptime": 221.741209919 2024-08-20T01:08:55.960 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 msgr2=0x7fe0200b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 0x7fe0200b61f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55e91903b090 tx=0x7fe02c005a20 comp rx=0 tx=0).stop 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 msgr2=0x7fe03c1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c1a0a60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe03400da10 tx=0x7fe03400ded0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 shutdown_connections 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0200b3d30 0x7fe0200b61f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe03c10b040 0x7fe03c1a0a60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe03c10a6c0 0x7fe03c19b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 --2- 172.21.15.138:0/16796972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe03c104cb0 0x7fe03c19b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:08:55.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.951+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 >> 172.21.15.138:0/16796972 conn(0x7fe03c100720 msgr2=0x7fe03c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:08:55.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.952+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 shutdown_connections 2024-08-20T01:08:55.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:08:55.952+0000 7fe042800640 1 -- 172.21.15.138:0/16796972 wait complete. 2024-08-20T01:08:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:55 smithi167 ceph-mon[21965]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-08-20T01:08:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:55 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:55 smithi142 ceph-mon[21799]: pgmap v386: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 2.4 KiB/s wr, 9 op/s 2024-08-20T01:08:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:55 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:08:56.877 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-08-20T01:08:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:57 smithi167 ceph-mon[21965]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-08-20T01:08:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:57 smithi142 ceph-mon[21799]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-08-20T01:08:58.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:57 smithi138 ceph-mon[18076]: pgmap v387: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-08-20T01:08:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:08:59 smithi167 ceph-mon[21965]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-08-20T01:08:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:08:59 smithi142 ceph-mon[21799]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-08-20T01:09:00.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:08:59 smithi138 ceph-mon[18076]: pgmap v388: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 3.8 KiB/s wr, 10 op/s 2024-08-20T01:09:00.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:00 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:09:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 -- 172.21.15.138:0/4262542781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108200 msgr2=0x7f4dd8108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 --2- 172.21.15.138:0/4262542781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108200 0x7f4dd8108640 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4009900 tx=0x7f4dc402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 -- 172.21.15.138:0/4262542781 shutdown_connections 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 --2- 172.21.15.138:0/4262542781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8108b80 0x7f4dd810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 --2- 172.21.15.138:0/4262542781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108200 0x7f4dd8108640 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 --2- 172.21.15.138:0/4262542781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8102800 0x7f4dd8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 -- 172.21.15.138:0/4262542781 >> 172.21.15.138:0/4262542781 conn(0x7f4dd80fe280 msgr2=0x7f4dd81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 -- 172.21.15.138:0/4262542781 shutdown_connections 2024-08-20T01:09:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.971+0000 7f4ddd800640 1 -- 172.21.15.138:0/4262542781 wait complete. 2024-08-20T01:09:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 Processor -- start 2024-08-20T01:09:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 -- start start 2024-08-20T01:09:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 0x7f4dd80751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8108200 0x7f4dd8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 0x7f4dd8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd8112820 con 0x7f4dd8108b80 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81126a0 con 0x7f4dd8102800 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.972+0000 7f4ddd800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81129a0 con 0x7f4dd8108200 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 0x7f4dd8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8108200 0x7f4dd8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 0x7f4dd80751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 0x7f4dd8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59324/0 (socket says 172.21.15.138:59324) 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4ddce00640 1 -- 172.21.15.138:0/206071282 learned_addr learned my addr 172.21.15.138:0/206071282 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 -- 172.21.15.138:0/206071282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8108200 msgr2=0x7f4dd8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8108200 0x7f4dd8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 -- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 msgr2=0x7f4dd8075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 0x7f4dd8075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 -- 172.21.15.138:0/206071282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd81a7150 con 0x7f4dd8102800 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.973+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 0x7f4dd80751d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e9f0 tx=0x7f4dc800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.974+0000 7f4dd4c00640 1 -- 172.21.15.138:0/206071282 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800cd80 con 0x7f4dd8102800 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.974+0000 7f4dd4c00640 1 -- 172.21.15.138:0/206071282 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc80035f0 con 0x7f4dd8102800 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.974+0000 7f4dd4c00640 1 -- 172.21.15.138:0/206071282 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8010670 con 0x7f4dd8102800 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.974+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd81a7440 con 0x7f4dd8102800 2024-08-20T01:09:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.974+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd81a7910 con 0x7f4dd8102800 2024-08-20T01:09:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.977+0000 7f4de1cb2740 1 Processor -- start 2024-08-20T01:09:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 -- start start 2024-08-20T01:09:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 0x559937a2de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a2e3c0 0x559937a307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559937997210 con 0x559937a22430 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559937986710 con 0x559937a2e3c0 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55993798a750 con 0x559937a21b10 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.978+0000 7f4dd4c00640 1 -- 172.21.15.138:0/206071282 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 24) ==== 151134+0+0 (secure 0 0 0) 0x7f4dc80037a0 con 0x7f4dd8102800 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57192/0 (socket says 172.21.15.138:57192) 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 -- 172.21.15.138:0/877996255 learned_addr learned my addr 172.21.15.138:0/877996255 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4dd7400640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a2e3c0 0x559937a307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 -- 172.21.15.138:0/877996255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a2e3c0 msgr2=0x559937a307b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 0x559937a2de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a2e3c0 0x559937a307b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 -- 172.21.15.138:0/877996255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 msgr2=0x559937a2de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 0x559937a2de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 -- 172.21.15.138:0/877996255 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559937952de0 con 0x559937a21b10 2024-08-20T01:09:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 0x559937a2de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:09:00.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.979+0000 7f4ddce00640 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0011570 tx=0x7f4dd0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fd458b55cc7eddc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dcd600640 1 -- 172.21.15.138:0/877996255 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0004b20 con 0x559937a21b10 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dcd600640 1 -- 172.21.15.138:0/877996255 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd0004cc0 con 0x559937a21b10 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dcd600640 1 -- 172.21.15.138:0/877996255 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd0005490 con 0x559937a21b10 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dd4c00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 0x7f4dbc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dd4c00640 1 -- 172.21.15.138:0/206071282 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4dc80e44b0 con 0x7f4dd8102800 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4de1cb2740 1 -- 172.21.15.138:0/877996255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 msgr2=0x559937a21ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0011570 tx=0x7f4dd0011a30 comp rx=0 tx=0).stop 2024-08-20T01:09:00.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.980+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 0x7f4dbc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 -- 172.21.15.138:0/877996255 shutdown_connections 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a2e3c0 0x559937a307b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a22430 0x559937a2de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/877996255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a21b10 0x559937a21ef0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 -- 172.21.15.138:0/877996255 >> 172.21.15.138:0/877996255 conn(0x559937a1f940 msgr2=0x559937a20ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 -- 172.21.15.138:0/877996255 shutdown_connections 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 -- 172.21.15.138:0/877996255 wait complete. 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4dd7e00640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 0x7f4dbc0b6170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559937a24c90 tx=0x7f4dc8003d90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.981+0000 7f4de1cb2740 1 -- start start 2024-08-20T01:09:00.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 Processor -- start 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a22430 0x559937a2af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a2e3c0 0x559937a2b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5599378bfab0 con 0x559937a21b10 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55993792d140 con 0x559937a22430 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559937a310a0 con 0x559937a2e3c0 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4de1cb2740 10 client.0 _ll_register_callbacks cb 0x5599379405d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59340/0 (socket says 172.21.15.138:59340) 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4ddce00640 1 -- 172.21.15.138:0/2888064128 learned_addr learned my addr 172.21.15.138:0/2888064128 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.982+0000 7f4dd7400640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a22430 0x559937a2af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a2e3c0 0x559937a2b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 -- 172.21.15.138:0/2888064128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a2e3c0 msgr2=0x559937a2b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a2e3c0 0x559937a2b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 -- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a22430 msgr2=0x559937a2af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a22430 0x559937a2af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 -- 172.21.15.138:0/2888064128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993792e170 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4ddce00640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4dd001b040 tx=0x7f4dd0005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acba277eb007e46e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd002b020 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.983+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4dd00053a0 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dd0004cc0 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dd0036050 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4de1cb2740 10 client.15516 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5599376e9790 con 0x559937a21b10 2024-08-20T01:09:00.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dd0004cc0 con 0x559937a21b10 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.984+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4dd00278f0 con 0x559937a21b10 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 10 client.15516 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 10 client.15516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 20 client.15516 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937a3de10 0x559937a40200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 4 client.15516 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559937999160 con 0x559937a3de10 2024-08-20T01:09:00.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.985+0000 7f4dd7400640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937a3de10 0x559937a40200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.986+0000 7f4dd7400640 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937a3de10 0x559937a40200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4488f677e5ed4a8 server_cookie=e43786bbe202a0ed in_seq=0 out_seq=0 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.986+0000 7f4dc2000640 10 client.15516 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.986+0000 7f4dc2000640 1 -- 172.21.15.138:0/2888064128 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559937999160 con 0x559937a3de10 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.986+0000 7f4dc2000640 10 client.15516 handle_command_reply: tid=0 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.986+0000 7f4de1cb2740 1 client.15516 shutdown 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 msgr2=0x559937a2daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:00.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4dd001b040 tx=0x7f4dd0005000 comp rx=0 tx=0).stop 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 shutdown_connections 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937a3de10 0x559937a40200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a2e3c0 0x559937a2b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a22430 0x559937a2af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/2888064128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a21b10 0x559937a2daf0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4dc2000640 0 client.15516 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 >> 172.21.15.138:0/2888064128 conn(0x559937a1f940 msgr2=0x559937a254e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 shutdown_connections 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 1 -- 172.21.15.138:0/2888064128 wait complete. 2024-08-20T01:09:00.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:00.987+0000 7f4de1cb2740 20 client.15516 trim_cache size 0 max 16384 2024-08-20T01:09:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:00 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 Processor -- start 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 -- start start 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a211b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a45a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b4380 con 0x559937a2e3c0 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b1c50 con 0x559937a21b10 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.004+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81afaa0 con 0x559937a22430 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a45a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a45a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59354/0 (socket says 172.21.15.138:59354) 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4ddce00640 1 -- 172.21.15.138:0/1338478996 learned_addr learned my addr 172.21.15.138:0/1338478996 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4ddce00640 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a211b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 -- 172.21.15.138:0/1338478996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 msgr2=0x559937a211b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a211b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 -- 172.21.15.138:0/1338478996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 msgr2=0x559937a45a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a45a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 -- 172.21.15.138:0/1338478996 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559937a22260 con 0x559937a22430 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.005+0000 7f4dd7400640 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a264d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4dc402a9e0 tx=0x7f4dc4005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bd8af5dce61e484 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.006+0000 7f4dccc00640 1 -- 172.21.15.138:0/1338478996 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc4035040 con 0x559937a22430 2024-08-20T01:09:01.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.006+0000 7f4dccc00640 1 -- 172.21.15.138:0/1338478996 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4004770 con 0x559937a22430 2024-08-20T01:09:01.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.006+0000 7f4dccc00640 1 -- 172.21.15.138:0/1338478996 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc40455a0 con 0x559937a22430 2024-08-20T01:09:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.006+0000 7f4de1cb2740 1 -- 172.21.15.138:0/1338478996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 msgr2=0x559937a264d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.006+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a264d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4dc402a9e0 tx=0x7f4dc4005a20 comp rx=0 tx=0).stop 2024-08-20T01:09:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 -- 172.21.15.138:0/1338478996 shutdown_connections 2024-08-20T01:09:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a45a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a264d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/1338478996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a211b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 -- 172.21.15.138:0/1338478996 >> 172.21.15.138:0/1338478996 conn(0x559937a1f940 msgr2=0x559937a25dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 -- 172.21.15.138:0/1338478996 shutdown_connections 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 -- 172.21.15.138:0/1338478996 wait complete. 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.007+0000 7f4de1cb2740 1 -- start start 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 Processor -- start 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a2f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a2f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b0110 con 0x559937a2e3c0 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b1580 con 0x559937a21b10 2024-08-20T01:09:01.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd81b07e0 con 0x559937a22430 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4de1cb2740 10 client.0 _ll_register_callbacks cb 0x7f4dd000fb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.008+0000 7f4ddce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59370/0 (socket says 172.21.15.138:59370) 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4dd7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a2f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 -- 172.21.15.138:0/527933041 learned_addr learned my addr 172.21.15.138:0/527933041 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a2f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 -- 172.21.15.138:0/527933041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 msgr2=0x559937a2f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a2f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 -- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 msgr2=0x559937a2f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a2f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 -- 172.21.15.138:0/527933041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559937762620 con 0x559937a2e3c0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4dd7400640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a2f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4ddce00640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0034040 tx=0x7f4dd000f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3efb29853ada14fe server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:01.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4dc2000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dd002b040 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4dd0027740 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.009+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc4004770 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4dd0049050 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 10 client.15522 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559937a36af0 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4004770 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f4dd00254f0 con 0x559937a2e3c0 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 10 client.15522 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 10 client.15522 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 20 client.15522 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937aa89d0 0x559937aaadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:01.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 4 client.15522 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:09:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.010+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559937994c00 con 0x559937aa89d0 2024-08-20T01:09:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.011+0000 7f4dd7400640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937aa89d0 0x559937aaadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.011+0000 7f4dd7400640 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937aa89d0 0x559937aaadc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5dd3da0c88101df server_cookie=2203063a611020f3 in_seq=0 out_seq=0 2024-08-20T01:09:01.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.011+0000 7f4dc2000640 10 client.15522 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:01.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:09:01.040 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:09:00.649+0000 7f570e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:09:01.040 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:09:00.649+0000 7f570e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:09:01.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:09:00.649+0000 7f570e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:09:01.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:09:00.649+0000 7f570e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:09:01.041 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:09:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:09:00.649+0000 7f570e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-08-20T01:09:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.140+0000 7f4dc2000640 1 -- 172.21.15.138:0/527933041 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559937994c00 con 0x559937aa89d0 2024-08-20T01:09:01.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.140+0000 7f4dc2000640 10 client.15522 handle_command_reply: tid=0 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.140+0000 7f4de1cb2740 1 client.15522 shutdown 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 msgr2=0x559937a35330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0034040 tx=0x7f4dd000f8d0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 shutdown_connections 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559937aa89d0 0x559937aaadc0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559937a2e3c0 0x559937a35330 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559937a22430 0x559937a2f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 --2- 172.21.15.138:0/527933041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559937a21b10 0x559937a2f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4dc2000640 0 client.15522 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 >> 172.21.15.138:0/527933041 conn(0x559937a1f940 msgr2=0x559937a24cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 shutdown_connections 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 1 -- 172.21.15.138:0/527933041 wait complete. 2024-08-20T01:09:01.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.141+0000 7f4de1cb2740 20 client.15522 trim_cache size 0 max 16384 2024-08-20T01:09:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:09:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:09:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:09:01.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 msgr2=0x7f4dbc0b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 0x7f4dbc0b6170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x559937a24c90 tx=0x7f4dc8003d90 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 msgr2=0x7f4dd80751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 0x7f4dd80751d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800e9f0 tx=0x7f4dc800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 shutdown_connections 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4dbc0b3cb0 0x7f4dbc0b6170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dd8108b80 0x7f4dd8075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4dd8108200 0x7f4dd8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 --2- 172.21.15.138:0/206071282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4dd8102800 0x7f4dd80751d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 >> 172.21.15.138:0/206071282 conn(0x7f4dd80fe280 msgr2=0x7f4dd8100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 shutdown_connections 2024-08-20T01:09:01.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:01.142+0000 7f4ddd800640 1 -- 172.21.15.138:0/206071282 wait complete. 2024-08-20T01:09:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:01 smithi167 ceph-mon[21965]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-20T01:09:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:01 smithi142 ceph-mon[21799]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-20T01:09:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:01 smithi138 ceph-mon[18076]: pgmap v389: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-08-20T01:09:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:03 smithi167 ceph-mon[21965]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-08-20T01:09:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:03 smithi167 ceph-mon[21965]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-08-20T01:09:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:03 smithi142 ceph-mon[21799]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-08-20T01:09:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:03 smithi142 ceph-mon[21799]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-08-20T01:09:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:03 smithi138 ceph-mon[18076]: pgmap v390: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.9 KiB/s wr, 8 op/s 2024-08-20T01:09:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:03 smithi138 ceph-mon[18076]: mgrmap e25: x(active, since 10m), standbys: z, y 2024-08-20T01:09:04.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:04 smithi167 ceph-mon[21965]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-08-20T01:09:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:04 smithi142 ceph-mon[21799]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-08-20T01:09:05.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:04 smithi138 ceph-mon[18076]: pgmap v391: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 13 KiB/s wr, 19 op/s 2024-08-20T01:09:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:07 smithi167 ceph-mon[21965]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-08-20T01:09:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:07 smithi142 ceph-mon[21799]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-08-20T01:09:08.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:07 smithi138 ceph-mon[18076]: pgmap v392: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 26 KiB/s wr, 25 op/s 2024-08-20T01:09:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:09 smithi138 ceph-mon[18076]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-08-20T01:09:09.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:09 smithi167 ceph-mon[21965]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-08-20T01:09:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:09 smithi142 ceph-mon[21799]: pgmap v393: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 22 op/s 2024-08-20T01:09:11.878 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-08-20T01:09:11.878 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:09:11.947 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:11 smithi138 ceph-mon[18076]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-08-20T01:09:11.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:11 smithi167 ceph-mon[21965]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-08-20T01:09:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:11 smithi142 ceph-mon[21799]: pgmap v394: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-08-20T01:09:12.316 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.036+0000 7f8bb2000640 1 -- 172.21.15.138:0/2367258753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac104ca0 msgr2=0x7f8bac105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 --2- 172.21.15.138:0/2367258753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac104ca0 0x7f8bac105080 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009900 tx=0x7f8b9c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 -- 172.21.15.138:0/2367258753 shutdown_connections 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 --2- 172.21.15.138:0/2367258753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac10b030 0x7f8bac111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 --2- 172.21.15.138:0/2367258753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 0x7f8bac10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 --2- 172.21.15.138:0/2367258753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac104ca0 0x7f8bac105080 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 -- 172.21.15.138:0/2367258753 >> 172.21.15.138:0/2367258753 conn(0x7f8bac100730 msgr2=0x7f8bac102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 -- 172.21.15.138:0/2367258753 shutdown_connections 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.037+0000 7f8bb2000640 1 -- 172.21.15.138:0/2367258753 wait complete. 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.038+0000 7f8bb2000640 1 Processor -- start 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 -- start start 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 0x7f8bac19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 0x7f8bac1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac114d40 con 0x7f8bac104ca0 2024-08-20T01:09:13.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac114bc0 con 0x7f8bac10b030 2024-08-20T01:09:13.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.039+0000 7f8bb2000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8bac114ec0 con 0x7f8bac10a6b0 2024-08-20T01:09:13.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:13.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8babe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 0x7f8bac1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:13.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8babe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 0x7f8bac1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:41946/0 (socket says 172.21.15.138:41946) 2024-08-20T01:09:13.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40996/0 (socket says 172.21.15.138:40996) 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8babe00640 1 -- 172.21.15.138:0/956699142 learned_addr learned my addr 172.21.15.138:0/956699142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8baaa00640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 0x7f8bac19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 -- 172.21.15.138:0/956699142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 msgr2=0x7f8bac19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 0x7f8bac19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 -- 172.21.15.138:0/956699142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 msgr2=0x7f8bac1a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 0x7f8bac1a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.040+0000 7f8bab400640 1 -- 172.21.15.138:0/956699142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bac1a1260 con 0x7f8bac104ca0 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.041+0000 7f8bab400640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c02a9e0 tx=0x7f8b9c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.041+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c03f070 con 0x7f8bac104ca0 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.042+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b9c031b50 con 0x7f8bac104ca0 2024-08-20T01:09:13.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.042+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c035030 con 0x7f8bac104ca0 2024-08-20T01:09:13.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.042+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bac1a14f0 con 0x7f8bac104ca0 2024-08-20T01:09:13.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.043+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bac1a7c10 con 0x7f8bac104ca0 2024-08-20T01:09:13.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.044+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bac105080 con 0x7f8bac104ca0 2024-08-20T01:09:13.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.048+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8b9c042070 con 0x7f8bac104ca0 2024-08-20T01:09:13.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.051+0000 7f8ba3e00640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 0x7f8b900b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:13.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.051+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8b9c105300 con 0x7f8bac104ca0 2024-08-20T01:09:13.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.052+0000 7f8baaa00640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 0x7f8b900b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:13.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.053+0000 7f8baaa00640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 0x7f8b900b6350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bac103f80 tx=0x7f8b98005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:13.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.054+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8b9c0cc2e0 con 0x7f8bac104ca0 2024-08-20T01:09:13.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.512+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bac02e340 con 0x7f8bac104ca0 2024-08-20T01:09:13.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.513+0000 7f8ba3e00640 1 -- 172.21.15.138:0/956699142 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f8b9c031d00 con 0x7f8bac104ca0 2024-08-20T01:09:13.511 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:09:13.511 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":27,"btime":"2024-08-20T01:08:59:610307+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:59.217338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:09:13.512 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 27 2024-08-20T01:09:13.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 msgr2=0x7f8b900b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 0x7f8b900b6350 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bac103f80 tx=0x7f8b98005f90 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 msgr2=0x7f8bac19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c02a9e0 tx=0x7f8b9c003b90 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 shutdown_connections 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8b900b3e90 0x7f8b900b6350 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bac10b030 0x7f8bac1a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8bac10a6b0 0x7f8bac19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 --2- 172.21.15.138:0/956699142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bac104ca0 0x7f8bac19b1b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:13.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.516+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 >> 172.21.15.138:0/956699142 conn(0x7f8bac100730 msgr2=0x7f8bac102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:13.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.517+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 shutdown_connections 2024-08-20T01:09:13.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:13.517+0000 7f8bb2000640 1 -- 172.21.15.138:0/956699142 wait complete. 2024-08-20T01:09:13.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:13 smithi138 ceph-mon[18076]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-20T01:09:13.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:13 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/956699142' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:13.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:13 smithi167 ceph-mon[21965]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-20T01:09:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:13 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/956699142' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:13 smithi142 ceph-mon[21799]: pgmap v395: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-08-20T01:09:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:13 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/956699142' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 -- 172.21.15.138:0/96817504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e810ef30 msgr2=0x7f03e8111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 --2- 172.21.15.138:0/96817504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e810ef30 0x7f03e8111320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009ca0 tx=0x7f03e002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 -- 172.21.15.138:0/96817504 shutdown_connections 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 --2- 172.21.15.138:0/96817504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e810ef30 0x7f03e8111320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 --2- 172.21.15.138:0/96817504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e806aa70 0x7f03e806aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 --2- 172.21.15.138:0/96817504 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e81033c0 0x7f03e806a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.315+0000 7f03e7200640 1 -- 172.21.15.138:0/96817504 >> 172.21.15.138:0/96817504 conn(0x7f03e8077270 msgr2=0x7f03e8077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.316+0000 7f03e7200640 1 -- 172.21.15.138:0/96817504 shutdown_connections 2024-08-20T01:09:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.316+0000 7f03e7200640 1 -- 172.21.15.138:0/96817504 wait complete. 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 Processor -- start 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 -- start start 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e81033c0 0x7f03e819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e810ef30 0x7f03e81a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8114a00 con 0x7f03e81033c0 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8114880 con 0x7f03e806aa70 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.317+0000 7f03e7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8114b80 con 0x7f03e810ef30 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e81033c0 0x7f03e819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:41976/0 (socket says 172.21.15.138:41976) 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 -- 172.21.15.138:0/3228937136 learned_addr learned my addr 172.21.15.138:0/3228937136 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e6800640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e810ef30 0x7f03e81a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 -- 172.21.15.138:0/3228937136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e810ef30 msgr2=0x7f03e81a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e810ef30 0x7f03e81a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 -- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e81033c0 msgr2=0x7f03e819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e81033c0 0x7f03e819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 -- 172.21.15.138:0/3228937136 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e81a0fa0 con 0x7f03e806aa70 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.318+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03d800ea60 tx=0x7f03d800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.319+0000 7f03dea00640 1 -- 172.21.15.138:0/3228937136 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800cde0 con 0x7f03e806aa70 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.319+0000 7f03dea00640 1 -- 172.21.15.138:0/3228937136 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d80035f0 con 0x7f03e806aa70 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.319+0000 7f03dea00640 1 -- 172.21.15.138:0/3228937136 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8010690 con 0x7f03e806aa70 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.319+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e81a1290 con 0x7f03e806aa70 2024-08-20T01:09:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.319+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a7950 con 0x7f03e806aa70 2024-08-20T01:09:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.323+0000 7f03efbdd740 1 Processor -- start 2024-08-20T01:09:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.323+0000 7f03efbdd740 1 -- start start 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2abf210 con 0x55b9a2b497d0 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2aae710 con 0x55b9a2b56080 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03efbdd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2ab2750 con 0x55b9a2b4a0f0 2024-08-20T01:09:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.324+0000 7f03dea00640 1 -- 172.21.15.138:0/3228937136 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f03d80037a0 con 0x7f03e806aa70 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:41978/0 (socket says 172.21.15.138:41978) 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41026/0 (socket says 172.21.15.138:41026) 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 -- 172.21.15.138:0/3679139716 learned_addr learned my addr 172.21.15.138:0/3679139716 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 -- 172.21.15.138:0/3679139716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b4a0f0 msgr2=0x55b9a2b55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 -- 172.21.15.138:0/3679139716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 msgr2=0x55b9a2b49bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.325+0000 7f03e5400640 1 -- 172.21.15.138:0/3679139716 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a2a7ade0 con 0x55b9a2b56080 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03e6800640 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03e5400640 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009c90 tx=0x7f03d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba53884a9b05dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03d3000640 1 -- 172.21.15.138:0/3679139716 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d401e070 con 0x55b9a2b56080 2024-08-20T01:09:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03d3000640 1 -- 172.21.15.138:0/3679139716 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d4019ae0 con 0x55b9a2b56080 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03d3000640 1 -- 172.21.15.138:0/3679139716 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d400e9c0 con 0x55b9a2b56080 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.326+0000 7f03dea00640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 0x7f03cc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03dea00640 1 -- 172.21.15.138:0/3228937136 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f03d80e4730 con 0x7f03e806aa70 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03e6800640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 0x7f03cc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 -- 172.21.15.138:0/3679139716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 msgr2=0x55b9a2b58470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009c90 tx=0x7f03d4011810 comp rx=0 tx=0).stop 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 -- 172.21.15.138:0/3679139716 shutdown_connections 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b58470 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3679139716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b497d0 0x55b9a2b49bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.327+0000 7f03efbdd740 1 -- 172.21.15.138:0/3679139716 >> 172.21.15.138:0/3679139716 conn(0x55b9a2b47600 msgr2=0x55b9a2b487a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.328+0000 7f03efbdd740 1 -- 172.21.15.138:0/3679139716 shutdown_connections 2024-08-20T01:09:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.328+0000 7f03efbdd740 1 -- 172.21.15.138:0/3679139716 wait complete. 2024-08-20T01:09:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.328+0000 7f03e6800640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 0x7f03cc0b6200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03e0004af0 tx=0x7f03e0005da0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.328+0000 7f03efbdd740 1 -- start start 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 Processor -- start 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b53560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b53aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.330 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9a29e7ab0 con 0x55b9a2b4a0f0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2a55140 con 0x55b9a2b56080 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9a2b61570 con 0x55b9a2b497d0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.329+0000 7f03efbdd740 10 client.0 _ll_register_callbacks cb 0x55b9a2a685d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44104/0 (socket says 172.21.15.138:44104) 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 -- 172.21.15.138:0/519491494 learned_addr learned my addr 172.21.15.138:0/519491494 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5400640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b53560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e6800640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b53aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 -- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 msgr2=0x55b9a2b53aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b53aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 -- 172.21.15.138:0/519491494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 msgr2=0x55b9a2b53560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b53560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 -- 172.21.15.138:0/519491494 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a2a56170 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03e5e00640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009920 tx=0x7f03d8003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1735da8b448ff01 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.330+0000 7f03c7400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80ab690 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9a2a85a30 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f03d80e1d50 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03d80e1f50 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03efbdd740 10 client.25543 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9a2811790 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b9a2a85a30 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.331+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03d80e4320 con 0x55b9a2b497d0 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 10 client.25543 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 10 client.25543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 20 client.25543 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2b6e7e0 0x55b9a2b70bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 4 client.25543 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9a2ac1160 con 0x55b9a2b6e7e0 2024-08-20T01:09:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.332+0000 7f03e5400640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2b6e7e0 0x55b9a2b70bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.333+0000 7f03e5400640 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2b6e7e0 0x55b9a2b70bd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=915f9e20f390f933 server_cookie=8a9ffeeb71b48c50 in_seq=0 out_seq=0 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.333+0000 7f03c7400640 10 client.25543 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03c7400640 1 -- 172.21.15.138:0/519491494 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9a2ac1160 con 0x55b9a2b6e7e0 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03c7400640 10 client.25543 handle_command_reply: tid=0 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 client.25543 shutdown 2024-08-20T01:09:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 msgr2=0x55b9a2b53020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009920 tx=0x7f03d8003cf0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 shutdown_connections 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2b6e7e0 0x55b9a2b70bd0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b53aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b53560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 --2- 172.21.15.138:0/519491494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b53020 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 >> 172.21.15.138:0/519491494 conn(0x55b9a2b47600 msgr2=0x55b9a2b4d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03c7400640 0 client.25543 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.334+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 shutdown_connections 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.335+0000 7f03efbdd740 1 -- 172.21.15.138:0/519491494 wait complete. 2024-08-20T01:09:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.335+0000 7f03efbdd740 20 client.25543 trim_cache size 0 max 16384 2024-08-20T01:09:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 Processor -- start 2024-08-20T01:09:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 -- start start 2024-08-20T01:09:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b764a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b56080 0x55b9a2b769e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81aff00 con 0x55b9a2b56080 2024-08-20T01:09:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ad7d0 con 0x55b9a2b4a0f0 2024-08-20T01:09:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.350+0000 7f03efbdd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ab680 con 0x55b9a2b497d0 2024-08-20T01:09:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b56080 0x55b9a2b769e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b764a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44118/0 (socket says 172.21.15.138:44118) 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 -- 172.21.15.138:0/4178667851 learned_addr learned my addr 172.21.15.138:0/4178667851 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 -- 172.21.15.138:0/4178667851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 msgr2=0x55b9a2b764a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b764a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 -- 172.21.15.138:0/4178667851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b56080 msgr2=0x55b9a2b769e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b56080 0x55b9a2b769e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 -- 172.21.15.138:0/4178667851 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9a2b70f40 con 0x55b9a2b497d0 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5400640 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.351+0000 7f03e5e00640 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03d80095f0 tx=0x7f03d800ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134af59e6f5b70c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03d2600640 1 -- 172.21.15.138:0/4178667851 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80a6e50 con 0x55b9a2b497d0 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03d2600640 1 -- 172.21.15.138:0/4178667851 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d80aaf90 con 0x55b9a2b497d0 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03d2600640 1 -- 172.21.15.138:0/4178667851 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80aa700 con 0x55b9a2b497d0 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 -- 172.21.15.138:0/4178667851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 msgr2=0x55b9a2b4d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03d80095f0 tx=0x7f03d800ef90 comp rx=0 tx=0).stop 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 -- 172.21.15.138:0/4178667851 shutdown_connections 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b56080 0x55b9a2b769e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b764a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.352+0000 7f03efbdd740 1 --2- 172.21.15.138:0/4178667851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b4d920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.353+0000 7f03efbdd740 1 -- 172.21.15.138:0/4178667851 >> 172.21.15.138:0/4178667851 conn(0x55b9a2b47600 msgr2=0x55b9a2b4e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.353+0000 7f03efbdd740 1 -- 172.21.15.138:0/4178667851 shutdown_connections 2024-08-20T01:09:14.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.353+0000 7f03efbdd740 1 -- 172.21.15.138:0/4178667851 wait complete. 2024-08-20T01:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.353+0000 7f03efbdd740 1 -- start start 2024-08-20T01:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.353+0000 7f03efbdd740 1 Processor -- start 2024-08-20T01:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b66d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.355 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b4fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81abcd0 con 0x55b9a2b4a0f0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ad100 con 0x55b9a2b56080 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ac360 con 0x55b9a2b497d0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03efbdd740 10 client.0 _ll_register_callbacks cb 0x7f03d800e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b4fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b4fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41058/0 (socket says 172.21.15.138:41058) 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5400640 1 -- 172.21.15.138:0/3460980957 learned_addr learned my addr 172.21.15.138:0/3460980957 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e6800640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b66d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.356 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5e00640 1 -- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 msgr2=0x55b9a2b501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5e00640 1 -- 172.21.15.138:0/3460980957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 msgr2=0x55b9a2b4fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.354+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b4fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03e5e00640 1 -- 172.21.15.138:0/3460980957 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9a288a620 con 0x55b9a2b497d0 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03e6800640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03e5e00640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b66d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03d80a8200 tx=0x7f03d80e1bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1acb8697e4eb28f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03c7400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80aa700 con 0x55b9a2b497d0 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f03d80e1d50 con 0x55b9a2b497d0 2024-08-20T01:09:14.357 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03d80aaf90 con 0x55b9a2b497d0 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03d80a6e50 con 0x55b9a2b497d0 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03efbdd740 10 client.25549 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.355+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b9a2bd9190 con 0x55b9a2b497d0 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d80aaf90 con 0x55b9a2b497d0 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f03d80a6900 con 0x55b9a2b497d0 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 10 client.25549 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 10 client.25549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 20 client.25549 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2bd95c0 0x55b9a2bdb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:14.358 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 4 client.25549 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:09:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9a2abcc00 con 0x55b9a2bd95c0 2024-08-20T01:09:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.356+0000 7f03e5400640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2bd95c0 0x55b9a2bdb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.357+0000 7f03e5400640 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2bd95c0 0x55b9a2bdb9e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=220ed5f2efc595d4 server_cookie=867f97b72b86d567 in_seq=0 out_seq=0 2024-08-20T01:09:14.359 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.357+0000 7f03c7400640 10 client.25549 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.366+0000 7f03c7400640 1 -- 172.21.15.138:0/3460980957 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9a2abcc00 con 0x55b9a2bd95c0 2024-08-20T01:09:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.366+0000 7f03c7400640 10 client.25549 handle_command_reply: tid=0 2024-08-20T01:09:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.366+0000 7f03efbdd740 1 client.25549 shutdown 2024-08-20T01:09:14.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.369+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 msgr2=0x55b9a2b66d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.369+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b66d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03d80a8200 tx=0x7f03d80e1bb0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.369+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 shutdown_connections 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.369+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9a2bd95c0 0x55b9a2bdb9e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.370+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9a2b56080 0x55b9a2b501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.370+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9a2b4a0f0 0x55b9a2b4fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.370+0000 7f03efbdd740 1 --2- 172.21.15.138:0/3460980957 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9a2b497d0 0x55b9a2b66d40 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f03d80a8200 tx=0x7f03d80e1bb0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.370+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 >> 172.21.15.138:0/3460980957 conn(0x55b9a2b47600 msgr2=0x55b9a2b45d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.370+0000 7f03c7400640 0 client.25549 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.371+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 shutdown_connections 2024-08-20T01:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.371+0000 7f03efbdd740 1 -- 172.21.15.138:0/3460980957 wait complete. 2024-08-20T01:09:14.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.371+0000 7f03efbdd740 20 client.25549 trim_cache size 0 max 16384 2024-08-20T01:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:09:14.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 msgr2=0x7f03cc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 0x7f03cc0b6200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03e0004af0 tx=0x7f03e0005da0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 msgr2=0x7f03e819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03d800ea60 tx=0x7f03d800ef20 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 shutdown_connections 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f03cc0b3d40 0x7f03cc0b6200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f03e810ef30 0x7f03e81a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03e81033c0 0x7f03e819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 --2- 172.21.15.138:0/3228937136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f03e806aa70 0x7f03e819af00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.372+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 >> 172.21.15.138:0/3228937136 conn(0x7f03e8077270 msgr2=0x7f03e81008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.373+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 shutdown_connections 2024-08-20T01:09:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:14.373+0000 7f03e7200640 1 -- 172.21.15.138:0/3228937136 wait complete. 2024-08-20T01:09:14.409 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-08-20T01:09:14.410 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-08-20T01:09:14.844 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 -- 172.21.15.138:0/1381857203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717410a520 msgr2=0x7f717410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 --2- 172.21.15.138:0/1381857203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717410a520 0x7f717410a960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7160009900 tx=0x7f716002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 -- 172.21.15.138:0/1381857203 shutdown_connections 2024-08-20T01:09:15.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 --2- 172.21.15.138:0/1381857203 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 --2- 172.21.15.138:0/1381857203 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717410a520 0x7f717410a960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 --2- 172.21.15.138:0/1381857203 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7174104b10 0x7f7174104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 -- 172.21.15.138:0/1381857203 >> 172.21.15.138:0/1381857203 conn(0x7f7174100540 msgr2=0x7f7174102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 -- 172.21.15.138:0/1381857203 shutdown_connections 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.565+0000 7f7179c00640 1 -- 172.21.15.138:0/1381857203 wait complete. 2024-08-20T01:09:15.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.566+0000 7f7179c00640 1 Processor -- start 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 -- start start 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717410a520 0x7f717419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7174114b40 con 0x7f717410a520 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71741149c0 con 0x7f7174104b10 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.567+0000 7f7179c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7174114cc0 con 0x7f717410aea0 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.568+0000 7f7173400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717410a520 0x7f717419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.568+0000 7f7173e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.568+0000 7f7173e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42040/0 (socket says 172.21.15.138:42040) 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.568+0000 7f7173e00640 1 -- 172.21.15.138:0/2743352919 learned_addr learned my addr 172.21.15.138:0/2743352919 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7179200640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 -- 172.21.15.138:0/2743352919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 msgr2=0x7f71741a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 -- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717410a520 msgr2=0x7f717419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717410a520 0x7f717419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 -- 172.21.15.138:0/2743352919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71741a0fa0 con 0x7f7174104b10 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7179200640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.569+0000 7f7173e00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f716400d950 tx=0x7f716400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.570+0000 7f7170c00640 1 -- 172.21.15.138:0/2743352919 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164017030 con 0x7f7174104b10 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.570+0000 7f7170c00640 1 -- 172.21.15.138:0/2743352919 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7164004020 con 0x7f7174104b10 2024-08-20T01:09:15.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.570+0000 7f7170c00640 1 -- 172.21.15.138:0/2743352919 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640047a0 con 0x7f7174104b10 2024-08-20T01:09:15.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.570+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71741a1290 con 0x7f7174104b10 2024-08-20T01:09:15.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.570+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71741a7950 con 0x7f7174104b10 2024-08-20T01:09:15.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.575+0000 7f717dc7d740 1 Processor -- start 2024-08-20T01:09:15.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.575+0000 7f717dc7d740 1 -- start start 2024-08-20T01:09:15.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8de0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f7179200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f7179200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42050/0 (socket says 172.21.15.138:42050) 2024-08-20T01:09:15.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f7179200640 1 -- 172.21.15.138:0/3779531792 learned_addr learned my addr 172.21.15.138:0/3779531792 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f7173400640 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8de0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.576+0000 7f717dc7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642f8d71670 con 0x5642f8dd1ef0 2024-08-20T01:09:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.577+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642f8d457c0 con 0x5642f8dd15d0 2024-08-20T01:09:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.577+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642f8d34cc0 con 0x5642f8ddde80 2024-08-20T01:09:15.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.577+0000 7f7170c00640 1 -- 172.21.15.138:0/2743352919 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7164009a30 con 0x7f7174104b10 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.577+0000 7f7173e00640 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 -- 172.21.15.138:0/3779531792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 msgr2=0x5642f8de0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8de0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 -- 172.21.15.138:0/3779531792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 msgr2=0x5642f8dd19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 -- 172.21.15.138:0/3779531792 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f8cc8110 con 0x5642f8dd1ef0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.578+0000 7f7173e00640 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddd940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71640e1150 tx=0x7f71640ec800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f686d8e6c3fcb6c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.579+0000 7f7169600640 1 -- 172.21.15.138:0/3779531792 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640f5440 con 0x5642f8dd1ef0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.579+0000 7f7169600640 1 -- 172.21.15.138:0/3779531792 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71640e98c0 con 0x5642f8dd1ef0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.579+0000 7f7169600640 1 -- 172.21.15.138:0/3779531792 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71640f4860 con 0x5642f8dd1ef0 2024-08-20T01:09:15.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.579+0000 7f7170c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 0x7f71580b6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.579+0000 7f7170c00640 1 -- 172.21.15.138:0/2743352919 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f71640e3db0 con 0x7f7174104b10 2024-08-20T01:09:15.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.580+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 msgr2=0x5642f8ddd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.580+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddd940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71640e1150 tx=0x7f71640ec800 comp rx=0 tx=0).stop 2024-08-20T01:09:15.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.581+0000 7f7179200640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 0x7f71580b6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.581+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 shutdown_connections 2024-08-20T01:09:15.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.581+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8de0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.581+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddd940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.582+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3779531792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.582+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 >> 172.21.15.138:0/3779531792 conn(0x5642f8dcf400 msgr2=0x5642f8dd05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.582+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 shutdown_connections 2024-08-20T01:09:15.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.582+0000 7f7179200640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 0x7f71580b6480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f716c012640 tx=0x7f716c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.582+0000 7f717dc7d740 1 -- 172.21.15.138:0/3779531792 wait complete. 2024-08-20T01:09:15.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.583+0000 7f717dc7d740 1 -- start start 2024-08-20T01:09:15.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.583+0000 7f717dc7d740 1 Processor -- start 2024-08-20T01:09:15.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.584+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8ddb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642f8cda780 con 0x5642f8dd1ef0 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642f8d38d00 con 0x5642f8dd15d0 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642f8b977a0 con 0x5642f8ddde80 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f717dc7d740 10 client.0 _ll_register_callbacks cb 0x5642f8ce2940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f7173e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f7173e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42062/0 (socket says 172.21.15.138:42062) 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f7173e00640 1 -- 172.21.15.138:0/2376086456 learned_addr learned my addr 172.21.15.138:0/2376086456 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f7173400640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.585+0000 7f7179200640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8ddb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7179200640 1 -- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 msgr2=0x5642f8ddae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7179200640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7179200640 1 -- 172.21.15.138:0/2376086456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 msgr2=0x5642f8ddb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7179200640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7179200640 1 -- 172.21.15.138:0/2376086456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f8ce79c0 con 0x5642f8ddde80 2024-08-20T01:09:15.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.586+0000 7f7173400640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f7173e00640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f7179200640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8ddb910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f716c0109c0 tx=0x7f716c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2b98645d95a0ae0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f715e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c003d80 con 0x5642f8ddde80 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f716c01e830 con 0x5642f8ddde80 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.587+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71640e98c0 con 0x5642f8ddde80 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.588+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f716c01ea30 con 0x5642f8ddde80 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.588+0000 7f717dc7d740 10 client.25558 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.588+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642f8aa3b90 con 0x5642f8ddde80 2024-08-20T01:09:15.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.588+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71640e98c0 con 0x5642f8ddde80 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.589+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f716c004030 con 0x5642f8ddde80 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.589+0000 7f717dc7d740 10 client.25558 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.589+0000 7f717dc7d740 10 client.25558 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.590+0000 7f717dc7d740 20 client.25558 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.590+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8df6210 0x5642f8df8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.590+0000 7f717dc7d740 4 client.25558 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.590+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642f8d47710 con 0x5642f8df6210 2024-08-20T01:09:15.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.590+0000 7f7173e00640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8df6210 0x5642f8df8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.591+0000 7f7173e00640 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8df6210 0x5642f8df8600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d79890e26441369c server_cookie=480b1ceb0303ed6d in_seq=0 out_seq=0 2024-08-20T01:09:15.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.591+0000 7f715e000640 10 client.25558 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:15.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.591+0000 7f715e000640 1 -- 172.21.15.138:0/2376086456 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642f8d47710 con 0x5642f8df6210 2024-08-20T01:09:15.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.591+0000 7f715e000640 10 client.25558 handle_command_reply: tid=0 2024-08-20T01:09:15.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 client.25558 shutdown 2024-08-20T01:09:15.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 msgr2=0x5642f8ddb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8ddb910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f716c0109c0 tx=0x7f716c004a40 comp rx=0 tx=0).stop 2024-08-20T01:09:15.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 shutdown_connections 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8df6210 0x5642f8df8600 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8ddb910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8ddb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 --2- 172.21.15.138:0/2376086456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8ddae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 >> 172.21.15.138:0/2376086456 conn(0x5642f8dcf400 msgr2=0x5642f8dd4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.592+0000 7f715e000640 0 client.25558 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:15.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.593+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 shutdown_connections 2024-08-20T01:09:15.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.593+0000 7f717dc7d740 1 -- 172.21.15.138:0/2376086456 wait complete. 2024-08-20T01:09:15.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.593+0000 7f717dc7d740 20 client.25558 trim_cache size 0 max 16384 2024-08-20T01:09:15.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.628+0000 7f717dc7d740 1 Processor -- start 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.628+0000 7f717dc7d740 1 -- start start 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8dfded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71741aff40 con 0x5642f8dd1ef0 2024-08-20T01:09:15.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71741ad810 con 0x5642f8dd15d0 2024-08-20T01:09:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f717dc7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71741ab6a0 con 0x5642f8ddde80 2024-08-20T01:09:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f7179200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f7179200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44184/0 (socket says 172.21.15.138:44184) 2024-08-20T01:09:15.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f7179200640 1 -- 172.21.15.138:0/1273632660 learned_addr learned my addr 172.21.15.138:0/1273632660 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.629+0000 7f7173400640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8dfded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7173e00640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7179200640 1 -- 172.21.15.138:0/1273632660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 msgr2=0x5642f8dd56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7179200640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7179200640 1 -- 172.21.15.138:0/1273632660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 msgr2=0x5642f8dfded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7179200640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8dfded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7179200640 1 -- 172.21.15.138:0/1273632660 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642f8dcfe80 con 0x5642f8ddde80 2024-08-20T01:09:15.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7173400640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8dfded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:09:15.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.630+0000 7f7173e00640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:15.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.631+0000 7f7179200640 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f716c012f70 tx=0x7f716c009a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf097fd715575bf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.630 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:15 smithi138 ceph-mon[18076]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-08-20T01:09:15.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.631+0000 7f7168c00640 1 -- 172.21.15.138:0/1273632660 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c039070 con 0x5642f8ddde80 2024-08-20T01:09:15.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.631+0000 7f7168c00640 1 -- 172.21.15.138:0/1273632660 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f716c004000 con 0x5642f8ddde80 2024-08-20T01:09:15.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.631+0000 7f7168c00640 1 -- 172.21.15.138:0/1273632660 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c01ebd0 con 0x5642f8ddde80 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.633+0000 7f717dc7d740 1 -- 172.21.15.138:0/1273632660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 msgr2=0x5642f8dfe410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.633+0000 7f717dc7d740 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f716c012f70 tx=0x7f716c009a70 comp rx=0 tx=0).stop 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 -- 172.21.15.138:0/1273632660 shutdown_connections 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8ddde80 0x5642f8dfe410 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd1ef0 0x5642f8dfded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 --2- 172.21.15.138:0/1273632660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8dd15d0 0x5642f8dd56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 -- 172.21.15.138:0/1273632660 >> 172.21.15.138:0/1273632660 conn(0x5642f8dcf400 msgr2=0x5642f8ddc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 -- 172.21.15.138:0/1273632660 shutdown_connections 2024-08-20T01:09:15.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.634+0000 7f717dc7d740 1 -- 172.21.15.138:0/1273632660 wait complete. 2024-08-20T01:09:15.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.635+0000 7f717dc7d740 1 -- start start 2024-08-20T01:09:15.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.635+0000 7f717dc7d740 1 Processor -- start 2024-08-20T01:09:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 0x5642f8de9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8ddde80 0x5642f8dee270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71741abcf0 con 0x5642f8dd15d0 2024-08-20T01:09:15.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71741ad140 con 0x5642f8ddde80 2024-08-20T01:09:15.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71741ac3a0 con 0x5642f8dd1ef0 2024-08-20T01:09:15.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.636+0000 7f717dc7d740 10 client.0 _ll_register_callbacks cb 0x7f716c009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:15.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.637+0000 7f7173e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 0x5642f8de9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.637+0000 7f7173400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.637+0000 7f7173400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44192/0 (socket says 172.21.15.138:44192) 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.637+0000 7f7173400640 1 -- 172.21.15.138:0/3997248733 learned_addr learned my addr 172.21.15.138:0/3997248733 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.637+0000 7f7179200640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8ddde80 0x5642f8dee270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173400640 1 -- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8ddde80 msgr2=0x5642f8dee270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173400640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8ddde80 0x5642f8dee270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173400640 1 -- 172.21.15.138:0/3997248733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 msgr2=0x5642f8de9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173400640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 0x5642f8de9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173400640 1 -- 172.21.15.138:0/3997248733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642f8dd5c10 con 0x5642f8dd1ef0 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.638+0000 7f7173e00640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 0x5642f8de9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.639+0000 7f7173400640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7160009230 tx=0x7f71600052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb7d7e3be9e30572 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:15.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.639+0000 7f715e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:15.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.640+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160035040 con 0x5642f8dd1ef0 2024-08-20T01:09:15.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.640+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f7160004970 con 0x5642f8dd1ef0 2024-08-20T01:09:15.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.640+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f716c004000 con 0x5642f8dd1ef0 2024-08-20T01:09:15.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.641+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f716004f050 con 0x5642f8dd1ef0 2024-08-20T01:09:15.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.641+0000 7f717dc7d740 10 client.25564 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:15.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.641+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642f8e71790 con 0x5642f8dd1ef0 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.641+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71600035f0 con 0x5642f8dd1ef0 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f7160044020 con 0x5642f8dd1ef0 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f717dc7d740 10 client.25564 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f717dc7d740 10 client.25564 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f717dc7d740 20 client.25564 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:15.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8e72010 0x5642f8e74430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:15.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.642+0000 7f717dc7d740 4 client.25564 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-08-20T01:09:15.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.643+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x7f716c004000 con 0x5642f8e72010 2024-08-20T01:09:15.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.643+0000 7f7173e00640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8e72010 0x5642f8e74430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:15.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.644+0000 7f7173e00640 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8e72010 0x5642f8e74430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=783b5c85a183144f server_cookie=8cb577bf08bdb616 in_seq=0 out_seq=0 2024-08-20T01:09:15.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.644+0000 7f715e000640 10 client.25564 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.651+0000 7f715e000640 1 -- 172.21.15.138:0/3997248733 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f716c004000 con 0x5642f8e72010 2024-08-20T01:09:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.651+0000 7f715e000640 10 client.25564 handle_command_reply: tid=0 2024-08-20T01:09:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.651+0000 7f717dc7d740 1 client.25564 shutdown 2024-08-20T01:09:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 msgr2=0x5642f8dea320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7160009230 tx=0x7f71600052e0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 shutdown_connections 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5642f8e72010 0x5642f8e74430 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642f8ddde80 0x5642f8dee270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5642f8dd1ef0 0x5642f8dea320 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f715e000640 0 client.25564 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.652+0000 7f717dc7d740 1 --2- 172.21.15.138:0/3997248733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642f8dd15d0 0x5642f8de9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.653+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 >> 172.21.15.138:0/3997248733 conn(0x5642f8dcf400 msgr2=0x5642f8ddad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.653+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 shutdown_connections 2024-08-20T01:09:15.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.653+0000 7f717dc7d740 1 -- 172.21.15.138:0/3997248733 wait complete. 2024-08-20T01:09:15.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.653+0000 7f717dc7d740 20 client.25564 trim_cache size 0 max 16384 2024-08-20T01:09:15.652 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:09:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-08-20T01:09:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:09:15.652 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 2, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "d5539d4a": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 2, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.0, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:15.653 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:15.654 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 msgr2=0x7f71580b6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 0x7f71580b6480 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f716c012640 tx=0x7f716c009500 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 msgr2=0x7f717419aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f716400d950 tx=0x7f716400de10 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 shutdown_connections 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71580b3fc0 0x7f71580b6480 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717410aea0 0x7f71741a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717410a520 0x7f717419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 --2- 172.21.15.138:0/2743352919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7174104b10 0x7f717419aef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.655+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 >> 172.21.15.138:0/2743352919 conn(0x7f7174100540 msgr2=0x7f7174102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:15.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.656+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 shutdown_connections 2024-08-20T01:09:15.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:15.657+0000 7f7179c00640 1 -- 172.21.15.138:0/2743352919 wait complete. 2024-08-20T01:09:15.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:15 smithi167 ceph-mon[21965]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-08-20T01:09:15.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:15 smithi142 ceph-mon[21799]: pgmap v396: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-08-20T01:09:16.507 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd5539d4a', quiesce took 4.6 seconds. Will release after: 5.4 2024-08-20T01:09:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:17 smithi167 ceph-mon[21965]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-08-20T01:09:17.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:17 smithi142 ceph-mon[21799]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-08-20T01:09:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:17 smithi138 ceph-mon[18076]: pgmap v397: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-08-20T01:09:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:19 smithi167 ceph-mon[21965]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-08-20T01:09:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:19 smithi142 ceph-mon[21799]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-08-20T01:09:20.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:19 smithi138 ceph-mon[18076]: pgmap v398: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-08-20T01:09:21.908 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:09:21.959 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:21 smithi138 ceph-mon[18076]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:21.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:21 smithi167 ceph-mon[21965]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:21.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:21 smithi142 ceph-mon[21799]: pgmap v399: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:22.325 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 -- 172.21.15.138:0/3401211905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c0751f0 msgr2=0x7f275c0755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 --2- 172.21.15.138:0/3401211905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c0751f0 0x7f275c0755d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f274c009900 tx=0x7f274c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 -- 172.21.15.138:0/3401211905 shutdown_connections 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 --2- 172.21.15.138:0/3401211905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c10ef30 0x7f275c111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 --2- 172.21.15.138:0/3401211905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c075b10 0x7f275c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.004 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 --2- 172.21.15.138:0/3401211905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c0751f0 0x7f275c0755d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.005+0000 7f275b200640 1 -- 172.21.15.138:0/3401211905 >> 172.21.15.138:0/3401211905 conn(0x7f275c0fe350 msgr2=0x7f275c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:23.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.006+0000 7f275b200640 1 -- 172.21.15.138:0/3401211905 shutdown_connections 2024-08-20T01:09:23.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.006+0000 7f275b200640 1 -- 172.21.15.138:0/3401211905 wait complete. 2024-08-20T01:09:23.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.007+0000 7f275b200640 1 Processor -- start 2024-08-20T01:09:23.005 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.007+0000 7f275b200640 1 -- start start 2024-08-20T01:09:23.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c0751f0 0x7f275c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:23.006 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c10ef30 0x7f275c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f275c114b10 con 0x7f275c0751f0 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f275c114990 con 0x7f275c10ef30 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f275c114c90 con 0x7f275c075b10 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f2759e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c0751f0 0x7f275c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f275a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c10ef30 0x7f275c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f2759400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f2759400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47358/0 (socket says 172.21.15.138:47358) 2024-08-20T01:09:23.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.008+0000 7f2759400640 1 -- 172.21.15.138:0/138065556 learned_addr learned my addr 172.21.15.138:0/138065556 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:23.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 -- 172.21.15.138:0/138065556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c10ef30 msgr2=0x7f275c1a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c10ef30 0x7f275c1a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 -- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c0751f0 msgr2=0x7f275c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c0751f0 0x7f275c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 -- 172.21.15.138:0/138065556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f275c1a1030 con 0x7f275c075b10 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.009+0000 7f2759400640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f274800e9f0 tx=0x7f274800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.010+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274800cd80 con 0x7f275c075b10 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.010+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27480054d0 con 0x7f275c075b10 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.010+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275c1a1320 con 0x7f275c075b10 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.010+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275c1a79e0 con 0x7f275c075b10 2024-08-20T01:09:23.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.011+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748010690 con 0x7f275c075b10 2024-08-20T01:09:23.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.012+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f275c0755d0 con 0x7f275c075b10 2024-08-20T01:09:23.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.014+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2748004e90 con 0x7f275c075b10 2024-08-20T01:09:23.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.016+0000 7f2752a00640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 0x7f27400b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:23.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.016+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f27480e4740 con 0x7f275c075b10 2024-08-20T01:09:23.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.017+0000 7f2759e00640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 0x7f27400b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:23.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.017+0000 7f2759e00640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 0x7f27400b6320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f274c031440 tx=0x7f274c005b50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:23.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.019+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f27480ab6a0 con 0x7f275c075b10 2024-08-20T01:09:23.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.465+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f275c1a7cf0 con 0x7f275c075b10 2024-08-20T01:09:23.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.466+0000 7f2752a00640 1 -- 172.21.15.138:0/138065556 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f27480aafa0 con 0x7f275c075b10 2024-08-20T01:09:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:09:23.466 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":27,"btime":"2024-08-20T01:08:59:610307+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:59.217338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:09:23.466 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 27 2024-08-20T01:09:23.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 msgr2=0x7f27400b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:23.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 0x7f27400b6320 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f274c031440 tx=0x7f274c005b50 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 msgr2=0x7f275c19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f274800e9f0 tx=0x7f274800edf0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 shutdown_connections 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27400b3e60 0x7f27400b6320 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f275c10ef30 0x7f275c1a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f275c075b10 0x7f275c19b4c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 --2- 172.21.15.138:0/138065556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f275c0751f0 0x7f275c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:23.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 >> 172.21.15.138:0/138065556 conn(0x7f275c0fe350 msgr2=0x7f275c1110d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:23.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 shutdown_connections 2024-08-20T01:09:23.470 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:23.470+0000 7f275b200640 1 -- 172.21.15.138:0/138065556 wait complete. 2024-08-20T01:09:23.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:23 smithi138 ceph-mon[18076]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:23.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:23 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/138065556' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:23 smithi167 ceph-mon[21965]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:23 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/138065556' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:23.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:23 smithi142 ceph-mon[21799]: pgmap v400: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:23 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/138065556' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:24.354 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id d5539d4a --release --await' 2024-08-20T01:09:24.376 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id d5539d4a --release --await 2024-08-20T01:09:24.815 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.510+0000 7f310ea00640 1 -- 172.21.15.138:0/2320670321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 msgr2=0x7f3108105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.510+0000 7f310ea00640 1 --2- 172.21.15.138:0/2320670321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f3108105080 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009860 tx=0x7f30f802a340 comp rx=0 tx=0).stop 2024-08-20T01:09:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 -- 172.21.15.138:0/2320670321 shutdown_connections 2024-08-20T01:09:25.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 --2- 172.21.15.138:0/2320670321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810b030 0x7f3108111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 --2- 172.21.15.138:0/2320670321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810a6b0 0x7f310810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 --2- 172.21.15.138:0/2320670321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f3108105080 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 -- 172.21.15.138:0/2320670321 >> 172.21.15.138:0/2320670321 conn(0x7f3108100730 msgr2=0x7f3108102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 -- 172.21.15.138:0/2320670321 shutdown_connections 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.511+0000 7f310ea00640 1 -- 172.21.15.138:0/2320670321 wait complete. 2024-08-20T01:09:25.510 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.512+0000 7f310ea00640 1 Processor -- start 2024-08-20T01:09:25.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 -- start start 2024-08-20T01:09:25.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810a6b0 0x7f310819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810b030 0x7f31081a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3108114cf0 con 0x7f310810b030 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3108114b70 con 0x7f3108104ca0 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.513+0000 7f310ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3108114e70 con 0x7f310810a6b0 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810a6b0 0x7f310819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37040/0 (socket says 172.21.15.138:37040) 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 -- 172.21.15.138:0/3861277561 learned_addr learned my addr 172.21.15.138:0/3861277561 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:25.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f310ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810b030 0x7f31081a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 -- 172.21.15.138:0/3861277561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810a6b0 msgr2=0x7f310819b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810a6b0 0x7f310819b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 -- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810b030 msgr2=0x7f31081a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810b030 0x7f31081a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.514+0000 7f3107e00640 1 -- 172.21.15.138:0/3861277561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31081a12d0 con 0x7f3108104ca0 2024-08-20T01:09:25.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.515+0000 7f3107e00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035040 tx=0x7f30f8032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.515+0000 7f3104c00640 1 -- 172.21.15.138:0/3861277561 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8037040 con 0x7f3108104ca0 2024-08-20T01:09:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.515+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31081a1560 con 0x7f3108104ca0 2024-08-20T01:09:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.515+0000 7f3104c00640 1 -- 172.21.15.138:0/3861277561 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f8041070 con 0x7f3108104ca0 2024-08-20T01:09:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.515+0000 7f3104c00640 1 -- 172.21.15.138:0/3861277561 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80035f0 con 0x7f3108104ca0 2024-08-20T01:09:25.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.516+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31081a7c80 con 0x7f3108104ca0 2024-08-20T01:09:25.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 Processor -- start 2024-08-20T01:09:25.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 -- start start 2024-08-20T01:09:25.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 0x55bf809900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf8099c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf80931670 con 0x55bf8098fcf0 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf80905d70 con 0x55bf80990610 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.519+0000 7f31114d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf808f5270 con 0x55bf8099c5a0 2024-08-20T01:09:25.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f3104c00640 1 -- 172.21.15.138:0/3861277561 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f30f8004050 con 0x7f3108104ca0 2024-08-20T01:09:25.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f3107400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f3107400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47388/0 (socket says 172.21.15.138:47388) 2024-08-20T01:09:25.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f3107400640 1 -- 172.21.15.138:0/166382243 learned_addr learned my addr 172.21.15.138:0/166382243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:25.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f3107e00640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf8099c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.520+0000 7f310ca00640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 0x55bf809900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f3107400640 1 -- 172.21.15.138:0/166382243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 msgr2=0x55bf8099c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f3107400640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf8099c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f3107400640 1 -- 172.21.15.138:0/166382243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 msgr2=0x55bf809900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f3107400640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 0x55bf809900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f3107400640 1 -- 172.21.15.138:0/166382243 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf80891870 con 0x55bf8099c5a0 2024-08-20T01:09:25.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.521+0000 7f310ca00640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 0x55bf809900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:09:25.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.522+0000 7f3107400640 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009ca0 tx=0x7f30f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=225aacdad4aa6d32 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.522+0000 7f3104c00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 0x7f30ec0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.522+0000 7f30fd200640 1 -- 172.21.15.138:0/166382243 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004230 con 0x55bf8099c5a0 2024-08-20T01:09:25.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.523+0000 7f3104c00640 1 -- 172.21.15.138:0/3861277561 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f30f8103d20 con 0x7f3108104ca0 2024-08-20T01:09:25.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.523+0000 7f30fd200640 1 -- 172.21.15.138:0/166382243 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40043d0 con 0x55bf8099c5a0 2024-08-20T01:09:25.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.523+0000 7f30fd200640 1 -- 172.21.15.138:0/166382243 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f40196d0 con 0x55bf8099c5a0 2024-08-20T01:09:25.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.523+0000 7f310ca00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 0x7f30ec0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 -- 172.21.15.138:0/166382243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 msgr2=0x55bf8099e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f30f4009ca0 tx=0x7f30f4011810 comp rx=0 tx=0).stop 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f310ca00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 0x7f30ec0b6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 -- 172.21.15.138:0/166382243 shutdown_connections 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8099c5a0 0x55bf8099e990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf8099c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 --2- 172.21.15.138:0/166382243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8098fcf0 0x55bf809900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 -- 172.21.15.138:0/166382243 >> 172.21.15.138:0/166382243 conn(0x55bf8098db20 msgr2=0x55bf8098ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 -- 172.21.15.138:0/166382243 shutdown_connections 2024-08-20T01:09:25.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.524+0000 7f31114d7740 1 -- 172.21.15.138:0/166382243 wait complete. 2024-08-20T01:09:25.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.525+0000 7f31114d7740 1 -- start start 2024-08-20T01:09:25.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.525+0000 7f31114d7740 1 Processor -- start 2024-08-20T01:09:25.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f31114d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 0x55bf8099bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f31114d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf80990610 0x55bf80999210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f31114d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80999750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f3107e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 0x55bf8099bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f3107e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 0x55bf8099bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37050/0 (socket says 172.21.15.138:37050) 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f3107e00640 1 -- 172.21.15.138:0/3760400211 learned_addr learned my addr 172.21.15.138:0/3760400211 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f3107400640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf80990610 0x55bf80999210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.526+0000 7f310ca00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80999750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.527+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bf8087bff0 con 0x55bf8099c5a0 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.527+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bf808f92b0 con 0x55bf8098fcf0 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.527+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bf80757bb0 con 0x55bf80990610 2024-08-20T01:09:25.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.527+0000 7f31114d7740 10 client.0 _ll_register_callbacks cb 0x55bf80600d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 -- 172.21.15.138:0/3760400211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf80990610 msgr2=0x55bf80999210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf80990610 0x55bf80999210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 -- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 msgr2=0x55bf8099bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 0x55bf8099bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf80654b00 con 0x55bf8099c5a0 2024-08-20T01:09:25.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f310ca00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80999750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f31000042c0 tx=0x7f3100004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e00ccfa3553ef7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.528+0000 7f30f1600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.529+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3100011c20 con 0x55bf8099c5a0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.529+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f40043d0 con 0x55bf8099c5a0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.529+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3100011dc0 con 0x55bf8099c5a0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.530+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f310002c560 con 0x55bf8099c5a0 2024-08-20T01:09:25.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.530+0000 7f31114d7740 10 client.15576 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:25.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.530+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf806cb960 con 0x55bf8099c5a0 2024-08-20T01:09:25.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.530+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f40043d0 con 0x55bf8099c5a0 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3100020420 con 0x55bf8099c5a0 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 10 client.15576 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 10 client.15576 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 20 client.15576 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf809b4480 0x55bf809b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 4 client.15576 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.531+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf80907cc0 con 0x55bf809b4480 2024-08-20T01:09:25.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.532+0000 7f3107e00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf809b4480 0x55bf809b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.533+0000 7f30f1600640 10 client.15576 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:25.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.533+0000 7f3107e00640 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf809b4480 0x55bf809b68c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76ef9a297d278ebd server_cookie=e8a4b51d884e33cc in_seq=0 out_seq=0 2024-08-20T01:09:25.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.533+0000 7f30f1600640 1 -- 172.21.15.138:0/3760400211 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf80907cc0 con 0x55bf809b4480 2024-08-20T01:09:25.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.533+0000 7f30f1600640 10 client.15576 handle_command_reply: tid=0 2024-08-20T01:09:25.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 client.15576 shutdown 2024-08-20T01:09:25.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 msgr2=0x55bf80999750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80999750 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f31000042c0 tx=0x7f3100004a40 comp rx=0 tx=0).stop 2024-08-20T01:09:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 shutdown_connections 2024-08-20T01:09:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf809b4480 0x55bf809b68c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80999750 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf80990610 0x55bf80999210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 --2- 172.21.15.138:0/3760400211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8098fcf0 0x55bf8099bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f30f1600640 0 client.15576 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.534+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 >> 172.21.15.138:0/3760400211 conn(0x55bf8098db20 msgr2=0x55bf809936c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.535+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 shutdown_connections 2024-08-20T01:09:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.535+0000 7f31114d7740 1 -- 172.21.15.138:0/3760400211 wait complete. 2024-08-20T01:09:25.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.535+0000 7f31114d7740 20 client.15576 trim_cache size 0 max 16384 2024-08-20T01:09:25.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 Processor -- start 2024-08-20T01:09:25.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 -- start start 2024-08-20T01:09:25.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf80990610 0x55bf809b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8099c5a0 0x55bf80997ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31081b01c0 con 0x55bf80990610 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ada90 con 0x55bf8099c5a0 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.554+0000 7f31114d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ab900 con 0x55bf8098fcf0 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47404/0 (socket says 172.21.15.138:47404) 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 -- 172.21.15.138:0/1033570994 learned_addr learned my addr 172.21.15.138:0/1033570994 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f310ca00640 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8099c5a0 0x55bf80997ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 -- 172.21.15.138:0/1033570994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8099c5a0 msgr2=0x55bf80997ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8099c5a0 0x55bf80997ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 -- 172.21.15.138:0/1033570994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf80990610 msgr2=0x55bf809b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107400640 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf80990610 0x55bf809b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf80990610 0x55bf809b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 -- 172.21.15.138:0/1033570994 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf80990440 con 0x55bf8098fcf0 2024-08-20T01:09:25.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f3107e00640 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30f802a380 tx=0x7f30f80d1040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb8cb2993612fa71 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f30f3e00640 1 -- 172.21.15.138:0/1033570994 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80ca5b0 con 0x55bf8098fcf0 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f30f3e00640 1 -- 172.21.15.138:0/1033570994 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f80328f0 con 0x55bf8098fcf0 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.555+0000 7f30f3e00640 1 -- 172.21.15.138:0/1033570994 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80c9f50 con 0x55bf8098fcf0 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 -- 172.21.15.138:0/1033570994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 msgr2=0x55bf8099aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f30f802a380 tx=0x7f30f80d1040 comp rx=0 tx=0).stop 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 -- 172.21.15.138:0/1033570994 shutdown_connections 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf8099c5a0 0x55bf80997ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf80990610 0x55bf809b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 --2- 172.21.15.138:0/1033570994 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf8099aeb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.556+0000 7f31114d7740 1 -- 172.21.15.138:0/1033570994 >> 172.21.15.138:0/1033570994 conn(0x55bf8098db20 msgr2=0x55bf8098cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.557+0000 7f31114d7740 1 -- 172.21.15.138:0/1033570994 shutdown_connections 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.557+0000 7f31114d7740 1 -- 172.21.15.138:0/1033570994 wait complete. 2024-08-20T01:09:25.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.557+0000 7f31114d7740 1 -- start start 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.557+0000 7f31114d7740 1 Processor -- start 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf809ab930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80a1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f31081abf50 con 0x55bf8099c5a0 2024-08-20T01:09:25.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ad3c0 con 0x55bf80990610 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f31081ac620 con 0x55bf8098fcf0 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f31114d7740 10 client.0 _ll_register_callbacks cb 0x7f3100013f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f3107e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f3107e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47406/0 (socket says 172.21.15.138:47406) 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.558+0000 7f3107e00640 1 -- 172.21.15.138:0/332396422 learned_addr learned my addr 172.21.15.138:0/332396422 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107400640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf809ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f310ca00640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80a1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 -- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 msgr2=0x55bf809ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf809ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 -- 172.21.15.138:0/332396422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 msgr2=0x55bf80a1e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80a1e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 -- 172.21.15.138:0/332396422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf80a1e8a0 con 0x55bf8098fcf0 2024-08-20T01:09:25.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f3107e00640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009230 tx=0x7f30f81071a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37166221e0b07fa2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f30f1600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f80ca130 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.559+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f30f80c6dd0 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30f80328f0 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f30f80c9f50 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 10 client.25588 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bf80a1fbe0 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30f80328f0 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f30f8004340 con 0x55bf8098fcf0 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 10 client.25588 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 10 client.25588 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 20 client.25588 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:25.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf80a20200 0x55bf80a22620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 4 client.25588 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d5539d4a", "release": true, "await": true}] 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.560+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d5539d4a", "release": true, "await": true}) -- 0x55bf80903760 con 0x55bf80a20200 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.561+0000 7f3107400640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf80a20200 0x55bf80a22620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.561+0000 7f3107400640 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf80a20200 0x55bf80a22620 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a802bd04f786ebf9 server_cookie=691f6c25f76e690f in_seq=0 out_seq=0 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.561+0000 7f30f1600640 10 client.25588 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:25.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.563+0000 7f30f1600640 1 -- 172.21.15.138:0/332396422 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55bf80903760 con 0x55bf80a20200 2024-08-20T01:09:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.563+0000 7f30f1600640 10 client.25588 handle_command_reply: tid=0 2024-08-20T01:09:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.563+0000 7f31114d7740 1 client.25588 shutdown 2024-08-20T01:09:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.563+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 msgr2=0x55bf809ab3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.563+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f30f8009230 tx=0x7f30f81071a0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 shutdown_connections 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bf80a20200 0x55bf80a22620 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bf8099c5a0 0x55bf80a1e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf80990610 0x55bf809ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 --2- 172.21.15.138:0/332396422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bf8098fcf0 0x55bf809ab3d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 >> 172.21.15.138:0/332396422 conn(0x55bf8098db20 msgr2=0x55bf809afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f30f1600640 0 client.25588 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 shutdown_connections 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 1 -- 172.21.15.138:0/332396422 wait complete. 2024-08-20T01:09:25.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.564+0000 7f31114d7740 20 client.25588 trim_cache size 0 max 16384 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 4, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "d5539d4a": { 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 4, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:09:25.564 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.0, 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:25.565 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.565+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 msgr2=0x7f30ec0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.565+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 0x7f30ec0b6300 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3100005fd0 tx=0x7f3100009500 comp rx=0 tx=0).stop 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.565+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 msgr2=0x7f310819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.565+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f30f8035040 tx=0x7f30f8032d50 comp rx=0 tx=0).stop 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 shutdown_connections 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f30ec0b3e40 0x7f30ec0b6300 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f310810b030 0x7f31081a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f310810a6b0 0x7f310819b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 --2- 172.21.15.138:0/3861277561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3108104ca0 0x7f310819b220 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 >> 172.21.15.138:0/3861277561 conn(0x7f3108100730 msgr2=0x7f3108102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 shutdown_connections 2024-08-20T01:09:25.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:25.566+0000 7f310ea00640 1 -- 172.21.15.138:0/3861277561 wait complete. 2024-08-20T01:09:25.793 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:25 smithi138 ceph-mon[18076]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:25 smithi167 ceph-mon[21965]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:25 smithi142 ceph-mon[21799]: pgmap v401: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-08-20T01:09:26.396 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd5539d4a', total seconds elapsed: 14.5 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 -- 172.21.15.138:0/3327275952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 msgr2=0x7f3ab4111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 --2- 172.21.15.138:0/3327275952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab4111780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009ca0 tx=0x7f3aac02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 -- 172.21.15.138:0/3327275952 shutdown_connections 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 --2- 172.21.15.138:0/3327275952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab4111780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 --2- 172.21.15.138:0/3327275952 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab410a600 0x7f3ab410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 --2- 172.21.15.138:0/3327275952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab4104c00 0x7f3ab4104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.499+0000 7f3ab9000640 1 -- 172.21.15.138:0/3327275952 >> 172.21.15.138:0/3327275952 conn(0x7f3ab4100640 msgr2=0x7f3ab4102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.500+0000 7f3ab9000640 1 -- 172.21.15.138:0/3327275952 shutdown_connections 2024-08-20T01:09:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.500+0000 7f3ab9000640 1 -- 172.21.15.138:0/3327275952 wait complete. 2024-08-20T01:09:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 Processor -- start 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 -- start start 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4104c00 0x7f3ab419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab41a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4114c90 con 0x7f3ab410a600 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4114b10 con 0x7f3ab410af80 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab9000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4114e10 con 0x7f3ab4104c00 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57276/0 (socket says 172.21.15.138:57276) 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.501+0000 7f3ab1400640 1 -- 172.21.15.138:0/2124829605 learned_addr learned my addr 172.21.15.138:0/2124829605 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:27.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4104c00 0x7f3ab419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab2800640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab41a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 -- 172.21.15.138:0/2124829605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4104c00 msgr2=0x7f3ab419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4104c00 0x7f3ab419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 -- 172.21.15.138:0/2124829605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 msgr2=0x7f3ab41a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab41a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 -- 172.21.15.138:0/2124829605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a1150 con 0x7f3ab410a600 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab2800640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.502+0000 7f3ab1400640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d950 tx=0x7f3aa000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.503+0000 7f3aaaa00640 1 -- 172.21.15.138:0/2124829605 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0017070 con 0x7f3ab410a600 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.503+0000 7f3aaaa00640 1 -- 172.21.15.138:0/2124829605 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa0009c30 con 0x7f3ab410a600 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.503+0000 7f3aaaa00640 1 -- 172.21.15.138:0/2124829605 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa000f620 con 0x7f3ab410a600 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.503+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a1440 con 0x7f3ab410a600 2024-08-20T01:09:27.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.503+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a7b00 con 0x7f3ab410a600 2024-08-20T01:09:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.507+0000 7f3abbe79740 1 Processor -- start 2024-08-20T01:09:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.507+0000 7f3aaaa00640 1 -- 172.21.15.138:0/2124829605 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3aa0010460 con 0x7f3ab410a600 2024-08-20T01:09:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.507+0000 7f3abbe79740 1 -- start start 2024-08-20T01:09:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557572a01e0 0x5557572a25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555757209210 con 0x555757294250 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5557571f8710 con 0x555757293930 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3abbe79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557571fc750 con 0x5557572a01e0 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3ab2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37092/0 (socket says 172.21.15.138:37092) 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.508+0000 7f3ab2800640 1 -- 172.21.15.138:0/3478042038 learned_addr learned my addr 172.21.15.138:0/3478042038 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab1400640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557572a01e0 0x5557572a25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 -- 172.21.15.138:0/3478042038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557572a01e0 msgr2=0x5557572a25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557572a01e0 0x5557572a25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 -- 172.21.15.138:0/3478042038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 msgr2=0x55575729fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 -- 172.21.15.138:0/3478042038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557571c4de0 con 0x555757293930 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3aaaa00640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 0x7f3a980b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3ab2800640 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3aac035000 tx=0x7f3aac009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a42eabc9576838b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.509+0000 7f3aaaa00640 1 -- 172.21.15.138:0/2124829605 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3aa00e3f00 con 0x7f3ab410a600 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.510+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 0x7f3a980b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.510+0000 7f3a9f400640 1 -- 172.21.15.138:0/3478042038 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac031cf0 con 0x555757293930 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.510+0000 7f3a9f400640 1 -- 172.21.15.138:0/3478042038 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x555757293930 2024-08-20T01:09:27.509 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.510+0000 7f3a9f400640 1 -- 172.21.15.138:0/3478042038 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0042a0 con 0x555757293930 2024-08-20T01:09:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 -- 172.21.15.138:0/3478042038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 msgr2=0x555757293d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3aac035000 tx=0x7f3aac009800 comp rx=0 tx=0).stop 2024-08-20T01:09:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 0x7f3a980b63d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x555757296ab0 tx=0x7f3aa4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.510 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 -- 172.21.15.138:0/3478042038 shutdown_connections 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557572a01e0 0x5557572a25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3478042038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x555757293d10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 -- 172.21.15.138:0/3478042038 >> 172.21.15.138:0/3478042038 conn(0x555757291760 msgr2=0x555757292900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 -- 172.21.15.138:0/3478042038 shutdown_connections 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.511+0000 7f3abbe79740 1 -- 172.21.15.138:0/3478042038 wait complete. 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 -- start start 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 Processor -- start 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x55575729d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555757131ab0 con 0x555757294250 2024-08-20T01:09:27.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55575719f140 con 0x5557572a01e0 2024-08-20T01:09:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557572a2e60 con 0x555757293930 2024-08-20T01:09:27.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.512+0000 7f3abbe79740 10 client.0 _ll_register_callbacks cb 0x5557571b25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47446/0 (socket says 172.21.15.138:47446) 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 -- 172.21.15.138:0/3047711931 learned_addr learned my addr 172.21.15.138:0/3047711931 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab2800640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x55575729d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 -- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 msgr2=0x55575729d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x55575729d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 -- 172.21.15.138:0/3047711931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 msgr2=0x55575729ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.513+0000 7f3ab1e00640 1 -- 172.21.15.138:0/3047711931 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557571a0170 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3aa401e9d0 tx=0x7f3aa4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3e67283ed396be1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa402e060 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aac036040 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3aa4018030 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3aa4009db0 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3abbe79740 10 client.25600 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.514+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555756f5b790 con 0x555757293930 2024-08-20T01:09:27.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.515+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac036040 con 0x555757293930 2024-08-20T01:09:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3aa4029420 con 0x555757293930 2024-08-20T01:09:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 10 client.25600 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 10 client.25600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 20 client.25600 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:27.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557572afe90 0x5557572b2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 4 client.25600 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.516+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575720b160 con 0x5557572afe90 2024-08-20T01:09:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.517+0000 7f3ab1400640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557572afe90 0x5557572b2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.517+0000 7f3ab1400640 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557572afe90 0x5557572b2300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b32f2bf6f29269 server_cookie=48afce51f966eadc in_seq=0 out_seq=0 2024-08-20T01:09:27.516 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.517+0000 7f3a93e00640 10 client.25600 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3a93e00640 1 -- 172.21.15.138:0/3047711931 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55575720b160 con 0x5557572afe90 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3a93e00640 10 client.25600 handle_command_reply: tid=0 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 client.25600 shutdown 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 msgr2=0x55575729fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3aa401e9d0 tx=0x7f3aa4004730 comp rx=0 tx=0).stop 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 shutdown_connections 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557572afe90 0x5557572b2300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x55575729d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 --2- 172.21.15.138:0/3047711931 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729fa50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 >> 172.21.15.138:0/3047711931 conn(0x555757291760 msgr2=0x555757297300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.517 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3a93e00640 0 client.25600 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 shutdown_connections 2024-08-20T01:09:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 1 -- 172.21.15.138:0/3047711931 wait complete. 2024-08-20T01:09:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.518+0000 7f3abbe79740 20 client.25600 trim_cache size 0 max 16384 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.533+0000 7f3abbe79740 1 Processor -- start 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 -- start start 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x5557572b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b4570 con 0x555757294250 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b1e40 con 0x5557572a01e0 2024-08-20T01:09:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3abbe79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41afc90 con 0x555757293930 2024-08-20T01:09:27.533 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3ab1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:47452/0 (socket says 172.21.15.138:47452) 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3ab1e00640 1 -- 172.21.15.138:0/4113438656 learned_addr learned my addr 172.21.15.138:0/4113438656 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.534+0000 7f3ab2800640 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x5557572b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 -- 172.21.15.138:0/4113438656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 msgr2=0x55575729c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 -- 172.21.15.138:0/4113438656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 msgr2=0x5557572b7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x5557572b7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 -- 172.21.15.138:0/4113438656 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555756fd4620 con 0x555757294250 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3ab1400640 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ebd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00e12a0 tx=0x7f3aa00e62d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01d3c3ede4a690d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3a9ea00640 1 -- 172.21.15.138:0/4113438656 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0014070 con 0x555757294250 2024-08-20T01:09:27.534 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3a9ea00640 1 -- 172.21.15.138:0/4113438656 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00aa270 con 0x555757294250 2024-08-20T01:09:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.535+0000 7f3a9ea00640 1 -- 172.21.15.138:0/4113438656 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa0020040 con 0x555757294250 2024-08-20T01:09:27.535 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 -- 172.21.15.138:0/4113438656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 msgr2=0x55575729ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ebd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00e12a0 tx=0x7f3aa00e62d0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 -- 172.21.15.138:0/4113438656 shutdown_connections 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557572a01e0 0x5557572b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555757294250 0x55575729ebd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 --2- 172.21.15.138:0/4113438656 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757293930 0x55575729c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.536+0000 7f3abbe79740 1 -- 172.21.15.138:0/4113438656 >> 172.21.15.138:0/4113438656 conn(0x555757291760 msgr2=0x55575729e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.537+0000 7f3abbe79740 1 -- 172.21.15.138:0/4113438656 shutdown_connections 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.537+0000 7f3abbe79740 1 -- 172.21.15.138:0/4113438656 wait complete. 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.537+0000 7f3abbe79740 1 -- start start 2024-08-20T01:09:27.536 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.537+0000 7f3abbe79740 1 Processor -- start 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757294250 0x5557572a5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557572a01e0 0x555757318ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b0300 con 0x5557572a01e0 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b1770 con 0x555757293930 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab41b09d0 con 0x555757294250 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3abbe79740 10 client.0 _ll_register_callbacks cb 0x7f3aa4009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3ab1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37130/0 (socket says 172.21.15.138:37130) 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3ab1e00640 1 -- 172.21.15.138:0/2043420616 learned_addr learned my addr 172.21.15.138:0/2043420616 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3ab2800640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557572a01e0 0x555757318ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.538+0000 7f3ab1e00640 1 -- 172.21.15.138:0/2043420616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757294250 msgr2=0x5557572a5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757294250 0x5557572a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3ab1e00640 1 -- 172.21.15.138:0/2043420616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557572a01e0 msgr2=0x555757318ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557572a01e0 0x555757318ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3ab1e00640 1 -- 172.21.15.138:0/2043420616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557573190e0 con 0x555757293930 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3ab1e00640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3aa401e9d0 tx=0x7f3aa4029c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96f9d8b57e1feb5f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3a93e00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa403f020 con 0x555757293930 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3aa00aa270 con 0x555757293930 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3aa403b680 con 0x555757293930 2024-08-20T01:09:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3aa4009ac0 con 0x555757293930 2024-08-20T01:09:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.539+0000 7f3abbe79740 10 client.25640 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.540+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55575731a2d0 con 0x555757293930 2024-08-20T01:09:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.540+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00aa270 con 0x555757293930 2024-08-20T01:09:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.540+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f3aa403b880 con 0x555757293930 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 10 client.25640 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 10 client.25640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 20 client.25640 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55575731aa40 0x55575731ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 4 client.25640 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555757206c00 con 0x55575731aa40 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3ab1400640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55575731aa40 0x55575731ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3a93e00640 10 client.25640 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:27.541 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.541+0000 7f3ab1400640 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55575731aa40 0x55575731ce60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663cbd992295ca57 server_cookie=309e7d72a8028fc4 in_seq=0 out_seq=0 2024-08-20T01:09:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.545+0000 7f3a93e00640 1 -- 172.21.15.138:0/2043420616 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555757206c00 con 0x55575731aa40 2024-08-20T01:09:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.545+0000 7f3a93e00640 10 client.25640 handle_command_reply: tid=0 2024-08-20T01:09:27.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.545+0000 7f3abbe79740 1 client.25640 shutdown 2024-08-20T01:09:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 msgr2=0x5557572a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3aa401e9d0 tx=0x7f3aa4029c80 comp rx=0 tx=0).stop 2024-08-20T01:09:27.544 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 shutdown_connections 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55575731aa40 0x55575731ce60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557572a01e0 0x555757318ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555757294250 0x5557572a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 --2- 172.21.15.138:0/2043420616 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555757293930 0x5557572a52c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 >> 172.21.15.138:0/2043420616 conn(0x555757291760 msgr2=0x555757290760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3a93e00640 0 client.25640 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 shutdown_connections 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 1 -- 172.21.15.138:0/2043420616 wait complete. 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.546+0000 7f3abbe79740 20 client.25640 trim_cache size 0 max 16384 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:09:27.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:09:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:09:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:09:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.547+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 msgr2=0x7f3a980b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.547+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 0x7f3a980b63d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x555757296ab0 tx=0x7f3aa4005f90 comp rx=0 tx=0).stop 2024-08-20T01:09:27.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.547+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 msgr2=0x7f3ab419b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.547+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d950 tx=0x7f3aa000de10 comp rx=0 tx=0).stop 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 shutdown_connections 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3a980b3f10 0x7f3a980b63d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ab410af80 0x7f3ab41a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab410a600 0x7f3ab419b5f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 --2- 172.21.15.138:0/2124829605 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3ab4104c00 0x7f3ab419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 >> 172.21.15.138:0/2124829605 conn(0x7f3ab4100640 msgr2=0x7f3ab4102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 shutdown_connections 2024-08-20T01:09:27.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:27.548+0000 7f3ab9000640 1 -- 172.21.15.138:0/2124829605 wait complete. 2024-08-20T01:09:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:27 smithi167 ceph-mon[21965]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:27.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:27 smithi142 ceph-mon[21799]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:28.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:27 smithi138 ceph-mon[18076]: pgmap v402: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-08-20T01:09:29.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:29 smithi167 ceph-mon[21965]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:09:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:29 smithi142 ceph-mon[21799]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:09:30.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:29 smithi138 ceph-mon[18076]: pgmap v403: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 5 op/s 2024-08-20T01:09:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:31 smithi167 ceph-mon[21965]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-20T01:09:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:31 smithi142 ceph-mon[21799]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-20T01:09:32.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:31 smithi138 ceph-mon[18076]: pgmap v404: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 853 B/s wr, 8 op/s 2024-08-20T01:09:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:33 smithi167 ceph-mon[21965]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T01:09:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:33 smithi142 ceph-mon[21799]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T01:09:34.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:33 smithi138 ceph-mon[18076]: pgmap v405: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 6 op/s 2024-08-20T01:09:34.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:34 smithi167 ceph-mon[21965]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:34 smithi142 ceph-mon[21799]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:35.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:34 smithi138 ceph-mon[18076]: pgmap v406: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:37 smithi167 ceph-mon[21965]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-20T01:09:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:37 smithi142 ceph-mon[21799]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-20T01:09:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:37 smithi138 ceph-mon[18076]: pgmap v407: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 767 B/s wr, 8 op/s 2024-08-20T01:09:39.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:39 smithi138 ceph-mon[18076]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:39 smithi167 ceph-mon[21965]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:39 smithi142 ceph-mon[21799]: pgmap v408: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 -- 172.21.15.138:0/2275439306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410a610 msgr2=0x7f968410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 --2- 172.21.15.138:0/2275439306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410a610 0x7f968410aa50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f967002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 -- 172.21.15.138:0/2275439306 shutdown_connections 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 --2- 172.21.15.138:0/2275439306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410af90 0x7f9684111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 --2- 172.21.15.138:0/2275439306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410a610 0x7f968410aa50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 --2- 172.21.15.138:0/2275439306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f9684104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 -- 172.21.15.138:0/2275439306 >> 172.21.15.138:0/2275439306 conn(0x7f9684100610 msgr2=0x7f9684102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 -- 172.21.15.138:0/2275439306 shutdown_connections 2024-08-20T01:09:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.674+0000 7f968a800640 1 -- 172.21.15.138:0/2275439306 wait complete. 2024-08-20T01:09:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.675+0000 7f968a800640 1 Processor -- start 2024-08-20T01:09:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.675+0000 7f968a800640 1 -- start start 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410a610 0x7f968419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9684114ca0 con 0x7f968410af90 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9684114b20 con 0x7f9684104c10 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f968a800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9684114e20 con 0x7f968410a610 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56058/0 (socket says 172.21.15.138:56058) 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48914/0 (socket says 172.21.15.138:48914) 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683e00640 1 -- 172.21.15.138:0/3460225315 learned_addr learned my addr 172.21.15.138:0/3460225315 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9682a00640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410a610 0x7f968419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 -- 172.21.15.138:0/3460225315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410a610 msgr2=0x7f968419b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410a610 0x7f968419b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 -- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 msgr2=0x7f96841a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683400640 1 -- 172.21.15.138:0/3460225315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96841a11e0 con 0x7f9684104c10 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.676+0000 7f9683e00640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f9683400640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f967400e9f0 tx=0x7f967400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f967be00640 1 -- 172.21.15.138:0/3460225315 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967400ccc0 con 0x7f9684104c10 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96841a14d0 con 0x7f9684104c10 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f967be00640 1 -- 172.21.15.138:0/3460225315 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96740108e0 con 0x7f9684104c10 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f967be00640 1 -- 172.21.15.138:0/3460225315 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9674018880 con 0x7f9684104c10 2024-08-20T01:09:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.677+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841a7d70 con 0x7f9684104c10 2024-08-20T01:09:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.680+0000 7f968d6b6740 1 Processor -- start 2024-08-20T01:09:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.680+0000 7f968d6b6740 1 -- start start 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc159a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 0x55cdc15a9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1510210 con 0x55cdc15a6e80 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cdc14ff710 con 0x55cdc159aef0 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f968d6b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1503750 con 0x55cdc159a5d0 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f967be00640 1 -- 172.21.15.138:0/3460225315 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9674010430 con 0x7f9684104c10 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f9682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 0x55cdc15a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f9683e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc159a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f9682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 0x55cdc15a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56066/0 (socket says 172.21.15.138:56066) 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f9682a00640 1 -- 172.21.15.138:0/2945329495 learned_addr learned my addr 172.21.15.138:0/2945329495 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.681+0000 7f9683e00640 1 -- 172.21.15.138:0/2945329495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 msgr2=0x55cdc15a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9683e00640 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9683e00640 1 -- 172.21.15.138:0/2945329495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 msgr2=0x55cdc15a9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9683e00640 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 0x55cdc15a9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9683e00640 1 -- 172.21.15.138:0/2945329495 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc14cbde0 con 0x55cdc159a5d0 2024-08-20T01:09:40.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9683e00640 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc159a9b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f967c0117e0 tx=0x7f967c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f76ffb33d3ee08fa server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9678c00640 1 -- 172.21.15.138:0/2945329495 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c011310 con 0x55cdc159a5d0 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9678c00640 1 -- 172.21.15.138:0/2945329495 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f967c00f9a0 con 0x55cdc159a5d0 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f9678c00640 1 -- 172.21.15.138:0/2945329495 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c005700 con 0x55cdc159a5d0 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.682+0000 7f967be00640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 0x7f96680b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f967be00640 1 -- 172.21.15.138:0/3460225315 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f96740e4650 con 0x7f9684104c10 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f9683400640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 0x7f96680b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 -- 172.21.15.138:0/2945329495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 msgr2=0x55cdc159a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc159a9b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f967c0117e0 tx=0x7f967c011ca0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 -- 172.21.15.138:0/2945329495 shutdown_connections 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc15a6e80 0x55cdc15a9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 --2- 172.21.15.138:0/2945329495 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc159a9b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 -- 172.21.15.138:0/2945329495 >> 172.21.15.138:0/2945329495 conn(0x55cdc15983b0 msgr2=0x55cdc15995a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 -- 172.21.15.138:0/2945329495 shutdown_connections 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.683+0000 7f968d6b6740 1 -- 172.21.15.138:0/2945329495 wait complete. 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f9683400640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 0x7f96680b6280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55cdc159d750 tx=0x7f967400ce60 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 -- start start 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 Processor -- start 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc15a6e80 0x55cdc15a4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cdc1438ab0 con 0x55cdc159a5d0 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cdc14a6140 con 0x55cdc159aef0 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.684+0000 7f968d6b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cdc15a9b60 con 0x55cdc15a6e80 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f968d6b6740 10 client.0 _ll_register_callbacks cb 0x55cdc14b95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56072/0 (socket says 172.21.15.138:56072) 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9683e00640 1 -- 172.21.15.138:0/1660420494 learned_addr learned my addr 172.21.15.138:0/1660420494 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9683e00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc15a6e80 0x55cdc15a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 -- 172.21.15.138:0/1660420494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc15a6e80 msgr2=0x55cdc15a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc15a6e80 0x55cdc15a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 -- 172.21.15.138:0/1660420494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 msgr2=0x55cdc15a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9682a00640 1 -- 172.21.15.138:0/1660420494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc14a7170 con 0x55cdc159aef0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.685+0000 7f9683e00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f9682a00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a3cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f9670004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3816cae8e06c9f2f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f966d600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:09:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035040 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9670005510 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cdc14d6a30 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f967004f050 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f968d6b6740 10 client.25649 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cdc1262790 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.686+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55cdc14d6a30 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f9670003a00 con 0x55cdc159aef0 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 10 client.25649 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 10 client.25649 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 20 client.25649 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc15b6c50 0x55cdc15b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 4 client.25649 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdc1512160 con 0x55cdc15b6c50 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f9683e00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc15b6c50 0x55cdc15b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f9683e00640 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc15b6c50 0x55cdc15b90f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ef1fb0ba56acb66 server_cookie=14ef133c63f3dc1d in_seq=0 out_seq=0 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.687+0000 7f966d600640 10 client.25649 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.688+0000 7f966d600640 1 -- 172.21.15.138:0/1660420494 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cdc1512160 con 0x55cdc15b6c50 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.688+0000 7f966d600640 10 client.25649 handle_command_reply: tid=0 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.688+0000 7f968d6b6740 1 client.25649 shutdown 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.688+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 msgr2=0x55cdc15a3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.688+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a3cc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9670006010 tx=0x7f9670004440 comp rx=0 tx=0).stop 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 shutdown_connections 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc15b6c50 0x55cdc15b90f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc15a6e80 0x55cdc15a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc159aef0 0x55cdc15a3cc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f966d600640 0 client.25649 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 --2- 172.21.15.138:0/1660420494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc15a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 >> 172.21.15.138:0/1660420494 conn(0x55cdc15983b0 msgr2=0x55cdc159dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 shutdown_connections 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 1 -- 172.21.15.138:0/1660420494 wait complete. 2024-08-20T01:09:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.689+0000 7f968d6b6740 20 client.25649 trim_cache size 0 max 16384 2024-08-20T01:09:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.704+0000 7f968d6b6740 1 Processor -- start 2024-08-20T01:09:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.704+0000 7f968d6b6740 1 -- start start 2024-08-20T01:09:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc1593230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 0x55cdc15be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b4550 con 0x55cdc159a5d0 2024-08-20T01:09:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b1e20 con 0x55cdc15a6e80 2024-08-20T01:09:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f968d6b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96841afc70 con 0x55cdc159aef0 2024-08-20T01:09:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc1593230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 0x55cdc15be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9683e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc1593230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56074/0 (socket says 172.21.15.138:56074) 2024-08-20T01:09:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9683e00640 1 -- 172.21.15.138:0/78270823 learned_addr learned my addr 172.21.15.138:0/78270823 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9683e00640 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 -- 172.21.15.138:0/78270823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 msgr2=0x55cdc15bef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15bef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 -- 172.21.15.138:0/78270823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 msgr2=0x55cdc1593230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc1593230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9682a00640 1 -- 172.21.15.138:0/78270823 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdc12db620 con 0x55cdc159aef0 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.705+0000 7f9683e00640 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.706+0000 7f9682a00640 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 0x55cdc15be9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f9670005d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9253ba2124af486 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.706+0000 7f966fe00640 1 -- 172.21.15.138:0/78270823 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035070 con 0x55cdc159aef0 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.706+0000 7f966fe00640 1 -- 172.21.15.138:0/78270823 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9670041070 con 0x55cdc159aef0 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.706+0000 7f966fe00640 1 -- 172.21.15.138:0/78270823 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670003be0 con 0x55cdc159aef0 2024-08-20T01:09:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- 172.21.15.138:0/78270823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 msgr2=0x55cdc15be9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 0x55cdc15be9c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f9670005d50 comp rx=0 tx=0).stop 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- 172.21.15.138:0/78270823 shutdown_connections 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159aef0 0x55cdc15be9c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 --2- 172.21.15.138:0/78270823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159a5d0 0x55cdc1593230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- 172.21.15.138:0/78270823 >> 172.21.15.138:0/78270823 conn(0x55cdc15983b0 msgr2=0x55cdc15a2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- 172.21.15.138:0/78270823 shutdown_connections 2024-08-20T01:09:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- 172.21.15.138:0/78270823 wait complete. 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 -- start start 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.707+0000 7f968d6b6740 1 Processor -- start 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc15a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 0x55cdc15a7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b02e0 con 0x55cdc159aef0 2024-08-20T01:09:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b1750 con 0x55cdc15a6e80 2024-08-20T01:09:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f96841b09b0 con 0x55cdc159a5d0 2024-08-20T01:09:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f968d6b6740 10 client.0 _ll_register_callbacks cb 0x7f96700040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9683e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9682a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 0x55cdc15a7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9683e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48950/0 (socket says 172.21.15.138:48950) 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9683e00640 1 -- 172.21.15.138:0/4123543940 learned_addr learned my addr 172.21.15.138:0/4123543940 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9683e00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc15a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9682a00640 1 -- 172.21.15.138:0/4123543940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 msgr2=0x55cdc15a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9682a00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc15a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9682a00640 1 -- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 msgr2=0x55cdc15ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.708+0000 7f9682a00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f9682a00640 1 -- 172.21.15.138:0/4123543940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdc15ae3e0 con 0x55cdc159aef0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f9683e00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f9682a00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 0x55cdc15a7fc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f9670003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=485dba7de187b340 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f966d600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9670035070 con 0x55cdc159aef0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9670003920 con 0x55cdc159aef0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9670041070 con 0x55cdc159aef0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9670062050 con 0x55cdc159aef0 2024-08-20T01:09:40.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f968d6b6740 10 client.15612 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.709+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cdc15af660 con 0x55cdc159aef0 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9670041070 con 0x55cdc159aef0 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f967005b060 con 0x55cdc159aef0 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 10 client.15612 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 20 client.15612 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc1621a30 0x55cdc1623e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 4 client.15612 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cdc150dc00 con 0x55cdc1621a30 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.710+0000 7f9683e00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc1621a30 0x55cdc1623e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.711+0000 7f9683e00640 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc1621a30 0x55cdc1623e50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bbaea8cca0f68dc server_cookie=1000a33473b35cbe in_seq=0 out_seq=0 2024-08-20T01:09:40.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.711+0000 7f966d600640 10 client.15612 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.715+0000 7f966d600640 1 -- 172.21.15.138:0/4123543940 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cdc150dc00 con 0x55cdc1621a30 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.715+0000 7f966d600640 10 client.15612 handle_command_reply: tid=0 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.715+0000 7f968d6b6740 1 client.15612 shutdown 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.715+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 msgr2=0x55cdc15a7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.715+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 0x55cdc15a7fc0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9670009900 tx=0x7f9670003820 comp rx=0 tx=0).stop 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 shutdown_connections 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cdc1621a30 0x55cdc1623e50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cdc15a6e80 0x55cdc15ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cdc159aef0 0x55cdc15a7fc0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 --2- 172.21.15.138:0/4123543940 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cdc159a5d0 0x55cdc15a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 >> 172.21.15.138:0/4123543940 conn(0x55cdc15983b0 msgr2=0x55cdc15990e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f966d600640 0 client.15612 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 shutdown_connections 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 1 -- 172.21.15.138:0/4123543940 wait complete. 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.716+0000 7f968d6b6740 20 client.15612 trim_cache size 0 max 16384 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:09:40.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:09:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:09:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 msgr2=0x7f96680b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 0x7f96680b6280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55cdc159d750 tx=0x7f967400ce60 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 msgr2=0x7f968419b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f967400e9f0 tx=0x7f967400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 shutdown_connections 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f96680b3dc0 0x7f96680b6280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f968410af90 0x7f96841a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f968410a610 0x7f968419b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 --2- 172.21.15.138:0/3460225315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9684104c10 0x7f968419b140 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 >> 172.21.15.138:0/3460225315 conn(0x7f9684100610 msgr2=0x7f9684102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:40.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 shutdown_connections 2024-08-20T01:09:40.728 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:40.717+0000 7f968a800640 1 -- 172.21.15.138:0/3460225315 wait complete. 2024-08-20T01:09:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:41 smithi167 ceph-mon[21965]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:41 smithi142 ceph-mon[21799]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:41 smithi138 ceph-mon[18076]: pgmap v409: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:42.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:42 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:09:42.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:42 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:09:43.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:42 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:09:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:43 smithi167 ceph-mon[21965]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:09:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:09:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:09:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:43 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:09:43.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:43 smithi142 ceph-mon[21799]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:09:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:09:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:09:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:43 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:09:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:43 smithi138 ceph-mon[18076]: pgmap v410: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-08-20T01:09:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:09:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:09:44.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:43 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:09:44.351 INFO:tasks.workunit.client.0.smithi138.stderr:Updating files: 64% (7798/12088) Updating files: 65% (7858/12088) Updating files: 66% (7979/12088) Updating files: 67% (8099/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-08-20T01:09:44.968 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-08-20T01:09:44.969 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:09:45.352 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:Note: switching to 'ef4275410d8483698016fe484b2a3c664080b4c1'. 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:state without impacting any branches by switching back to a branch. 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr:do so (now or later) by using -c with the switch command. Example: 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.547 INFO:tasks.workunit.client.0.smithi138.stderr: git switch -c 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr:Or undo this operation with: 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr: git switch - 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr: 2024-08-20T01:09:45.548 INFO:tasks.workunit.client.0.smithi138.stderr:HEAD is now at ef4275410d8 Merge branch 'mds-conf-sorted' of https://github.com/batrick/ceph into wip-hemanth-testing-2024-08-19-1357 2024-08-20T01:09:45.561 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-08-20T01:09:45.608 INFO:tasks.workunit.client.0.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-08-20T01:09:45.613 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T01:09:45.613 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-08-20T01:09:45.769 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:45 smithi138 ceph-mon[18076]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T01:09:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:45 smithi167 ceph-mon[21965]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T01:09:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:45 smithi142 ceph-mon[21799]: pgmap v411: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 -- 172.21.15.138:0/99182323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c102920 msgr2=0x7f911c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 --2- 172.21.15.138:0/99182323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c102920 0x7f911c102d00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f910c009900 tx=0x7f910c02a410 comp rx=0 tx=0).stop 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 -- 172.21.15.138:0/99182323 shutdown_connections 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 --2- 172.21.15.138:0/99182323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c10f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 --2- 172.21.15.138:0/99182323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c108330 0x7f911c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 --2- 172.21.15.138:0/99182323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c102920 0x7f911c102d00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 -- 172.21.15.138:0/99182323 >> 172.21.15.138:0/99182323 conn(0x7f911c0fe350 msgr2=0x7f911c100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 -- 172.21.15.138:0/99182323 shutdown_connections 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.269+0000 7f911b200640 1 -- 172.21.15.138:0/99182323 wait complete. 2024-08-20T01:09:46.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.270+0000 7f911b200640 1 Processor -- start 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.270+0000 7f911b200640 1 -- start start 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c108330 0x7f911c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f911c1129c0 con 0x7f911c108330 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f911c112840 con 0x7f911c102920 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f911c112b40 con 0x7f911c108cb0 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48956/0 (socket says 172.21.15.138:48956) 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 -- 172.21.15.138:0/1926634462 learned_addr learned my addr 172.21.15.138:0/1926634462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9111400640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c108330 0x7f911c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:46.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911a800640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 -- 172.21.15.138:0/1926634462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 msgr2=0x7f911c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 -- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c108330 msgr2=0x7f911c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c108330 0x7f911c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f9119e00640 1 -- 172.21.15.138:0/1926634462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f911c19ede0 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.271+0000 7f911a800640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f9119e00640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f910c005ab0 tx=0x7f910c003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c03f070 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910c031d60 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f911c19f070 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910c035030 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.272+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f911c0750b0 con 0x7f911c102920 2024-08-20T01:09:46.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.273+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f911c102d00 con 0x7f911c102920 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.275+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f910c042070 con 0x7f911c102920 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.277+0000 7f9113400640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 0x7f91000b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.277+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f910c105260 con 0x7f911c102920 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.277+0000 7f9111400640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 0x7f91000b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.278+0000 7f9111400640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 0x7f91000b62a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f911c101920 tx=0x7f9104005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:46.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.279+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f910c0cc1c0 con 0x7f911c102920 2024-08-20T01:09:46.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.500+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f911c075310 con 0x7f911c102920 2024-08-20T01:09:46.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.501+0000 7f9113400640 1 -- 172.21.15.138:0/1926634462 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f910c0cbac0 con 0x7f911c102920 2024-08-20T01:09:46.500 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:09:46.500 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":27,"btime":"2024-08-20T01:08:59:610307+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:59.217338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:09:46.501 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 27 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 msgr2=0x7f91000b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 0x7f91000b62a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f911c101920 tx=0x7f9104005f90 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 msgr2=0x7f911c198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f910c005ab0 tx=0x7f910c003c80 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 shutdown_connections 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f91000b3de0 0x7f91000b62a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f911c108cb0 0x7f911c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f911c108330 0x7f911c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 --2- 172.21.15.138:0/1926634462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f911c102920 0x7f911c198d30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 >> 172.21.15.138:0/1926634462 conn(0x7f911c0fe350 msgr2=0x7f911c0ffad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:46.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 shutdown_connections 2024-08-20T01:09:46.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:46.503+0000 7f911b200640 1 -- 172.21.15.138:0/1926634462 wait complete. 2024-08-20T01:09:46.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:46 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1926634462' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:46 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1926634462' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:46 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1926634462' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:47.521 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-08-20T01:09:47.659 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-08-20T01:09:47.726 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-08-20T01:09:47.727 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-08-20T01:09:47.731 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-08-20T01:09:47.734 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T01:09:47.735 INFO:tasks.workunit.client.0.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-08-20T01:09:47.801 INFO:tasks.workunit.client.0.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-08-20T01:09:47.809 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:09:47.809 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-08-20T01:09:47.886 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-08-20T01:09:47.888 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-08-20T01:09:47.888 DEBUG:teuthology.orchestra.run.smithi138:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-08-20T01:09:47.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:47 smithi167 ceph-mon[21965]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:47.984 INFO:tasks.workunit.client.0.smithi138.stderr:+ dbench 1 2024-08-20T01:09:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:47 smithi142 ceph-mon[21799]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:47.994 INFO:tasks.workunit.client.0.smithi138.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-20T01:09:47.995 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-08-20T01:09:47.995 INFO:tasks.workunit.client.0.smithi138.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-20T01:09:48.004 INFO:tasks.workunit.client.0.smithi138.stdout:0 of 1 processes prepared for launch 0 sec 2024-08-20T01:09:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:47 smithi138 ceph-mon[18076]: pgmap v412: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 682 B/s wr, 7 op/s 2024-08-20T01:09:48.105 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:48.105 INFO:tasks.workunit.client.0.smithi138.stdout:1 of 1 processes prepared for launch 0 sec 2024-08-20T01:09:48.105 INFO:tasks.workunit.client.0.smithi138.stdout:releasing clients 2024-08-20T01:09:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.762+0000 7f86f6200640 1 -- 172.21.15.138:0/3561053601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f01084c0 msgr2=0x7f86f0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.762+0000 7f86f6200640 1 --2- 172.21.15.138:0/3561053601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f01084c0 0x7f86f0108900 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f86dc009900 tx=0x7f86dc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 -- 172.21.15.138:0/3561053601 shutdown_connections 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 --2- 172.21.15.138:0/3561053601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f0108e40 0x7f86f010f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 --2- 172.21.15.138:0/3561053601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f01084c0 0x7f86f0108900 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 --2- 172.21.15.138:0/3561053601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 -- 172.21.15.138:0/3561053601 >> 172.21.15.138:0/3561053601 conn(0x7f86f00fe500 msgr2=0x7f86f0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 -- 172.21.15.138:0/3561053601 shutdown_connections 2024-08-20T01:09:48.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.763+0000 7f86f6200640 1 -- 172.21.15.138:0/3561053601 wait complete. 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.764+0000 7f86f6200640 1 Processor -- start 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.764+0000 7f86f6200640 1 -- start start 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f01084c0 0x7f86f01994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112ac0 con 0x7f86f01084c0 2024-08-20T01:09:48.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112940 con 0x7f86f0108e40 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86f6200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86f0112c40 con 0x7f86f0102ab0 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:46902/0 (socket says 172.21.15.138:46902) 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 -- 172.21.15.138:0/2584085266 learned_addr learned my addr 172.21.15.138:0/2584085266 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86ef400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:53160/0 (socket says 172.21.15.138:53160) 2024-08-20T01:09:48.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86eea00640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f01084c0 0x7f86f01994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 -- 172.21.15.138:0/2584085266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 msgr2=0x7f86f0198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 -- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f01084c0 msgr2=0x7f86f01994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f01084c0 0x7f86f01994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.765+0000 7f86efe00640 1 -- 172.21.15.138:0/2584085266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f019f040 con 0x7f86f0108e40 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86efe00640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86e800efd0 tx=0x7f86e800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86e7e00640 1 -- 172.21.15.138:0/2584085266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8003b70 con 0x7f86f0108e40 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f019f330 con 0x7f86f0108e40 2024-08-20T01:09:48.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86e7e00640 1 -- 172.21.15.138:0/2584085266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e8003d10 con 0x7f86f0108e40 2024-08-20T01:09:48.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86e7e00640 1 -- 172.21.15.138:0/2584085266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80108a0 con 0x7f86f0108e40 2024-08-20T01:09:48.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.766+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01a59f0 con 0x7f86f0108e40 2024-08-20T01:09:48.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.769+0000 7f86f8dee740 1 Processor -- start 2024-08-20T01:09:48.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86e7e00640 1 -- 172.21.15.138:0/2584085266 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f86e8004020 con 0x7f86f0108e40 2024-08-20T01:09:48.768 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 -- start start 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a375760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a378090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55789a309670 con 0x55789a369d10 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55789a2dd7c0 con 0x55789a375ca0 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86f8dee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55789a2cccc0 con 0x55789a3693f0 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86eea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86eea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:53162/0 (socket says 172.21.15.138:53162) 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.770+0000 7f86eea00640 1 -- 172.21.15.138:0/1697816198 learned_addr learned my addr 172.21.15.138:0/1697816198 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a378090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.769 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86ef400640 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a375760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 -- 172.21.15.138:0/1697816198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 msgr2=0x55789a378090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a378090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 -- 172.21.15.138:0/1697816198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 msgr2=0x55789a375760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a375760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 -- 172.21.15.138:0/1697816198 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789a260110 con 0x55789a3693f0 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.771+0000 7f86eea00640 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02a910 tx=0x7f86dc005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a38d5b949d03b8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86e7e00640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 0x7f86d40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86dbe00640 1 -- 172.21.15.138:0/1697816198 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc035030 con 0x55789a3693f0 2024-08-20T01:09:48.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86dbe00640 1 -- 172.21.15.138:0/1697816198 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc004030 con 0x55789a3693f0 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86dbe00640 1 -- 172.21.15.138:0/1697816198 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc003a10 con 0x55789a3693f0 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86e7e00640 1 -- 172.21.15.138:0/2584085266 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f86e80e4820 con 0x7f86f0108e40 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86ef400640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 0x7f86d40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86f8dee740 1 -- 172.21.15.138:0/1697816198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 msgr2=0x55789a3697d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.772+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f86dc02a910 tx=0x7f86dc005b50 comp rx=0 tx=0).stop 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 -- 172.21.15.138:0/1697816198 shutdown_connections 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a378090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a375760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1697816198 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3697d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 -- 172.21.15.138:0/1697816198 >> 172.21.15.138:0/1697816198 conn(0x55789a367220 msgr2=0x55789a3683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 -- 172.21.15.138:0/1697816198 shutdown_connections 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86ef400640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 0x7f86d40b6200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55789a368100 tx=0x7f86e0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 -- 172.21.15.138:0/1697816198 wait complete. 2024-08-20T01:09:48.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.773+0000 7f86f8dee740 1 -- start start 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 Processor -- start 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 0x55789a3731c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 0x55789a373700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55789a272780 con 0x55789a375ca0 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55789a2d0d00 con 0x55789a3693f0 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55789a12f7a0 con 0x55789a369d10 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86f8dee740 10 client.0 _ll_register_callbacks cb 0x55789a27a940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 0x55789a373700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:46914/0 (socket says 172.21.15.138:46914) 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86ef400640 1 -- 172.21.15.138:0/1732000727 learned_addr learned my addr 172.21.15.138:0/1732000727 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:48.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.774+0000 7f86eea00640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 0x55789a3731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 -- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 msgr2=0x55789a372c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 -- 172.21.15.138:0/1732000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 msgr2=0x55789a373700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 0x55789a373700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 -- 172.21.15.138:0/1732000727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789a27f9c0 con 0x55789a369d10 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86ef400640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86efe00640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 0x55789a373700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86eea00640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 0x55789a3731c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86dc031440 tx=0x7f86dc032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa638e4ad29b55a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86dc04f060 con 0x55789a369d10 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86dc004030 con 0x55789a369d10 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.775+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f86dc0043e0 con 0x55789a369d10 2024-08-20T01:09:48.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86dc056050 con 0x55789a369d10 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 10 client.25639 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55789a03bb90 con 0x55789a369d10 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86dc004030 con 0x55789a369d10 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86dc003790 con 0x55789a369d10 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 10 client.25639 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 10 client.25639 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 20 client.25639 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a385570 0x55789a387960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 4 client.25639 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.776+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55789a2df710 con 0x55789a385570 2024-08-20T01:09:48.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.777+0000 7f86ef400640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a385570 0x55789a387960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.777+0000 7f86ef400640 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a385570 0x55789a387960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f969be1234d83ad2 server_cookie=6ea951caa1cdfdca in_seq=0 out_seq=0 2024-08-20T01:09:48.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.777+0000 7f86d8c00640 10 client.25639 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:48.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86d8c00640 1 -- 172.21.15.138:0/1732000727 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55789a2df710 con 0x55789a385570 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86d8c00640 10 client.25639 handle_command_reply: tid=0 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 client.25639 shutdown 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 msgr2=0x55789a3731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 0x55789a3731c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f86dc031440 tx=0x7f86dc032da0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 shutdown_connections 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a385570 0x55789a387960 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a375ca0 0x55789a373700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a369d10 0x55789a3731c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1732000727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a372c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86d8c00640 0 client.25639 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 >> 172.21.15.138:0/1732000727 conn(0x55789a367220 msgr2=0x55789a36cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 shutdown_connections 2024-08-20T01:09:48.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 1 -- 172.21.15.138:0/1732000727 wait complete. 2024-08-20T01:09:48.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.778+0000 7f86f8dee740 20 client.25639 trim_cache size 0 max 16384 2024-08-20T01:09:48.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 Processor -- start 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 -- start start 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a38aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a375ca0 0x55789a36fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01b2210 con 0x55789a369d10 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01afae0 con 0x55789a3693f0 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86f8dee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ad930 con 0x55789a375ca0 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a38aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:46930/0 (socket says 172.21.15.138:46930) 2024-08-20T01:09:48.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.799+0000 7f86ef400640 1 -- 172.21.15.138:0/3888692526 learned_addr learned my addr 172.21.15.138:0/3888692526 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:48.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86efe00640 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a375ca0 0x55789a36fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 -- 172.21.15.138:0/3888692526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a375ca0 msgr2=0x55789a36fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a375ca0 0x55789a36fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 -- 172.21.15.138:0/3888692526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 msgr2=0x55789a38aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a38aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 -- 172.21.15.138:0/3888692526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55789a369b40 con 0x55789a3693f0 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86eea00640 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a38aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:48.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86ef400640 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005b90 tx=0x7f86e001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a96f970f153ce3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.800+0000 7f86db400640 1 -- 172.21.15.138:0/3888692526 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0005020 con 0x55789a3693f0 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86db400640 1 -- 172.21.15.138:0/3888692526 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e00047a0 con 0x55789a3693f0 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86db400640 1 -- 172.21.15.138:0/3888692526 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e002b6d0 con 0x55789a3693f0 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 -- 172.21.15.138:0/3888692526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 msgr2=0x55789a38a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f86e0005b90 tx=0x7f86e001b000 comp rx=0 tx=0).stop 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 -- 172.21.15.138:0/3888692526 shutdown_connections 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a375ca0 0x55789a36fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a38aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 --2- 172.21.15.138:0/3888692526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a3693f0 0x55789a38a980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 -- 172.21.15.138:0/3888692526 >> 172.21.15.138:0/3888692526 conn(0x55789a367220 msgr2=0x55789a3812c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.801+0000 7f86f8dee740 1 -- 172.21.15.138:0/3888692526 shutdown_connections 2024-08-20T01:09:48.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 -- 172.21.15.138:0/3888692526 wait complete. 2024-08-20T01:09:48.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 -- start start 2024-08-20T01:09:48.802 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 Processor -- start 2024-08-20T01:09:48.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a388880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01adfa0 con 0x55789a369d10 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01af410 con 0x55789a375ca0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f86f01ae670 con 0x55789a3693f0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.802+0000 7f86f8dee740 10 client.0 _ll_register_callbacks cb 0x7f86dc040040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86eea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a388880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:46942/0 (socket says 172.21.15.138:46942) 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 -- 172.21.15.138:0/1533279882 learned_addr learned my addr 172.21.15.138:0/1533279882 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86ef400640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 -- 172.21.15.138:0/1533279882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 msgr2=0x55789a3882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 -- 172.21.15.138:0/1533279882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 msgr2=0x55789a388880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:48.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a388880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86efe00640 1 -- 172.21.15.138:0/1533279882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55789a3efdb0 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.803+0000 7f86ef400640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.804+0000 7f86efe00640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009c30 tx=0x7f86e8004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8f6375df597a51 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.804+0000 7f86d8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.804+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e80af680 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.804+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f86e80b3eb0 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.804+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86e00047a0 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f86e80b4130 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 10 client.25676 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55789a3f8300 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86e00047a0 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f86e801f020 con 0x55789a375ca0 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 10 client.25676 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:48.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 10 client.25676 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 20 client.25676 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.805+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a3f8ba0 0x55789a3fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.806+0000 7f86f8dee740 4 client.25676 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.806+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x55789a2db1b0 con 0x55789a3f8ba0 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.806+0000 7f86ef400640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a3f8ba0 0x55789a3fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:48.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.806+0000 7f86ef400640 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a3f8ba0 0x55789a3fafc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d5da08706362ddb server_cookie=3da4db9fae0b1ec8 in_seq=0 out_seq=0 2024-08-20T01:09:48.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.807+0000 7f86d8c00640 10 client.25676 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:48.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86d8c00640 1 -- 172.21.15.138:0/1533279882 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55789a2db1b0 con 0x55789a3f8ba0 2024-08-20T01:09:48.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86d8c00640 10 client.25676 handle_command_reply: tid=0 2024-08-20T01:09:48.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 client.25676 shutdown 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 msgr2=0x55789a3ef670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f86e8009c30 tx=0x7f86e8004790 comp rx=0 tx=0).stop 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 shutdown_connections 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55789a3f8ba0 0x55789a3fafc0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55789a375ca0 0x55789a3ef670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55789a369d10 0x55789a388880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 --2- 172.21.15.138:0/1533279882 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55789a3693f0 0x55789a3882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86d8c00640 0 client.25676 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.919+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 >> 172.21.15.138:0/1533279882 conn(0x55789a367220 msgr2=0x55789a36c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.920+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 shutdown_connections 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.920+0000 7f86f8dee740 1 -- 172.21.15.138:0/1533279882 wait complete. 2024-08-20T01:09:48.919 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.920+0000 7f86f8dee740 20 client.25676 trim_cache size 0 max 16384 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 6, 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "322f488e": { 2024-08-20T01:09:48.920 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 6, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.0, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:09:48.921 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:48.922 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 msgr2=0x7f86d40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 0x7f86d40b6200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55789a368100 tx=0x7f86e0005f90 comp rx=0 tx=0).stop 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 msgr2=0x7f86f019e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f86e800efd0 tx=0x7f86e800c6a0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 shutdown_connections 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f86d40b3d40 0x7f86d40b6200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f0108e40 0x7f86f019e870 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86f01084c0 0x7f86f01994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 --2- 172.21.15.138:0/2584085266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f86f0102ab0 0x7f86f0198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:48.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 >> 172.21.15.138:0/2584085266 conn(0x7f86f00fe500 msgr2=0x7f86f01008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:48.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 shutdown_connections 2024-08-20T01:09:48.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:48.921+0000 7f86f6200640 1 -- 172.21.15.138:0/2584085266 wait complete. 2024-08-20T01:09:49.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 27.90 MB/sec warmup 1 sec latency 184.158 ms 2024-08-20T01:09:49.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:49 smithi138 ceph-mon[18076]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T01:09:49.860 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '322f488e', quiesce took 4.9 seconds. Will release after: 5.1 2024-08-20T01:09:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:49 smithi167 ceph-mon[21965]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T01:09:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:49 smithi142 ceph-mon[21799]: pgmap v413: 129 pgs: 129 active+clean; 1.4 MiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 682 B/s wr, 5 op/s 2024-08-20T01:09:50.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 13.95 MB/sec warmup 2 sec latency 1184.284 ms 2024-08-20T01:09:51.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 9.30 MB/sec warmup 3 sec latency 2184.397 ms 2024-08-20T01:09:51.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:51 smithi167 ceph-mon[21965]: pgmap v414: 129 pgs: 129 active+clean; 1.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-20T01:09:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:51 smithi142 ceph-mon[21799]: pgmap v414: 129 pgs: 129 active+clean; 1.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-20T01:09:52.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:51 smithi138 ceph-mon[18076]: pgmap v414: 129 pgs: 129 active+clean; 1.6 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 13 KiB/s wr, 7 op/s 2024-08-20T01:09:52.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 6.98 MB/sec warmup 4 sec latency 3184.562 ms 2024-08-20T01:09:53.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 5.58 MB/sec warmup 5 sec latency 4184.687 ms 2024-08-20T01:09:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 -- 172.21.15.138:0/2827237140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff490104c00 msgr2=0x7ff490104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 --2- 172.21.15.138:0/2827237140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff490104c00 0x7ff490104fe0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff480009900 tx=0x7ff48002a410 comp rx=0 tx=0).stop 2024-08-20T01:09:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 -- 172.21.15.138:0/2827237140 shutdown_connections 2024-08-20T01:09:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 --2- 172.21.15.138:0/2827237140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff490111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 --2- 172.21.15.138:0/2827237140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff49010a600 0x7ff49010aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 --2- 172.21.15.138:0/2827237140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff490104c00 0x7ff490104fe0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.832+0000 7ff496c00640 1 -- 172.21.15.138:0/2827237140 >> 172.21.15.138:0/2827237140 conn(0x7ff490100640 msgr2=0x7ff490102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.833+0000 7ff496c00640 1 -- 172.21.15.138:0/2827237140 shutdown_connections 2024-08-20T01:09:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.833+0000 7ff496c00640 1 -- 172.21.15.138:0/2827237140 wait complete. 2024-08-20T01:09:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.833+0000 7ff496c00640 1 Processor -- start 2024-08-20T01:09:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 -- start start 2024-08-20T01:09:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490104c00 0x7ff49019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 0x7ff49019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff490114c90 con 0x7ff49010af80 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff490114b10 con 0x7ff49010a600 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff496c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff490114e10 con 0x7ff490104c00 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff494c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff494c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33878/0 (socket says 172.21.15.138:33878) 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff48f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 0x7ff49019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff494c00640 1 -- 172.21.15.138:0/1674620040 learned_addr learned my addr 172.21.15.138:0/1674620040 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.834+0000 7ff48fe00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490104c00 0x7ff49019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 -- 172.21.15.138:0/1674620040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490104c00 msgr2=0x7ff49019b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490104c00 0x7ff49019b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 -- 172.21.15.138:0/1674620040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 msgr2=0x7ff49019b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 0x7ff49019b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 -- 172.21.15.138:0/1674620040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a11b0 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff48f400640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 0x7ff49019b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff494c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff48800eb10 tx=0x7ff48800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff48cc00640 1 -- 172.21.15.138:0/1674620040 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488004b20 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff48cc00640 1 -- 172.21.15.138:0/1674620040 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff488004cc0 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a14a0 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff48cc00640 1 -- 172.21.15.138:0/1674620040 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488010670 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.835+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a7b60 con 0x7ff49010af80 2024-08-20T01:09:53.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.838+0000 7ff499bcc740 1 Processor -- start 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff48cc00640 1 -- 172.21.15.138:0/1674620040 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff488005020 con 0x7ff49010af80 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 -- start start 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564216451ef0 0x56421645d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x564216460270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642163c7210 con 0x564216451ef0 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5642163b6710 con 0x5642164515d0 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff499bcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5642163ba750 con 0x56421645de80 2024-08-20T01:09:53.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff48f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff48f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:46976/0 (socket says 172.21.15.138:46976) 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff48f400640 1 -- 172.21.15.138:0/1237224972 learned_addr learned my addr 172.21.15.138:0/1237224972 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.839+0000 7ff48fe00640 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564216451ef0 0x56421645d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x564216460270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 -- 172.21.15.138:0/1237224972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 msgr2=0x564216460270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x564216460270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 -- 172.21.15.138:0/1237224972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564216451ef0 msgr2=0x56421645d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564216451ef0 0x56421645d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 -- 172.21.15.138:0/1237224972 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564216382de0 con 0x5642164515d0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48f400640 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff47c013990 tx=0x7ff47c009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fda758c1f4e5972 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff485200640 1 -- 172.21.15.138:0/1237224972 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c004030 con 0x5642164515d0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff485200640 1 -- 172.21.15.138:0/1237224972 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff47c00f940 con 0x5642164515d0 2024-08-20T01:09:53.847 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff485200640 1 -- 172.21.15.138:0/1237224972 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c003b00 con 0x5642164515d0 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48cc00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 0x7ff4740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.840+0000 7ff48cc00640 1 -- 172.21.15.138:0/1674620040 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff4880e4980 con 0x7ff49010af80 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff48fe00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 0x7ff4740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 -- 172.21.15.138:0/1237224972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 msgr2=0x5642164519b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff47c013990 tx=0x7ff47c009c50 comp rx=0 tx=0).stop 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 -- 172.21.15.138:0/1237224972 shutdown_connections 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x564216460270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564216451ef0 0x56421645d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 --2- 172.21.15.138:0/1237224972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642164515d0 0x5642164519b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 -- 172.21.15.138:0/1237224972 >> 172.21.15.138:0/1237224972 conn(0x56421644f3b0 msgr2=0x5642164505a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 -- 172.21.15.138:0/1237224972 shutdown_connections 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff499bcc740 1 -- 172.21.15.138:0/1237224972 wait complete. 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.841+0000 7ff48fe00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 0x7ff4740b6200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff480005ab0 tx=0x7ff480005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 -- start start 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 Processor -- start 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x56421645adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x56421645b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5642162efab0 con 0x5642164515d0 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56421635d140 con 0x564216451ef0 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564216460b60 con 0x56421645de80 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.842+0000 7ff499bcc740 10 client.0 _ll_register_callbacks cb 0x5642163705d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33908/0 (socket says 172.21.15.138:33908) 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 -- 172.21.15.138:0/181656335 learned_addr learned my addr 172.21.15.138:0/181656335 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48f400640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x56421645adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff494c00640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x56421645b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 -- 172.21.15.138:0/181656335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 msgr2=0x56421645b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x56421645b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 -- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 msgr2=0x56421645adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x56421645adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 -- 172.21.15.138:0/181656335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421635e170 con 0x5642164515d0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff48fe00640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff480033000 tx=0x7ff480009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5892240462e019c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff479600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48004e070 con 0x5642164515d0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.843+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff480004970 con 0x5642164515d0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56421638da30 con 0x5642164515d0 2024-08-20T01:09:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff48005a050 con 0x5642164515d0 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 10 client.15648 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564216119790 con 0x5642164515d0 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56421638da30 con 0x5642164515d0 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff4800324b0 con 0x5642164515d0 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 10 client.15648 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 20 client.15648 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421646dd40 0x5642164701d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 4 client.15648 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.844+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642163c9160 con 0x56421646dd40 2024-08-20T01:09:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.845+0000 7ff48f400640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421646dd40 0x5642164701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.845+0000 7ff48f400640 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421646dd40 0x5642164701d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9bf77500b7ca5c server_cookie=52f9711fa0ff2d50 in_seq=0 out_seq=0 2024-08-20T01:09:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.845+0000 7ff479600640 10 client.15648 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff479600640 1 -- 172.21.15.138:0/181656335 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5642163c9160 con 0x56421646dd40 2024-08-20T01:09:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff479600640 10 client.15648 handle_command_reply: tid=0 2024-08-20T01:09:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 client.15648 shutdown 2024-08-20T01:09:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 msgr2=0x56421645d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff480033000 tx=0x7ff480009260 comp rx=0 tx=0).stop 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 shutdown_connections 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421646dd40 0x5642164701d0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421645de80 0x56421645b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x56421645adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 --2- 172.21.15.138:0/181656335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x56421645d940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff479600640 0 client.15648 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.846+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 >> 172.21.15.138:0/181656335 conn(0x56421644f3b0 msgr2=0x564216454f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.847+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 shutdown_connections 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.847+0000 7ff499bcc740 1 -- 172.21.15.138:0/181656335 wait complete. 2024-08-20T01:09:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.847+0000 7ff499bcc740 20 client.15648 trim_cache size 0 max 16384 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.861+0000 7ff499bcc740 1 Processor -- start 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.861+0000 7ff499bcc740 1 -- start start 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x564216450450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x564216450990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216475ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b44a0 con 0x5642164515d0 2024-08-20T01:09:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b1d70 con 0x564216451ef0 2024-08-20T01:09:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff499bcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901afbc0 con 0x56421644f3b0 2024-08-20T01:09:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x564216450990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x564216450450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x564216450990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33918/0 (socket says 172.21.15.138:33918) 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48f400640 1 -- 172.21.15.138:0/3728023006 learned_addr learned my addr 172.21.15.138:0/3728023006 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff494c00640 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216475ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 -- 172.21.15.138:0/3728023006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 msgr2=0x564216475ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216475ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 -- 172.21.15.138:0/3728023006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 msgr2=0x564216450990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x564216450990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff48fe00640 1 -- 172.21.15.138:0/3728023006 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564216192620 con 0x56421644f3b0 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.862+0000 7ff494c00640 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216475ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.863+0000 7ff48fe00640 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x564216450450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff480032900 tx=0x7ff48002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f301b7221f5f56e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.863+0000 7ff47be00640 1 -- 172.21.15.138:0/3728023006 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48004e070 con 0x56421644f3b0 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.863+0000 7ff47be00640 1 -- 172.21.15.138:0/3728023006 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480005850 con 0x56421644f3b0 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.863+0000 7ff47be00640 1 -- 172.21.15.138:0/3728023006 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480035040 con 0x56421644f3b0 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- 172.21.15.138:0/3728023006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 msgr2=0x564216450450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x564216450450 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff480032900 tx=0x7ff48002a410 comp rx=0 tx=0).stop 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- 172.21.15.138:0/3728023006 shutdown_connections 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216475ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x564216450990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 --2- 172.21.15.138:0/3728023006 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x564216450450 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- 172.21.15.138:0/3728023006 >> 172.21.15.138:0/3728023006 conn(0x56421645daa0 msgr2=0x5642164595e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- 172.21.15.138:0/3728023006 shutdown_connections 2024-08-20T01:09:53.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- 172.21.15.138:0/3728023006 wait complete. 2024-08-20T01:09:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 -- start start 2024-08-20T01:09:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.864+0000 7ff499bcc740 1 Processor -- start 2024-08-20T01:09:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x5642164d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.869 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216465b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0230 con 0x5642164515d0 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b16a0 con 0x564216451ef0 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901b0900 con 0x56421644f3b0 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff499bcc740 10 client.0 _ll_register_callbacks cb 0x7ff480009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff494c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216465b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33922/0 (socket says 172.21.15.138:33922) 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 -- 172.21.15.138:0/713630986 learned_addr learned my addr 172.21.15.138:0/713630986 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48fe00640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x5642164d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 -- 172.21.15.138:0/713630986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 msgr2=0x5642164d9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x5642164d9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 -- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 msgr2=0x564216465b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216465b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff48f400640 1 -- 172.21.15.138:0/713630986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564216466270 con 0x5642164515d0 2024-08-20T01:09:53.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.865+0000 7ff494c00640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216465b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff48f400640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009b70 tx=0x7ff47c003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95834a3266edaf01 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff479600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c0054e0 con 0x5642164515d0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff47c00f840 con 0x5642164515d0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff480005850 con 0x5642164515d0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff47c03b050 con 0x5642164515d0 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff499bcc740 10 client.15654 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:53.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5642164e0e30 con 0x5642164515d0 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.866+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff480005850 con 0x5642164515d0 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff47c01e070 con 0x5642164515d0 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 10 client.15654 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 10 client.15654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 20 client.15654 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421645de80 0x5642164e3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 4 client.15654 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642163c4c00 con 0x56421645de80 2024-08-20T01:09:53.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff48fe00640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421645de80 0x5642164e3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff479600640 10 client.15654 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.867+0000 7ff48fe00640 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421645de80 0x5642164e3440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c496fc8396dfa2a7 server_cookie=5b180ce7a38fd71a in_seq=0 out_seq=0 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.872+0000 7ff479600640 1 -- 172.21.15.138:0/713630986 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5642163c4c00 con 0x56421645de80 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.872+0000 7ff479600640 10 client.15654 handle_command_reply: tid=0 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.872+0000 7ff499bcc740 1 client.15654 shutdown 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 msgr2=0x5642164d99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff47c009b70 tx=0x7ff47c003700 comp rx=0 tx=0).stop 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 shutdown_connections 2024-08-20T01:09:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56421645de80 0x5642164e3440 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564216451ef0 0x564216465b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5642164515d0 0x5642164d99f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 --2- 172.21.15.138:0/713630986 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56421644f3b0 0x5642164d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 >> 172.21.15.138:0/713630986 conn(0x56421645daa0 msgr2=0x564216459470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff479600640 0 client.15654 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 shutdown_connections 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 1 -- 172.21.15.138:0/713630986 wait complete. 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.873+0000 7ff499bcc740 20 client.15654 trim_cache size 0 max 16384 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:09:53.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:09:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:09:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 msgr2=0x7ff4740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 0x7ff4740b6200 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff480005ab0 tx=0x7ff480005a20 comp rx=0 tx=0).stop 2024-08-20T01:09:53.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 msgr2=0x7ff4901a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff48800eb10 tx=0x7ff48800c2d0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 shutdown_connections 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4740b3d40 0x7ff4740b6200 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49010af80 0x7ff4901a09e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff49010a600 0x7ff49019b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 --2- 172.21.15.138:0/1674620040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff490104c00 0x7ff49019b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 >> 172.21.15.138:0/1674620040 conn(0x7ff490100640 msgr2=0x7ff490101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 shutdown_connections 2024-08-20T01:09:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:09:53.874+0000 7ff496c00640 1 -- 172.21.15.138:0/1674620040 wait complete. 2024-08-20T01:09:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:53 smithi167 ceph-mon[21965]: pgmap v415: 129 pgs: 129 active+clean; 2.6 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 119 KiB/s wr, 25 op/s 2024-08-20T01:09:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:53 smithi142 ceph-mon[21799]: pgmap v415: 129 pgs: 129 active+clean; 2.6 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 119 KiB/s wr, 25 op/s 2024-08-20T01:09:54.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:53 smithi138 ceph-mon[18076]: pgmap v415: 129 pgs: 129 active+clean; 2.6 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 119 KiB/s wr, 25 op/s 2024-08-20T01:09:54.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 4.65 MB/sec warmup 6 sec latency 5184.760 ms 2024-08-20T01:09:54.960 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:09:55.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 3.99 MB/sec warmup 7 sec latency 6184.825 ms 2024-08-20T01:09:55.332 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:55.728 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:55 smithi138 ceph-mon[18076]: pgmap v416: 129 pgs: 129 active+clean; 5.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 352 KiB/s wr, 25 op/s 2024-08-20T01:09:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:55 smithi167 ceph-mon[21965]: pgmap v416: 129 pgs: 129 active+clean; 5.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 352 KiB/s wr, 25 op/s 2024-08-20T01:09:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:55 smithi142 ceph-mon[21799]: pgmap v416: 129 pgs: 129 active+clean; 5.3 MiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 352 KiB/s wr, 25 op/s 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 -- 172.21.15.138:0/4046724956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c10b030 msgr2=0x7f075c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 --2- 172.21.15.138:0/4046724956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c10b030 0x7f075c111830 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0754009be0 tx=0x7f075402a6c0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 -- 172.21.15.138:0/4046724956 shutdown_connections 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 --2- 172.21.15.138:0/4046724956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c10b030 0x7f075c111830 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 --2- 172.21.15.138:0/4046724956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10a6b0 0x7f075c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 --2- 172.21.15.138:0/4046724956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c104ca0 0x7f075c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 -- 172.21.15.138:0/4046724956 >> 172.21.15.138:0/4046724956 conn(0x7f075c1006f0 msgr2=0x7f075c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 -- 172.21.15.138:0/4046724956 shutdown_connections 2024-08-20T01:09:55.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.962+0000 7f0764200640 1 -- 172.21.15.138:0/4046724956 wait complete. 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.963+0000 7f0764200640 1 Processor -- start 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 -- start start 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 0x7f075c19b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10b030 0x7f075c1a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f075c114cb0 con 0x7f075c104ca0 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f075c114b30 con 0x7f075c10a6b0 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0764200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f075c114e30 con 0x7f075c10b030 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0760e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0761800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 0x7f075c19b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.964+0000 7f0760e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47010/0 (socket says 172.21.15.138:47010) 2024-08-20T01:09:55.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 -- 172.21.15.138:0/1105014413 learned_addr learned my addr 172.21.15.138:0/1105014413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0762200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10b030 0x7f075c1a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 -- 172.21.15.138:0/1105014413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10b030 msgr2=0x7f075c1a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10b030 0x7f075c1a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 -- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 msgr2=0x7f075c19b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 0x7f075c19b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 -- 172.21.15.138:0/1105014413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075c1a11b0 con 0x7f075c10a6b0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0761800640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 0x7f075c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0760e00640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9f0 tx=0x7f074800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f074800ccc0 con 0x7f075c10a6b0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07480054d0 con 0x7f075c10a6b0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0748010430 con 0x7f075c10a6b0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.965+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075c1a14a0 con 0x7f075c10a6b0 2024-08-20T01:09:55.995 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.966+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f075c1a7b60 con 0x7f075c10a6b0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.967+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f075c105080 con 0x7f075c10a6b0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.969+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f07480046c0 con 0x7f075c10a6b0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.970+0000 7f075a000640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 0x7f07440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.970+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f07480e4850 con 0x7f075c10a6b0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.971+0000 7f0761800640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 0x7f07440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.971+0000 7f0761800640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 0x7f07440b6200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0750009fc0 tx=0x7f0750005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:55.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:55.972+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0748055e20 con 0x7f075c10a6b0 2024-08-20T01:09:56.105 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 3.49 MB/sec warmup 8 sec latency 7184.891 ms 2024-08-20T01:09:56.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.192+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f075c19c230 con 0x7f075c10a6b0 2024-08-20T01:09:56.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.192+0000 7f075a000640 1 -- 172.21.15.138:0/1105014413 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11935 (secure 0 0 0) 0x7f07480ab7b0 con 0x7f075c10a6b0 2024-08-20T01:09:56.191 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:09:56.191 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":27,"btime":"2024-08-20T01:08:59:610307+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:08:59.217338+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:09:56.191 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 27 2024-08-20T01:09:56.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 msgr2=0x7f07440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 0x7f07440b6200 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0750009fc0 tx=0x7f0750005f90 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 msgr2=0x7f075c19b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f074800e9f0 tx=0x7f074800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 shutdown_connections 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f07440b3d40 0x7f07440b6200 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f075c10b030 0x7f075c1a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f075c10a6b0 0x7f075c19b640 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 --2- 172.21.15.138:0/1105014413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f075c104ca0 0x7f075c19b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 >> 172.21.15.138:0/1105014413 conn(0x7f075c1006f0 msgr2=0x7f075c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 shutdown_connections 2024-08-20T01:09:56.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:56.196+0000 7f0764200640 1 -- 172.21.15.138:0/1105014413 wait complete. 2024-08-20T01:09:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:56 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1105014413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:56.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:56 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1105014413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:57.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:56 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1105014413' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:09:57.106 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 322f488e --release --await' 2024-08-20T01:09:57.106 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 322f488e --release --await 2024-08-20T01:09:57.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 3.10 MB/sec warmup 9 sec latency 8184.965 ms 2024-08-20T01:09:57.478 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:09:57.955 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:57 smithi138 ceph-mon[18076]: pgmap v417: 129 pgs: 129 active+clean; 17 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 32 op/s 2024-08-20T01:09:57.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:57 smithi167 ceph-mon[21965]: pgmap v417: 129 pgs: 129 active+clean; 17 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 32 op/s 2024-08-20T01:09:57.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:57 smithi142 ceph-mon[21799]: pgmap v417: 129 pgs: 129 active+clean; 17 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.3 MiB/s wr, 32 op/s 2024-08-20T01:09:58.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 -- 172.21.15.138:0/677803856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc1090e0 msgr2=0x7ff4fc1094c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 --2- 172.21.15.138:0/677803856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc1094c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec009900 tx=0x7ff4ec02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 -- 172.21.15.138:0/677803856 shutdown_connections 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 --2- 172.21.15.138:0/677803856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 0x7ff4fc115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 --2- 172.21.15.138:0/677803856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc10ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 --2- 172.21.15.138:0/677803856 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc1094c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 -- 172.21.15.138:0/677803856 >> 172.21.15.138:0/677803856 conn(0x7ff4fc078760 msgr2=0x7ff4fc07ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 -- 172.21.15.138:0/677803856 shutdown_connections 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.075+0000 7ff502800640 1 -- 172.21.15.138:0/677803856 wait complete. 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 Processor -- start 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 -- start start 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 0x7ff4fc1a4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1190f0 con 0x7ff4fc1090e0 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc118f70 con 0x7ff4fc10f470 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff502800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc119270 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff4fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.076+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4fb400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc19f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44640/0 (socket says 172.21.15.138:44640) 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43772/0 (socket says 172.21.15.138:43772) 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4fb400640 1 -- 172.21.15.138:0/3192675251 learned_addr learned my addr 172.21.15.138:0/3192675251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 0x7ff4fc1a4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 -- 172.21.15.138:0/3192675251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 msgr2=0x7ff4fc1a4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 0x7ff4fc1a4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 -- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 msgr2=0x7ff4fc19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 -- 172.21.15.138:0/3192675251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4fc1a56c0 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e9f0 tx=0x7ff4e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4f3e00640 1 -- 172.21.15.138:0/3192675251 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e800cd80 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4fc1a59b0 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4f3e00640 1 -- 172.21.15.138:0/3192675251 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80035f0 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.077+0000 7ff4f3e00640 1 -- 172.21.15.138:0/3192675251 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8010690 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.078+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4fc1ac070 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.081+0000 7ff5053ba740 1 Processor -- start 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.081+0000 7ff5053ba740 1 -- start start 2024-08-20T01:09:58.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf56060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf58990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e29ceeb670 con 0x55e29cf49cf0 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e29cebfd70 con 0x55e29cf4a610 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff5053ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e29ceaf270 con 0x55e29cf565a0 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44652/0 (socket says 172.21.15.138:44652) 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 -- 172.21.15.138:0/3377304678 learned_addr learned my addr 172.21.15.138:0/3377304678 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fb400640 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf56060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 -- 172.21.15.138:0/3377304678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 msgr2=0x55e29cf58990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf58990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 -- 172.21.15.138:0/3377304678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 msgr2=0x55e29cf56060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.101 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf56060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.082+0000 7ff4fbe00640 1 -- 172.21.15.138:0/3377304678 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29ce4b870 con 0x55e29cf49cf0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4fb400640 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf56060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4f3e00640 1 -- 172.21.15.138:0/3192675251 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff4e80037a0 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009f50 tx=0x7ff4f40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbf0eb639c1c0b79 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4f0c00640 1 -- 172.21.15.138:0/3377304678 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4004d20 con 0x55e29cf49cf0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4f0c00640 1 -- 172.21.15.138:0/3377304678 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f4004050 con 0x55e29cf49cf0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.083+0000 7ff4f0c00640 1 -- 172.21.15.138:0/3377304678 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f4005280 con 0x55e29cf49cf0 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- 172.21.15.138:0/3377304678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 msgr2=0x55e29cf4a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009f50 tx=0x7ff4f40118d0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- 172.21.15.138:0/3377304678 shutdown_connections 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf58990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf56060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3377304678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf49cf0 0x55e29cf4a0d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- 172.21.15.138:0/3377304678 >> 172.21.15.138:0/3377304678 conn(0x55e29cf47b20 msgr2=0x55e29cf48cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- 172.21.15.138:0/3377304678 shutdown_connections 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- 172.21.15.138:0/3377304678 wait complete. 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff4f3e00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 0x7ff4e00b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff5053ba740 1 -- start start 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.084+0000 7ff4f3e00640 1 -- 172.21.15.138:0/3192675251 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff4e80e46d0 con 0x7ff4fc10eaf0 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 Processor -- start 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 0x7ff4e00b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf565a0 0x55e29cf53870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e29ce35ff0 con 0x55e29cf565a0 2024-08-20T01:09:58.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e29ceb32b0 con 0x55e29cf4a610 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e29cd11bb0 con 0x55e29cf49cf0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff5053ba740 10 client.0 _ll_register_callbacks cb 0x55e29cbbad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 0x7ff4e00b6110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009250 tx=0x7ff4f4004710 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf565a0 0x55e29cf53870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.085+0000 7ff4fb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43782/0 (socket says 172.21.15.138:43782) 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 -- 172.21.15.138:0/3945681259 learned_addr learned my addr 172.21.15.138:0/3945681259 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 -- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 msgr2=0x55e29cf532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 -- 172.21.15.138:0/3945681259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf565a0 msgr2=0x55e29cf53870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf565a0 0x55e29cf53870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 -- 172.21.15.138:0/3945681259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29cc0eb00 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4fb400640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02a9e0 tx=0x7ff4ec004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=852c422c24aa3a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4e5600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec035040 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.086+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4f4004050 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff4ec003b40 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec04e050 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff5053ba740 10 client.25678 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e29cc85960 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4f4004050 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff4ec03c440 con 0x55e29cf49cf0 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff5053ba740 10 client.25678 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:58.105 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff5053ba740 10 client.25678 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.087+0000 7ff5053ba740 20 client.25678 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cf65cc0 0x55e29cf680e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff5053ba740 4 client.25678 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29cec1cc0 con 0x55e29cf65cc0 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cf65cc0 0x55e29cf680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff4faa00640 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cf65cc0 0x55e29cf680e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbc195aaf5954d8 server_cookie=e02aeb6842e1fc23 in_seq=0 out_seq=0 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.088+0000 7ff4e5600640 10 client.25678 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff4e5600640 1 -- 172.21.15.138:0/3945681259 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e29cec1cc0 con 0x55e29cf65cc0 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff4e5600640 10 client.25678 handle_command_reply: tid=0 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 client.25678 shutdown 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 msgr2=0x55e29cf55e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec02a9e0 tx=0x7ff4ec004480 comp rx=0 tx=0).stop 2024-08-20T01:09:58.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 shutdown_connections 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cf65cc0 0x55e29cf680e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf565a0 0x55e29cf53870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf4a610 0x55e29cf532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff4e5600640 0 client.25678 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 --2- 172.21.15.138:0/3945681259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf55e60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 >> 172.21.15.138:0/3945681259 conn(0x55e29cf47b20 msgr2=0x55e29cf4d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.089+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 shutdown_connections 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.090+0000 7ff5053ba740 1 -- 172.21.15.138:0/3945681259 wait complete. 2024-08-20T01:09:58.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.090+0000 7ff5053ba740 20 client.25678 trim_cache size 0 max 16384 2024-08-20T01:09:58.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.107+0000 7ff5053ba740 1 Processor -- start 2024-08-20T01:09:58.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 2368 2.79 MB/sec warmup 10 sec latency 9185.010 ms 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 -- start start 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 0x55e29cf621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf6d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b89c0 con 0x55e29cf4a610 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b6290 con 0x55e29cf49cf0 2024-08-20T01:09:58.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff5053ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b40e0 con 0x55e29cf565a0 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44672/0 (socket says 172.21.15.138:44672) 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff4faa00640 1 -- 172.21.15.138:0/4215892785 learned_addr learned my addr 172.21.15.138:0/4215892785 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf6d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.108+0000 7ff4fb400640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 0x55e29cf621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 -- 172.21.15.138:0/4215892785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 msgr2=0x55e29cf6d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf6d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 -- 172.21.15.138:0/4215892785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 msgr2=0x55e29cf621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 0x55e29cf621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 -- 172.21.15.138:0/4215892785 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29cf485a0 con 0x55e29cf4a610 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4fbe00640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf6d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:58.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4fb400640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 0x55e29cf621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4faa00640 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80a81a0 tx=0x7ff4e80aadb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c47c45496363bc22 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4e7e00640 1 -- 172.21.15.138:0/4215892785 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e80c4030 con 0x55e29cf4a610 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4e7e00640 1 -- 172.21.15.138:0/4215892785 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80aaf30 con 0x55e29cf4a610 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.109+0000 7ff4e7e00640 1 -- 172.21.15.138:0/4215892785 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e808be80 con 0x55e29cf4a610 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- 172.21.15.138:0/4215892785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 msgr2=0x55e29cf4d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80a81a0 tx=0x7ff4e80aadb0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- 172.21.15.138:0/4215892785 shutdown_connections 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf565a0 0x55e29cf6d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf4d4f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 --2- 172.21.15.138:0/4215892785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf49cf0 0x55e29cf621c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- 172.21.15.138:0/4215892785 >> 172.21.15.138:0/4215892785 conn(0x55e29cf47b20 msgr2=0x55e29cf4cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- 172.21.15.138:0/4215892785 shutdown_connections 2024-08-20T01:09:58.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- 172.21.15.138:0/4215892785 wait complete. 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.110+0000 7ff5053ba740 1 -- start start 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 Processor -- start 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf5a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf5ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf565a0 0x55e29cfcf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b4750 con 0x55e29cf4a610 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b5bc0 con 0x55e29cf565a0 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4fc1b4e20 con 0x55e29cf49cf0 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff5053ba740 10 client.0 _ll_register_callbacks cb 0x7ff4ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf5ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff4faa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf5ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44676/0 (socket says 172.21.15.138:44676) 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff4fbe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf565a0 0x55e29cfcf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff4faa00640 1 -- 172.21.15.138:0/225052784 learned_addr learned my addr 172.21.15.138:0/225052784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.111+0000 7ff4fb400640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf5a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 -- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf565a0 msgr2=0x55e29cfcf980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf565a0 0x55e29cfcf980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 -- 172.21.15.138:0/225052784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 msgr2=0x55e29cf5ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf5ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 -- 172.21.15.138:0/225052784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29cfcff70 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4fb400640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf5a950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005ab0 tx=0x7ff4ec000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65f9563791330363 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4e5600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.112+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec057060 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4e80aaf30 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff4ec003b40 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff4ec05e050 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 10 client.25684 fetch_fsmap learned FSMap version 27 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e29cfd12b0 con 0x55e29cf49cf0 2024-08-20T01:09:58.116 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4e80aaf30 con 0x55e29cf49cf0 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7ff4ec031950 con 0x55e29cf49cf0 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 10 client.25684 fetch_fsmap finished waiting for FSMap version 27 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 10 client.25684 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 20 client.25684 populate_metadata read hostname 'smithi138' 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cfd1a80 0x55e29cfd3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 4 client.25684 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "322f488e", "release": true, "await": true}] 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.113+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "322f488e", "release": true, "await": true}) -- 0x55e29cebd760 con 0x55e29cfd1a80 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.114+0000 7ff4faa00640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cfd1a80 0x55e29cfd3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.114+0000 7ff4faa00640 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cfd1a80 0x55e29cfd3ea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f445c2424f1b55 server_cookie=1f377e766bc1936d in_seq=0 out_seq=0 2024-08-20T01:09:58.117 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.114+0000 7ff4e5600640 10 client.25684 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff4e5600640 1 -- 172.21.15.138:0/225052784 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55e29cebd760 con 0x55e29cfd1a80 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff4e5600640 10 client.25684 handle_command_reply: tid=0 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 client.25684 shutdown 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 msgr2=0x55e29cf5a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf5a950 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005ab0 tx=0x7ff4ec000bc0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 shutdown_connections 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e29cfd1a80 0x55e29cfd3ea0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e29cf565a0 0x55e29cfcf980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e29cf4a610 0x55e29cf5ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 --2- 172.21.15.138:0/225052784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e29cf49cf0 0x55e29cf5a950 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 >> 172.21.15.138:0/225052784 conn(0x55e29cf47b20 msgr2=0x55e29cf4d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff4e5600640 0 client.25684 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.125+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 shutdown_connections 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff5053ba740 1 -- 172.21.15.138:0/225052784 wait complete. 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff5053ba740 20 client.25684 trim_cache size 0 max 16384 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 27, 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 8, 2024-08-20T01:09:58.125 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "322f488e": { 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 8, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.0, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:09:58.126 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:09:58.127 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 msgr2=0x7ff4e00b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 0x7ff4e00b6110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4009250 tx=0x7ff4f4004710 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 msgr2=0x7ff4fc19fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff4e800e9f0 tx=0x7ff4e800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 shutdown_connections 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4e00b3c50 0x7ff4e00b6110 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4fc10f470 0x7ff4fc1a4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff4fc10eaf0 0x7ff4fc19fb50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 --2- 172.21.15.138:0/3192675251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4fc1090e0 0x7ff4fc19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.126+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 >> 172.21.15.138:0/3192675251 conn(0x7ff4fc078760 msgr2=0x7ff4fc079070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.127+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 shutdown_connections 2024-08-20T01:09:58.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:09:58.127+0000 7ff502800640 1 -- 172.21.15.138:0/3192675251 wait complete. 2024-08-20T01:09:58.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:09:58 smithi138 ceph-mon[18076]: pgmap v418: 129 pgs: 129 active+clean; 20 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2024-08-20T01:09:58.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:09:58 smithi167 ceph-mon[21965]: pgmap v418: 129 pgs: 129 active+clean; 20 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2024-08-20T01:09:58.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:09:58 smithi142 ceph-mon[21799]: pgmap v418: 129 pgs: 129 active+clean; 20 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 1.5 MiB/s wr, 30 op/s 2024-08-20T01:09:59.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 4159 3.64 MB/sec warmup 11 sec latency 9195.197 ms 2024-08-20T01:09:59.277 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '322f488e', total seconds elapsed: 14.3 2024-08-20T01:10:00.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 7418 4.91 MB/sec warmup 12 sec latency 13.797 ms 2024-08-20T01:10:00.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:00 smithi167 ceph-mon[21965]: overall HEALTH_OK 2024-08-20T01:10:00.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:00 smithi142 ceph-mon[21799]: overall HEALTH_OK 2024-08-20T01:10:00.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:00 smithi138 ceph-mon[18076]: overall HEALTH_OK 2024-08-20T01:10:01.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 10341 5.60 MB/sec warmup 13 sec latency 51.900 ms 2024-08-20T01:10:01.462 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:01 smithi167 ceph-mon[21965]: pgmap v419: 129 pgs: 129 active+clean; 21 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-08-20T01:10:01.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:01 smithi142 ceph-mon[21799]: pgmap v419: 129 pgs: 129 active+clean; 21 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-08-20T01:10:01.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:01 smithi138 ceph-mon[18076]: pgmap v419: 129 pgs: 129 active+clean; 21 MiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 1.7 MiB/s wr, 35 op/s 2024-08-20T01:10:02.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 11244 5.59 MB/sec warmup 14 sec latency 60.184 ms 2024-08-20T01:10:03.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 14431 6.47 MB/sec warmup 15 sec latency 13.259 ms 2024-08-20T01:10:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:03 smithi167 ceph-mon[21965]: pgmap v420: 129 pgs: 129 active+clean; 25 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.1 MiB/s wr, 73 op/s 2024-08-20T01:10:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:03 smithi142 ceph-mon[21799]: pgmap v420: 129 pgs: 129 active+clean; 25 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.1 MiB/s wr, 73 op/s 2024-08-20T01:10:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:03 smithi138 ceph-mon[18076]: pgmap v420: 129 pgs: 129 active+clean; 25 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.1 MiB/s wr, 73 op/s 2024-08-20T01:10:04.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 17216 6.88 MB/sec warmup 16 sec latency 16.084 ms 2024-08-20T01:10:05.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 20047 7.20 MB/sec warmup 17 sec latency 22.461 ms 2024-08-20T01:10:05.893 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:05 smithi138 ceph-mon[18076]: pgmap v421: 129 pgs: 129 active+clean; 28 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-20T01:10:05.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:05 smithi167 ceph-mon[21965]: pgmap v421: 129 pgs: 129 active+clean; 28 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-20T01:10:05.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:05 smithi142 ceph-mon[21799]: pgmap v421: 129 pgs: 129 active+clean; 28 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 2.3 MiB/s wr, 106 op/s 2024-08-20T01:10:06.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 22512 7.65 MB/sec warmup 18 sec latency 13.869 ms 2024-08-20T01:10:06.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 -- 172.21.15.138:0/4103804470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 msgr2=0x7f292810f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 --2- 172.21.15.138:0/4103804470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f292810f3b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2920009ca0 tx=0x7f292002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 -- 172.21.15.138:0/4103804470 shutdown_connections 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 --2- 172.21.15.138:0/4103804470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f292810f3b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 --2- 172.21.15.138:0/4103804470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f2928105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 --2- 172.21.15.138:0/4103804470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 0x7f292806ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 -- 172.21.15.138:0/4103804470 >> 172.21.15.138:0/4103804470 conn(0x7f2928077270 msgr2=0x7f2928077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 -- 172.21.15.138:0/4103804470 shutdown_connections 2024-08-20T01:10:06.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.995+0000 7f292cc00640 1 -- 172.21.15.138:0/4103804470 wait complete. 2024-08-20T01:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 Processor -- start 2024-08-20T01:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 -- start start 2024-08-20T01:10:06.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 0x7f292819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f292819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2928112800 con 0x7f2928104fd0 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.996+0000 7f292cc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2928112680 con 0x7f2928105970 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f292cc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2928112980 con 0x7f292806a7a0 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2927e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f292819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2927e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39624/0 (socket says 172.21.15.138:39624) 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2927e00640 1 -- 172.21.15.138:0/1481940701 learned_addr learned my addr 172.21.15.138:0/1481940701 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2927400640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 0x7f292819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 -- 172.21.15.138:0/1481940701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 msgr2=0x7f292819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 0x7f292819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.997 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 -- 172.21.15.138:0/1481940701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 msgr2=0x7f29281a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2926a00640 1 -- 172.21.15.138:0/1481940701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29281a0ff0 con 0x7f2928104fd0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.997+0000 7f2927e00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f2926a00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f292819b490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9f0 tx=0x7f291400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f291fe00640 1 -- 172.21.15.138:0/1481940701 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291400cd80 con 0x7f2928104fd0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29281a12e0 con 0x7f2928104fd0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f291fe00640 1 -- 172.21.15.138:0/1481940701 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29140035f0 con 0x7f2928104fd0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f291fe00640 1 -- 172.21.15.138:0/1481940701 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914010690 con 0x7f2928104fd0 2024-08-20T01:10:06.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:06.998+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29281a7b80 con 0x7f2928104fd0 2024-08-20T01:10:06.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.001+0000 7f2931033740 1 Processor -- start 2024-08-20T01:10:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 -- start start 2024-08-20T01:10:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f291fe00640 1 -- 172.21.15.138:0/1481940701 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f29140037a0 con 0x7f2928104fd0 2024-08-20T01:10:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433c07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564843327210 con 0x5648433b1b10 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564843316710 con 0x5648433b2430 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2931033740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56484331a750 con 0x5648433be3c0 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39640/0 (socket says 172.21.15.138:39640) 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2927400640 1 -- 172.21.15.138:0/757594703 learned_addr learned my addr 172.21.15.138:0/757594703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2927e00640 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.002+0000 7f2926a00640 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433c07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 -- 172.21.15.138:0/757594703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 msgr2=0x5648433c07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433c07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 -- 172.21.15.138:0/757594703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 msgr2=0x5648433b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 -- 172.21.15.138:0/757594703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648432e2de0 con 0x5648433b2430 2024-08-20T01:10:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2927400640 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d20 tx=0x7f2918011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df5f41164ffdeb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2913e00640 1 -- 172.21.15.138:0/757594703 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291801e070 con 0x5648433b2430 2024-08-20T01:10:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2913e00640 1 -- 172.21.15.138:0/757594703 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2918019ae0 con 0x5648433b2430 2024-08-20T01:10:07.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.003+0000 7f2913e00640 1 -- 172.21.15.138:0/757594703 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800eac0 con 0x5648433b2430 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f291fe00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 0x7f290c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 -- 172.21.15.138:0/757594703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 msgr2=0x5648433bde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d20 tx=0x7f2918011810 comp rx=0 tx=0).stop 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f291fe00640 1 -- 172.21.15.138:0/1481940701 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f29140e4560 con 0x7f2928104fd0 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2927e00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 0x7f290c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 -- 172.21.15.138:0/757594703 shutdown_connections 2024-08-20T01:10:07.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433c07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b2430 0x5648433bde80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 --2- 172.21.15.138:0/757594703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 -- 172.21.15.138:0/757594703 >> 172.21.15.138:0/757594703 conn(0x5648433af940 msgr2=0x5648433b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 -- 172.21.15.138:0/757594703 shutdown_connections 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.004+0000 7f2931033740 1 -- 172.21.15.138:0/757594703 wait complete. 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 -- start start 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2927e00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 0x7f290c0b6280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2920005e30 tx=0x7f2920005c50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 Processor -- start 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433baf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56484324fab0 con 0x5648433b2430 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5648432bd140 con 0x5648433b1b10 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5648433c10a0 con 0x5648433be3c0 2024-08-20T01:10:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.005+0000 7f2931033740 10 client.0 _ll_register_callbacks cb 0x5648432d05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39642/0 (socket says 172.21.15.138:39642) 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 -- 172.21.15.138:0/734239618 learned_addr learned my addr 172.21.15.138:0/734239618 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927e00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2926a00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433baf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 -- 172.21.15.138:0/734239618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 msgr2=0x5648433bb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433bb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 -- 172.21.15.138:0/734239618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 msgr2=0x5648433baf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433baf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 -- 172.21.15.138:0/734239618 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648432be170 con 0x5648433b1b10 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2926a00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927e00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2927400640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2918005890 tx=0x7f291800e440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d93f9f785b0aa81 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2910c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.006+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918026020 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2918019ae0 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2918019840 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2918031050 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 10 client.25730 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564843079790 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2918019ae0 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f291800ec90 con 0x5648433b1b10 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 10 client.25730 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 20 client.25730 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5648433cde70 0x5648433d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 4 client.25730 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.007+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564843329160 con 0x5648433cde70 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.008+0000 7f2926a00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5648433cde70 0x5648433d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.008+0000 7f2910c00640 10 client.25730 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.008+0000 7f2926a00640 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5648433cde70 0x5648433d0260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30eed3fe5bb195c server_cookie=9da99131d852fbb8 in_seq=0 out_seq=0 2024-08-20T01:10:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2910c00640 1 -- 172.21.15.138:0/734239618 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564843329160 con 0x5648433cde70 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2910c00640 10 client.25730 handle_command_reply: tid=0 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 client.25730 shutdown 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 msgr2=0x5648433bdb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2918005890 tx=0x7f291800e440 comp rx=0 tx=0).stop 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 shutdown_connections 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5648433cde70 0x5648433d0260 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433bb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433baf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2910c00640 0 client.25730 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 --2- 172.21.15.138:0/734239618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433bdb50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 >> 172.21.15.138:0/734239618 conn(0x5648433af940 msgr2=0x5648433b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 shutdown_connections 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 1 -- 172.21.15.138:0/734239618 wait complete. 2024-08-20T01:10:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.009+0000 7f2931033740 20 client.25730 trim_cache size 0 max 16384 2024-08-20T01:10:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 Processor -- start 2024-08-20T01:10:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 -- start start 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433d5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433b5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29281b43e0 con 0x5648433b2430 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29281b1c80 con 0x5648433b1b10 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2931033740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29281afad0 con 0x5648433be3c0 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2927400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39658/0 (socket says 172.21.15.138:39658) 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2927400640 1 -- 172.21.15.138:0/701798139 learned_addr learned my addr 172.21.15.138:0/701798139 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2926a00640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433d5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.025+0000 7f2927e00640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433b5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 -- 172.21.15.138:0/701798139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 msgr2=0x5648433b5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433b5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 -- 172.21.15.138:0/701798139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 msgr2=0x5648433d5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433d5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 -- 172.21.15.138:0/701798139 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648430f2620 con 0x5648433b1b10 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2926a00640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2927400640 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d20 tx=0x7f2918009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c79200ab6771c1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2913400640 1 -- 172.21.15.138:0/701798139 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918026070 con 0x5648433b1b10 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2913400640 1 -- 172.21.15.138:0/701798139 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291800eab0 con 0x5648433b1b10 2024-08-20T01:10:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.026+0000 7f2913400640 1 -- 172.21.15.138:0/701798139 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291801e040 con 0x5648433b1b10 2024-08-20T01:10:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- 172.21.15.138:0/701798139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 msgr2=0x5648433d5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d20 tx=0x7f2918009780 comp rx=0 tx=0).stop 2024-08-20T01:10:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- 172.21.15.138:0/701798139 shutdown_connections 2024-08-20T01:10:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433be3c0 0x5648433b5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b2430 0x5648433d5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 --2- 172.21.15.138:0/701798139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433b1b10 0x5648433d5a70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- 172.21.15.138:0/701798139 >> 172.21.15.138:0/701798139 conn(0x5648433af940 msgr2=0x5648433b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- 172.21.15.138:0/701798139 shutdown_connections 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- 172.21.15.138:0/701798139 wait complete. 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.027+0000 7f2931033740 1 -- start start 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 Processor -- start 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 0x5648433c1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433be3c0 0x564843436a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f29281b0140 con 0x5648433b1b10 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f29281b15b0 con 0x5648433be3c0 2024-08-20T01:10:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f29281b0810 con 0x5648433b2430 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2931033740 10 client.0 _ll_register_callbacks cb 0x7f291801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2927400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2926a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 0x5648433c1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2927400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44738/0 (socket says 172.21.15.138:44738) 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2927400640 1 -- 172.21.15.138:0/2913051278 learned_addr learned my addr 172.21.15.138:0/2913051278 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.028+0000 7f2927e00640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433be3c0 0x564843436a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 -- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433be3c0 msgr2=0x564843436a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433be3c0 0x564843436a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 -- 172.21.15.138:0/2913051278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 msgr2=0x5648433c1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 -- 172.21.15.138:0/2913051278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564843436f70 con 0x5648433b2430 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2927400640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2926a00640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 0x5648433c1a90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29140e1960 tx=0x7f2914003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0821fc13be046be server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2910c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:10:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29140af420 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f291800eab0 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f29140a70b0 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f29140b3ed0 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2931033740 10 client.25702 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.029+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564843438140 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f291800eab0 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f29140e1be0 con 0x5648433b2430 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 10 client.25702 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 10 client.25702 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 20 client.25702 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564843438880 0x56484343aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 4 client.25702 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:10:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564843324c00 con 0x564843438880 2024-08-20T01:10:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.030+0000 7f2927400640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564843438880 0x56484343aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.031+0000 7f2927400640 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564843438880 0x56484343aca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66124ec45f9ac199 server_cookie=e6abf2902ad78c5f in_seq=0 out_seq=0 2024-08-20T01:10:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.031+0000 7f2910c00640 10 client.25702 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2910c00640 1 -- 172.21.15.138:0/2913051278 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564843324c00 con 0x564843438880 2024-08-20T01:10:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2910c00640 10 client.25702 handle_command_reply: tid=0 2024-08-20T01:10:07.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 client.25702 shutdown 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 msgr2=0x5648433c1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 0x5648433c1a90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f29140e1960 tx=0x7f2914003cf0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 shutdown_connections 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564843438880 0x56484343aca0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5648433be3c0 0x564843436a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5648433b2430 0x5648433c1a90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 --2- 172.21.15.138:0/2913051278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5648433b1b10 0x5648433c1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 >> 172.21.15.138:0/2913051278 conn(0x5648433af940 msgr2=0x5648433bc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2910c00640 0 client.25702 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 shutdown_connections 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 1 -- 172.21.15.138:0/2913051278 wait complete. 2024-08-20T01:10:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.037+0000 7f2931033740 20 client.25702 trim_cache size 0 max 16384 2024-08-20T01:10:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:10:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:10:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:10:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:10:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 msgr2=0x7f290c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 0x7f290c0b6280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2920005e30 tx=0x7f2920005c50 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 msgr2=0x7f292819b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f292819b490 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f291400e9f0 tx=0x7f291400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 shutdown_connections 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f290c0b3dc0 0x7f290c0b6280 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2928105970 0x7f29281a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2928104fd0 0x7f292819b490 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 --2- 172.21.15.138:0/1481940701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f292806a7a0 0x7f292819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 >> 172.21.15.138:0/1481940701 conn(0x7f2928077270 msgr2=0x7f2928106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 shutdown_connections 2024-08-20T01:10:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:07.038+0000 7f292cc00640 1 -- 172.21.15.138:0/1481940701 wait complete. 2024-08-20T01:10:07.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25409 8.15 MB/sec warmup 19 sec latency 10.145 ms 2024-08-20T01:10:07.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:07 smithi167 ceph-mon[21965]: pgmap v422: 129 pgs: 129 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T01:10:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:07 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:07 smithi142 ceph-mon[21799]: pgmap v422: 129 pgs: 129 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T01:10:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:07 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:08.031 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:07 smithi138 ceph-mon[18076]: pgmap v422: 129 pgs: 129 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 577 KiB/s rd, 2.4 MiB/s wr, 149 op/s 2024-08-20T01:10:08.031 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:07 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:08.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 7.74 MB/sec warmup 20 sec latency 961.715 ms 2024-08-20T01:10:09.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 7.37 MB/sec warmup 21 sec latency 1961.780 ms 2024-08-20T01:10:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:09 smithi167 ceph-mon[21965]: pgmap v423: 129 pgs: 129 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-08-20T01:10:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:09 smithi142 ceph-mon[21799]: pgmap v423: 129 pgs: 129 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-08-20T01:10:10.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:09 smithi138 ceph-mon[18076]: pgmap v423: 129 pgs: 129 active+clean; 31 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-08-20T01:10:10.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 7.04 MB/sec warmup 22 sec latency 2961.919 ms 2024-08-20T01:10:11.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 6.73 MB/sec warmup 23 sec latency 3961.990 ms 2024-08-20T01:10:11.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:11 smithi167 ceph-mon[21965]: pgmap v424: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-08-20T01:10:11.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:11 smithi142 ceph-mon[21799]: pgmap v424: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-08-20T01:10:12.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:11 smithi138 ceph-mon[18076]: pgmap v424: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 985 KiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-08-20T01:10:12.106 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 6.45 MB/sec warmup 24 sec latency 4962.093 ms 2024-08-20T01:10:13.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 6.19 MB/sec warmup 25 sec latency 5962.204 ms 2024-08-20T01:10:13.929 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:13 smithi138 ceph-mon[18076]: pgmap v425: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-20T01:10:13.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:13 smithi167 ceph-mon[21965]: pgmap v425: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-20T01:10:13.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:13 smithi142 ceph-mon[21799]: pgmap v425: 129 pgs: 129 active+clean; 34 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-08-20T01:10:14.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 5.96 MB/sec warmup 26 sec latency 6962.278 ms 2024-08-20T01:10:15.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 5.74 MB/sec warmup 27 sec latency 7962.325 ms 2024-08-20T01:10:15.886 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:15 smithi138 ceph-mon[18076]: pgmap v426: 129 pgs: 129 active+clean; 38 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-08-20T01:10:15.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:15 smithi167 ceph-mon[21965]: pgmap v426: 129 pgs: 129 active+clean; 38 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-08-20T01:10:15.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:15 smithi142 ceph-mon[21799]: pgmap v426: 129 pgs: 129 active+clean; 38 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-08-20T01:10:16.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 5.53 MB/sec warmup 28 sec latency 8962.368 ms 2024-08-20T01:10:17.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 25470 5.34 MB/sec warmup 29 sec latency 9962.407 ms 2024-08-20T01:10:17.850 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10.2 2024-08-20T01:10:17.850 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:10:17.865 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:17 smithi138 ceph-mon[18076]: pgmap v427: 129 pgs: 129 active+clean; 48 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-08-20T01:10:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:17 smithi167 ceph-mon[21965]: pgmap v427: 129 pgs: 129 active+clean; 48 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-08-20T01:10:17.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:17 smithi142 ceph-mon[21799]: pgmap v427: 129 pgs: 129 active+clean; 48 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.9 MiB/s wr, 120 op/s 2024-08-20T01:10:18.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 26469 5.33 MB/sec warmup 30 sec latency 10507.892 ms 2024-08-20T01:10:18.265 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:18.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 -- 172.21.15.138:0/1949772249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80075210 msgr2=0x7fdf800755f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:18.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 --2- 172.21.15.138:0/1949772249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80075210 0x7fdf800755f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009900 tx=0x7fdf7002a410 comp rx=0 tx=0).stop 2024-08-20T01:10:18.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 -- 172.21.15.138:0/1949772249 shutdown_connections 2024-08-20T01:10:18.908 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 --2- 172.21.15.138:0/1949772249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf8010f0a0 0x7fdf80111490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 --2- 172.21.15.138:0/1949772249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075b30 0x7fdf8010eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 --2- 172.21.15.138:0/1949772249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf80075210 0x7fdf800755f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 -- 172.21.15.138:0/1949772249 >> 172.21.15.138:0/1949772249 conn(0x7fdf800fe500 msgr2=0x7fdf80100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 -- 172.21.15.138:0/1949772249 shutdown_connections 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.892+0000 7fdf87000640 1 -- 172.21.15.138:0/1949772249 wait complete. 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.893+0000 7fdf87000640 1 Processor -- start 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.893+0000 7fdf87000640 1 -- start start 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075210 0x7fdf8019b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf80075b30 0x7fdf8019b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:18.909 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80114cd0 con 0x7fdf80075210 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80114b50 con 0x7fdf8010f0a0 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf87000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80114e50 con 0x7fdf80075b30 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35720/0 (socket says 172.21.15.138:35720) 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf77400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf80075b30 0x7fdf8019b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 -- 172.21.15.138:0/1410841426 learned_addr learned my addr 172.21.15.138:0/1410841426 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf7fe00640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075210 0x7fdf8019b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 -- 172.21.15.138:0/1410841426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf80075b30 msgr2=0x7fdf8019b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf80075b30 0x7fdf8019b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 -- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075210 msgr2=0x7fdf8019b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075210 0x7fdf8019b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:18.910 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.894+0000 7fdf85000640 1 -- 172.21.15.138:0/1410841426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf801a1230 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf85000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9d0 tx=0x7fdf7800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800cd80 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf801a1520 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf78005020 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780116d0 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.895+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf801a7be0 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.896+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf800755f0 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.898+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fdf780051d0 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.900+0000 7fdf7d600640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 0x7fdf640b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.900+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fdf780e4ee0 con 0x7fdf8010f0a0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.900+0000 7fdf7fe00640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 0x7fdf640b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.901+0000 7fdf7fe00640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 0x7fdf640b6200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005ab0 tx=0x7fdf70005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:18.911 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:18.902+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdf78016030 con 0x7fdf8010f0a0 2024-08-20T01:10:19.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 29197 5.63 MB/sec warmup 31 sec latency 25.789 ms 2024-08-20T01:10:19.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.161+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf801a7f70 con 0x7fdf8010f0a0 2024-08-20T01:10:19.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.162+0000 7fdf7d600640 1 -- 172.21.15.138:0/1410841426 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7fdf780ab7f0 con 0x7fdf8010f0a0 2024-08-20T01:10:19.161 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:10:19.161 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":28,"btime":"2024-08-20T01:10:06:605638+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:10:06.139777+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:10:19.161 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 28 2024-08-20T01:10:19.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 msgr2=0x7fdf640b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 0x7fdf640b6200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdf70005ab0 tx=0x7fdf70005a20 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 msgr2=0x7fdf801a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800e9d0 tx=0x7fdf7800ee90 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 shutdown_connections 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdf640b3d40 0x7fdf640b6200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf8010f0a0 0x7fdf801a0a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdf80075b30 0x7fdf8019b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 --2- 172.21.15.138:0/1410841426 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf80075210 0x7fdf8019b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.164+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 >> 172.21.15.138:0/1410841426 conn(0x7fdf800fe500 msgr2=0x7fdf80111240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.165+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 shutdown_connections 2024-08-20T01:10:19.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:19.165+0000 7fdf87000640 1 -- 172.21.15.138:0/1410841426 wait complete. 2024-08-20T01:10:19.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:19 smithi167 ceph-mon[21965]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-08-20T01:10:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:19 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1410841426' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:19.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:19 smithi142 ceph-mon[21799]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-08-20T01:10:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:19 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1410841426' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:20.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:19 smithi138 ceph-mon[18076]: pgmap v428: 129 pgs: 129 active+clean; 52 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.9 MiB/s wr, 80 op/s 2024-08-20T01:10:20.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:19 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1410841426' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:20.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 32177 6.02 MB/sec warmup 32 sec latency 41.360 ms 2024-08-20T01:10:20.143 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 130.2 --await-for 45.0' 2024-08-20T01:10:20.144 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 130.2 --await-for 45.0 2024-08-20T01:10:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 -- 172.21.15.138:0/1084946858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010a410 msgr2=0x7f3e4010a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1084946858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010a410 0x7f3e4010a850 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 -- 172.21.15.138:0/1084946858 shutdown_connections 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1084946858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e4010ad90 0x7f3e40111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1084946858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010a410 0x7f3e4010a850 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1084946858 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e40104a10 0x7f3e40104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 -- 172.21.15.138:0/1084946858 >> 172.21.15.138:0/1084946858 conn(0x7f3e40100410 msgr2=0x7f3e40102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 -- 172.21.15.138:0/1084946858 shutdown_connections 2024-08-20T01:10:20.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.163+0000 7f3e3f200640 1 -- 172.21.15.138:0/1084946858 wait complete. 2024-08-20T01:10:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 Processor -- start 2024-08-20T01:10:20.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 -- start start 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4010a410 0x7f3e4019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40114aa0 con 0x7f3e40104a10 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40114920 con 0x7f3e4010ad90 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.164+0000 7f3e3f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40114c20 con 0x7f3e4010a410 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35744/0 (socket says 172.21.15.138:35744) 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 -- 172.21.15.138:0/1390550955 learned_addr learned my addr 172.21.15.138:0/1390550955 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3de00640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35478/0 (socket says 172.21.15.138:35478) 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3d400640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4010a410 0x7f3e4019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 -- 172.21.15.138:0/1390550955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4010a410 msgr2=0x7f3e4019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4010a410 0x7f3e4019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 -- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 msgr2=0x7f3e4019afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 -- 172.21.15.138:0/1390550955 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e401a1060 con 0x7f3e4010ad90 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3de00640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e3e800640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800eb10 tx=0x7f3e3800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.165+0000 7f3e36a00640 1 -- 172.21.15.138:0/1390550955 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38004b20 con 0x7f3e4010ad90 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.166+0000 7f3e36a00640 1 -- 172.21.15.138:0/1390550955 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e38004cc0 con 0x7f3e4010ad90 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.166+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e401a1350 con 0x7f3e4010ad90 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.166+0000 7f3e36a00640 1 -- 172.21.15.138:0/1390550955 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38010670 con 0x7f3e4010ad90 2024-08-20T01:10:20.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.166+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e401a7a10 con 0x7f3e4010ad90 2024-08-20T01:10:20.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.169+0000 7f3e36a00640 1 -- 172.21.15.138:0/1390550955 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3e38005020 con 0x7f3e4010ad90 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.169+0000 7f3e480c8740 1 Processor -- start 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.169+0000 7f3e480c8740 1 -- start start 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dc1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565314d28210 con 0x565314db30f0 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565314d17710 con 0x565314db27d0 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e480c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565314d1b750 con 0x565314dbf080 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35490/0 (socket says 172.21.15.138:35490) 2024-08-20T01:10:20.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 -- 172.21.15.138:0/1334880291 learned_addr learned my addr 172.21.15.138:0/1334880291 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3d400640 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dc1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 -- 172.21.15.138:0/1334880291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 msgr2=0x565314dc1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dc1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 -- 172.21.15.138:0/1334880291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 msgr2=0x565314db2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3de00640 1 -- 172.21.15.138:0/1334880291 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565314ce3de0 con 0x565314db30f0 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.170+0000 7f3e3d400640 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e3de00640 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3e300115c0 tx=0x7f3e30011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17877e3122c13c7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e36a00640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 0x7f3e240b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e36a00640 1 -- 172.21.15.138:0/1390550955 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3e380e4880 con 0x7f3e4010ad90 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e2b400640 1 -- 172.21.15.138:0/1334880291 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000f640 con 0x565314db30f0 2024-08-20T01:10:20.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e3d400640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 0x7f3e240b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e2b400640 1 -- 172.21.15.138:0/1334880291 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3000fc90 con 0x565314db30f0 2024-08-20T01:10:20.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.171+0000 7f3e2b400640 1 -- 172.21.15.138:0/1334880291 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30019ce0 con 0x565314db30f0 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 -- 172.21.15.138:0/1334880291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 msgr2=0x565314dbeb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3e300115c0 tx=0x7f3e30011a80 comp rx=0 tx=0).stop 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e3d400640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 0x7f3e240b6280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005ab0 tx=0x7f3e2c005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 -- 172.21.15.138:0/1334880291 shutdown_connections 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dc1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dbeb40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 --2- 172.21.15.138:0/1334880291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 -- 172.21.15.138:0/1334880291 >> 172.21.15.138:0/1334880291 conn(0x565314db0600 msgr2=0x565314db17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 -- 172.21.15.138:0/1334880291 shutdown_connections 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.172+0000 7f3e480c8740 1 -- 172.21.15.138:0/1334880291 wait complete. 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 -- start start 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 Processor -- start 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db27d0 0x565314dbc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314dbcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x565314c50ab0 con 0x565314db27d0 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x565314cbe140 con 0x565314dbf080 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x565314dc1d60 con 0x565314db30f0 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.173+0000 7f3e480c8740 10 client.0 _ll_register_callbacks cb 0x565314cd15d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46210/0 (socket says 172.21.15.138:46210) 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 -- 172.21.15.138:0/131417786 learned_addr learned my addr 172.21.15.138:0/131417786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314dbcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db27d0 0x565314dbc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 -- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 msgr2=0x565314dbcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314dbcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 -- 172.21.15.138:0/131417786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db27d0 msgr2=0x565314dbc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db27d0 0x565314dbc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 -- 172.21.15.138:0/131417786 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565314cbf170 con 0x565314db30f0 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e3d400640 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d63a05e79dce97a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.174+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c04e020 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3e2c0314a0 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565314ceea30 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e2c035040 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e480c8740 10 client.25720 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565314a7a790 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565314ceea30 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f3e2c04a540 con 0x565314db30f0 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.175+0000 7f3e480c8740 10 client.25720 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e480c8740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e480c8740 20 client.25720 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314dcf0d0 0x565314dd1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e480c8740 4 client.25720 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:20.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565314d2a160 con 0x565314dcf0d0 2024-08-20T01:10:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.176+0000 7f3e3de00640 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314dcf0d0 0x565314dd1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.177+0000 7f3e3de00640 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314dcf0d0 0x565314dd1540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4676ff49433406a server_cookie=b5e789f8908ad31c in_seq=0 out_seq=0 2024-08-20T01:10:20.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.177+0000 7f3e1fe00640 10 client.25720 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e1fe00640 1 -- 172.21.15.138:0/131417786 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565314d2a160 con 0x565314dcf0d0 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e1fe00640 10 client.25720 handle_command_reply: tid=0 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 client.25720 shutdown 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 msgr2=0x565314dbc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c009900 tx=0x7f3e2c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 shutdown_connections 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314dcf0d0 0x565314dd1540 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314dbcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db30f0 0x565314dbc610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 --2- 172.21.15.138:0/131417786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db27d0 0x565314dbc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e1fe00640 0 client.25720 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 >> 172.21.15.138:0/131417786 conn(0x565314db0600 msgr2=0x565314db61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.178+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 shutdown_connections 2024-08-20T01:10:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.179+0000 7f3e480c8740 1 -- 172.21.15.138:0/131417786 wait complete. 2024-08-20T01:10:20.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.179+0000 7f3e480c8740 20 client.25720 trim_cache size 0 max 16384 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 Processor -- start 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 -- start start 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dd6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dd73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b43a0 con 0x565314db30f0 2024-08-20T01:10:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b1c40 con 0x565314db27d0 2024-08-20T01:10:20.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e480c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401afa90 con 0x565314dbf080 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e3d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dd6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35768/0 (socket says 172.21.15.138:35768) 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.197+0000 7f3e3de00640 1 -- 172.21.15.138:0/4196641361 learned_addr learned my addr 172.21.15.138:0/4196641361 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3e800640 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dd73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 -- 172.21.15.138:0/4196641361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 msgr2=0x565314dd73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dd73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 -- 172.21.15.138:0/4196641361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 msgr2=0x565314dd6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dd6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 -- 172.21.15.138:0/4196641361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565314af3620 con 0x565314db27d0 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e3de00640 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009250 tx=0x7f3e30019880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=652d178bd216e0d9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e2aa00640 1 -- 172.21.15.138:0/4196641361 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e300199b0 con 0x565314db27d0 2024-08-20T01:10:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.198+0000 7f3e2aa00640 1 -- 172.21.15.138:0/4196641361 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e300050d0 con 0x565314db27d0 2024-08-20T01:10:20.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.199+0000 7f3e2aa00640 1 -- 172.21.15.138:0/4196641361 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000fb10 con 0x565314db27d0 2024-08-20T01:10:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.199+0000 7f3e480c8740 1 -- 172.21.15.138:0/4196641361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 msgr2=0x565314db6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.199+0000 7f3e480c8740 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3e30009250 tx=0x7f3e30019880 comp rx=0 tx=0).stop 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 -- 172.21.15.138:0/4196641361 shutdown_connections 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314dbf080 0x565314dd73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dd6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 --2- 172.21.15.138:0/4196641361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314db27d0 0x565314db6090 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 -- 172.21.15.138:0/4196641361 >> 172.21.15.138:0/4196641361 conn(0x565314db0600 msgr2=0x565314db5980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 -- 172.21.15.138:0/4196641361 shutdown_connections 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 -- 172.21.15.138:0/4196641361 wait complete. 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 -- start start 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.200+0000 7f3e480c8740 1 Processor -- start 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dc2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314e37e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b0100 con 0x565314db30f0 2024-08-20T01:10:20.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b1570 con 0x565314dbf080 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b07d0 con 0x565314db27d0 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e480c8740 10 client.0 _ll_register_callbacks cb 0x7f3e30005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e3de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46222/0 (socket says 172.21.15.138:46222) 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e3de00640 1 -- 172.21.15.138:0/3062677825 learned_addr learned my addr 172.21.15.138:0/3062677825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e3e800640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314e37e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.201+0000 7f3e3d400640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dc2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 -- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 msgr2=0x565314e37e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314e37e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 -- 172.21.15.138:0/3062677825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 msgr2=0x565314dc2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dc2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 -- 172.21.15.138:0/3062677825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565314e383a0 con 0x565314db27d0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3e800640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314e37e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e3de00640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3e30013010 tx=0x7f3e30005d30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc49b206c37dea6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30035070 con 0x565314db27d0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3e30005490 con 0x565314db27d0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e300050d0 con 0x565314db27d0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.202+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e300198f0 con 0x565314db27d0 2024-08-20T01:10:20.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 10 client.25726 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565314e39520 con 0x565314db27d0 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e300050d0 con 0x565314db27d0 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f3e3002d070 con 0x565314db27d0 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 10 client.25726 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 10 client.25726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 20 client.25726 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314e399f0 0x565314e3be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 4 client.25726 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565314d25c00 con 0x565314e399f0 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.203+0000 7f3e3d400640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314e399f0 0x565314e3be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.204+0000 7f3e1fe00640 10 client.25726 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:20.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.204+0000 7f3e3d400640 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314e399f0 0x565314e3be10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d32fea6a94a89a server_cookie=deebcb2a99233095 in_seq=0 out_seq=0 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e1fe00640 1 -- 172.21.15.138:0/3062677825 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x565314d25c00 con 0x565314e399f0 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e1fe00640 10 client.25726 handle_command_reply: tid=0 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 client.25726 shutdown 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 msgr2=0x565314dc2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3e30013010 tx=0x7f3e30005d30 comp rx=0 tx=0).stop 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 shutdown_connections 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x565314e399f0 0x565314e3be10 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565314dbf080 0x565314e37e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x565314db30f0 0x565314dc2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e1fe00640 0 client.25726 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 --2- 172.21.15.138:0/3062677825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x565314db27d0 0x565314dc2210 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 >> 172.21.15.138:0/3062677825 conn(0x565314db0600 msgr2=0x565314dbc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.236+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 shutdown_connections 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.237+0000 7f3e480c8740 1 -- 172.21.15.138:0/3062677825 wait complete. 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.237+0000 7f3e480c8740 20 client.25726 trim_cache size 0 max 16384 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:10:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 msgr2=0x7f3e240b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 0x7f3e240b6280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3e2c005ab0 tx=0x7f3e2c005fd0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 msgr2=0x7f3e401a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800eb10 tx=0x7f3e3800c2d0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 shutdown_connections 2024-08-20T01:10:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3e240b3dc0 0x7f3e240b6280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e4010ad90 0x7f3e401a0890 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3e4010a410 0x7f3e4019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 --2- 172.21.15.138:0/1390550955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3e40104a10 0x7f3e4019afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 >> 172.21.15.138:0/1390550955 conn(0x7f3e40100410 msgr2=0x7f3e40102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 shutdown_connections 2024-08-20T01:10:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:20.238+0000 7f3e3f200640 1 -- 172.21.15.138:0/1390550955 wait complete. 2024-08-20T01:10:20.592 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:21.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34394 6.07 MB/sec warmup 33 sec latency 24.252 ms 2024-08-20T01:10:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.232+0000 7f2f93e00640 1 -- 172.21.15.138:0/1442304553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9410a520 msgr2=0x7f2f9410a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.232+0000 7f2f93e00640 1 --2- 172.21.15.138:0/1442304553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9410a520 0x7f2f9410a960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2f80009900 tx=0x7f2f8002a410 comp rx=0 tx=0).stop 2024-08-20T01:10:21.232 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.232+0000 7f2f93e00640 1 -- 172.21.15.138:0/1442304553 shutdown_connections 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.232+0000 7f2f93e00640 1 --2- 172.21.15.138:0/1442304553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410aea0 0x7f2f941116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.232+0000 7f2f93e00640 1 --2- 172.21.15.138:0/1442304553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f9410a520 0x7f2f9410a960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.233+0000 7f2f93e00640 1 --2- 172.21.15.138:0/1442304553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f94104b10 0x7f2f94104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.233+0000 7f2f93e00640 1 -- 172.21.15.138:0/1442304553 >> 172.21.15.138:0/1442304553 conn(0x7f2f94100540 msgr2=0x7f2f94102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.233+0000 7f2f93e00640 1 -- 172.21.15.138:0/1442304553 shutdown_connections 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.233+0000 7f2f93e00640 1 -- 172.21.15.138:0/1442304553 wait complete. 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.234+0000 7f2f93e00640 1 Processor -- start 2024-08-20T01:10:21.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.234+0000 7f2f93e00640 1 -- start start 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 0x7f2f9419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9410aea0 0x7f2f941a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94114bb0 con 0x7f2f9410aea0 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94114a30 con 0x7f2f9410a520 2024-08-20T01:10:21.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f94114d30 con 0x7f2f94104b10 2024-08-20T01:10:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 0x7f2f9419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46232/0 (socket says 172.21.15.138:46232) 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 0x7f2f9419b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35800/0 (socket says 172.21.15.138:35800) 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 -- 172.21.15.138:0/4164607404 learned_addr learned my addr 172.21.15.138:0/4164607404 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f93400640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9410aea0 0x7f2f941a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 -- 172.21.15.138:0/4164607404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 msgr2=0x7f2f9419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 0x7f2f9419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 -- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9410aea0 msgr2=0x7f2f941a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9410aea0 0x7f2f941a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.235+0000 7f2f92a00640 1 -- 172.21.15.138:0/4164607404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f941a0ff0 con 0x7f2f94104b10 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f92a00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e9f0 tx=0x7f2f8400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f8b400640 1 -- 172.21.15.138:0/4164607404 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400cd80 con 0x7f2f94104b10 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f8b400640 1 -- 172.21.15.138:0/4164607404 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f840108e0 con 0x7f2f94104b10 2024-08-20T01:10:21.236 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f941a12e0 con 0x7f2f94104b10 2024-08-20T01:10:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f8b400640 1 -- 172.21.15.138:0/4164607404 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84018980 con 0x7f2f94104b10 2024-08-20T01:10:21.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.236+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f941a7b70 con 0x7f2f94104b10 2024-08-20T01:10:21.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f8b400640 1 -- 172.21.15.138:0/4164607404 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2f84010430 con 0x7f2f94104b10 2024-08-20T01:10:21.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 Processor -- start 2024-08-20T01:10:21.239 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 -- start start 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f5c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 0x561091f68940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561091efc670 con 0x561091f5c5d0 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561091ed07c0 con 0x561091f68e80 2024-08-20T01:10:21.240 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.240+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561091ebfcc0 con 0x561091f5cef0 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 0x561091f68940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35816/0 (socket says 172.21.15.138:35816) 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 0x561091f68940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46240/0 (socket says 172.21.15.138:46240) 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 -- 172.21.15.138:0/3544084672 learned_addr learned my addr 172.21.15.138:0/3544084672 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f93400640 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f5c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 -- 172.21.15.138:0/3544084672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 msgr2=0x561091f68940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.241 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 0x561091f68940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.242 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 -- 172.21.15.138:0/3544084672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 msgr2=0x561091f5c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f5c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.241+0000 7f2f92000640 1 -- 172.21.15.138:0/3544084672 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561091e53110 con 0x561091f68e80 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f92000640 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f80000c00 tx=0x7f2f80004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9560f987cfc05b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f7fe00640 1 -- 172.21.15.138:0/3544084672 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f80035040 con 0x561091f68e80 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f7fe00640 1 -- 172.21.15.138:0/3544084672 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f800054f0 con 0x561091f68e80 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f7fe00640 1 -- 172.21.15.138:0/3544084672 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800035f0 con 0x561091f68e80 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f8b400640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 0x7f2f780b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f8b400640 1 -- 172.21.15.138:0/4164607404 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2f840e4700 con 0x7f2f94104b10 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.242+0000 7f2f93400640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 0x7f2f780b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/3544084672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 msgr2=0x561091f6b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f80000c00 tx=0x7f2f80004040 comp rx=0 tx=0).stop 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/3544084672 shutdown_connections 2024-08-20T01:10:21.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f6b270 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5cef0 0x561091f68940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/3544084672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f5c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/3544084672 >> 172.21.15.138:0/3544084672 conn(0x561091f5a400 msgr2=0x561091f5b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/3544084672 shutdown_connections 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f93400640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 0x7f2f780b6320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0099a0 tx=0x7f2f8c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.243+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/3544084672 wait complete. 2024-08-20T01:10:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 -- start start 2024-08-20T01:10:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 Processor -- start 2024-08-20T01:10:21.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 0x561091f65dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f66550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561091e65780 con 0x561091f5c5d0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561091ec3d00 con 0x561091f5cef0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.244+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561091d227a0 con 0x561091f68e80 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f9c3e3740 10 client.0 _ll_register_callbacks cb 0x561091e6d940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f93400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f66550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35534/0 (socket says 172.21.15.138:35534) 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 -- 172.21.15.138:0/1969211225 learned_addr learned my addr 172.21.15.138:0/1969211225 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92000640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 0x561091f65dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.246 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 -- 172.21.15.138:0/1969211225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 msgr2=0x561091f66550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f66550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 -- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 msgr2=0x561091f65dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 0x561091f65dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92a00640 1 -- 172.21.15.138:0/1969211225 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561091e729c0 con 0x561091f5c5d0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.245+0000 7f2f92000640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 0x561091f65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e9c0 tx=0x7f2f840aa8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1119820901b1a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f840e1f30 con 0x561091f5c5d0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2f840e42e0 con 0x561091f5c5d0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f800054f0 con 0x561091f5c5d0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f840e44e0 con 0x561091f5c5d0 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f9c3e3740 10 client.15729 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:21.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.246+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561091c2eb90 con 0x561091f5c5d0 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f800054f0 con 0x561091f5c5d0 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f2f840ae070 con 0x561091f5c5d0 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 10 client.15729 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 10 client.15729 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 20 client.15729 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091f78830 0x561091f7acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 4 client.15729 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561091ed2710 con 0x561091f78830 2024-08-20T01:10:21.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.247+0000 7f2f92000640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091f78830 0x561091f7acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.248+0000 7f2f7cc00640 10 client.15729 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.248+0000 7f2f92000640 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091f78830 0x561091f7acc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200531c269f74c38 server_cookie=5cf0582ada98e35c in_seq=0 out_seq=0 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1969211225 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561091ed2710 con 0x561091f78830 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f7cc00640 10 client.15729 handle_command_reply: tid=0 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 client.15729 shutdown 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 msgr2=0x561091f68930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e9c0 tx=0x7f2f840aa8b0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 shutdown_connections 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091f78830 0x561091f7acc0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f66550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5cef0 0x561091f65dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1969211225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5c5d0 0x561091f68930 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f7cc00640 0 client.15729 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 >> 172.21.15.138:0/1969211225 conn(0x561091f5a400 msgr2=0x561091f5ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 shutdown_connections 2024-08-20T01:10:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.249+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1969211225 wait complete. 2024-08-20T01:10:21.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.250+0000 7f2f9c3e3740 20 client.15729 trim_cache size 0 max 16384 2024-08-20T01:10:21.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.270+0000 7f2f9c3e3740 1 Processor -- start 2024-08-20T01:10:21.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 -- start start 2024-08-20T01:10:21.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f80b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.270 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f62fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941b43d0 con 0x561091f5cef0 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941b1c70 con 0x561091f68e80 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941afac0 con 0x561091f5c5d0 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46262/0 (socket says 172.21.15.138:46262) 2024-08-20T01:10:21.271 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 -- 172.21.15.138:0/2543155887 learned_addr learned my addr 172.21.15.138:0/2543155887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:21.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f93400640 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f62fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.272 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92000640 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f80b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 -- 172.21.15.138:0/2543155887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 msgr2=0x561091f62fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f62fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 -- 172.21.15.138:0/2543155887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 msgr2=0x561091f80b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f80b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.271+0000 7f2f92a00640 1 -- 172.21.15.138:0/2543155887 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561091f5cd20 con 0x561091f5c5d0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.272+0000 7f2f92a00640 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f840aac60 tx=0x7f2f840aa9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56f9c117f20278c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.272+0000 7f2f7f400640 1 -- 172.21.15.138:0/2543155887 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f840aaa70 con 0x561091f5c5d0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.272+0000 7f2f7f400640 1 -- 172.21.15.138:0/2543155887 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f840ab660 con 0x561091f5c5d0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.272+0000 7f2f7f400640 1 -- 172.21.15.138:0/2543155887 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f84018b20 con 0x561091f5c5d0 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/2543155887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 msgr2=0x561091f805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f840aac60 tx=0x7f2f840aa9e0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/2543155887 shutdown_connections 2024-08-20T01:10:21.273 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f68e80 0x561091f62fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f80b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/2543155887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f5c5d0 0x561091f805d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/2543155887 >> 172.21.15.138:0/2543155887 conn(0x561091f5a400 msgr2=0x561091f67bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/2543155887 shutdown_connections 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.273+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/2543155887 wait complete. 2024-08-20T01:10:21.274 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 -- start start 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 Processor -- start 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5c5d0 0x561091f62eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f633f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941b0130 con 0x561091f5cef0 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941b15a0 con 0x561091f5c5d0 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2f941b0800 con 0x561091f68e80 2024-08-20T01:10:21.276 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.274+0000 7f2f9c3e3740 10 client.0 _ll_register_callbacks cb 0x7f2f840ea220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f92000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46270/0 (socket says 172.21.15.138:46270) 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 -- 172.21.15.138:0/1740226275 learned_addr learned my addr 172.21.15.138:0/1740226275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5c5d0 0x561091f62eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 -- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5c5d0 msgr2=0x561091f62eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5c5d0 0x561091f62eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 -- 172.21.15.138:0/1740226275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 msgr2=0x561091f633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 -- 172.21.15.138:0/1740226275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561091f66130 con 0x561091f68e80 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f93400640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009ec0 tx=0x7f2f8c014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b131991819b04be2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.275+0000 7f2f7cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:10:21.277 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f8c003b70 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f840ab660 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2f8c003d10 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2f8c034050 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f9c3e3740 10 client.25744 fetch_fsmap learned FSMap version 28 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561091fe3830 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.276+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f840ab660 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 28) ==== 8366+0+0 (secure 0 0 0) 0x7f2f8c0296e0 con 0x561091f68e80 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 10 client.25744 fetch_fsmap finished waiting for FSMap version 28 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 10 client.25744 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 20 client.25744 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091fe4050 0x561091fe6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 4 client.25744 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.2, "await_for": 45.0}] 2024-08-20T01:10:21.278 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.2, "await_for": 45.0}) -- 0x561091ece1b0 con 0x561091fe4050 2024-08-20T01:10:21.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.277+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091fe4050 0x561091fe6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:21.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.278+0000 7f2f92a00640 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091fe4050 0x561091fe6470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13364aeb56087fef server_cookie=315d9964aa8d7a7c in_seq=0 out_seq=0 2024-08-20T01:10:21.279 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.278+0000 7f2f7cc00640 10 client.25744 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f7cc00640 1 -- 172.21.15.138:0/1740226275 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561091ece1b0 con 0x561091fe4050 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f7cc00640 10 client.25744 handle_command_reply: tid=0 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 client.25744 shutdown 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 msgr2=0x561091f63930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009ec0 tx=0x7f2f8c014040 comp rx=0 tx=0).stop 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 shutdown_connections 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561091fe4050 0x561091fe6470 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561091f68e80 0x561091f63930 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561091f5cef0 0x561091f633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f7cc00640 0 client.25744 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 --2- 172.21.15.138:0/1740226275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561091f5c5d0 0x561091f62eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.456+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 >> 172.21.15.138:0/1740226275 conn(0x561091f5a400 msgr2=0x561091f73b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 shutdown_connections 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f9c3e3740 1 -- 172.21.15.138:0/1740226275 wait complete. 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f9c3e3740 20 client.25744 trim_cache size 0 max 16384 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:10:21.457 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 28, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 10, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "afa60021": { 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 10, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.2, 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:10:21.458 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:21.459 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 msgr2=0x7f2f780b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 0x7f2f780b6320 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c0099a0 tx=0x7f2f8c009500 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 msgr2=0x7f2f9419af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.457+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400e9f0 tx=0x7f2f8400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 shutdown_connections 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2f780b3e60 0x7f2f780b6320 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f9410aea0 0x7f2f941a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f9410a520 0x7f2f9419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 --2- 172.21.15.138:0/4164607404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f94104b10 0x7f2f9419af40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 >> 172.21.15.138:0/4164607404 conn(0x7f2f94100540 msgr2=0x7f2f94103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 shutdown_connections 2024-08-20T01:10:21.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:21.458+0000 7f2f93e00640 1 -- 172.21.15.138:0/4164607404 wait complete. 2024-08-20T01:10:21.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:21 smithi167 ceph-mon[21965]: pgmap v429: 129 pgs: 129 active+clean; 48 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-08-20T01:10:21.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:21 smithi142 ceph-mon[21799]: pgmap v429: 129 pgs: 129 active+clean; 48 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-08-20T01:10:22.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:21 smithi138 ceph-mon[18076]: pgmap v429: 129 pgs: 129 active+clean; 48 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 128 op/s 2024-08-20T01:10:22.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 6.05 MB/sec warmup 34 sec latency 761.749 ms 2024-08-20T01:10:22.470 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'afa60021', quiesce took 4.6 seconds. Will release after: 5.6 2024-08-20T01:10:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:22 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:22 smithi138 ceph-mon[18076]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-08-20T01:10:22.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:22 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:22.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:22 smithi167 ceph-mon[21965]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-08-20T01:10:22.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:22 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:22 smithi142 ceph-mon[21799]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 121 op/s 2024-08-20T01:10:23.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.88 MB/sec warmup 35 sec latency 1761.815 ms 2024-08-20T01:10:24.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.71 MB/sec warmup 36 sec latency 2761.936 ms 2024-08-20T01:10:25.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.56 MB/sec warmup 37 sec latency 3762.062 ms 2024-08-20T01:10:25.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:25 smithi167 ceph-mon[21965]: pgmap v431: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-08-20T01:10:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:25 smithi142 ceph-mon[21799]: pgmap v431: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-08-20T01:10:26.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:25 smithi138 ceph-mon[18076]: pgmap v431: 129 pgs: 129 active+clean; 37 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 128 op/s 2024-08-20T01:10:26.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.41 MB/sec warmup 38 sec latency 4762.149 ms 2024-08-20T01:10:27.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.27 MB/sec warmup 39 sec latency 5762.229 ms 2024-08-20T01:10:27.918 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:27 smithi138 ceph-mon[18076]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:10:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:27 smithi167 ceph-mon[21965]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:10:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:27 smithi142 ceph-mon[21799]: pgmap v432: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:10:28.071 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:10:28.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.14 MB/sec warmup 40 sec latency 6762.336 ms 2024-08-20T01:10:28.441 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 -- 172.21.15.138:0/3168690788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca34104b00 msgr2=0x7fca34104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 --2- 172.21.15.138:0/3168690788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca34104b00 0x7fca34104ee0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fca24009900 tx=0x7fca2402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 -- 172.21.15.138:0/3168690788 shutdown_connections 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 --2- 172.21.15.138:0/3168690788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca34111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 --2- 172.21.15.138:0/3168690788 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca3410a510 0x7fca3410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 --2- 172.21.15.138:0/3168690788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca34104b00 0x7fca34104ee0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 -- 172.21.15.138:0/3168690788 >> 172.21.15.138:0/3168690788 conn(0x7fca34100550 msgr2=0x7fca34102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 -- 172.21.15.138:0/3168690788 shutdown_connections 2024-08-20T01:10:29.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.067+0000 7fca33e00640 1 -- 172.21.15.138:0/3168690788 wait complete. 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.068+0000 7fca33e00640 1 Processor -- start 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.068+0000 7fca33e00640 1 -- start start 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34104b00 0x7fca3419af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca341a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca34114b60 con 0x7fca3410a510 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca341149e0 con 0x7fca3410ae90 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca34114ce0 con 0x7fca34104b00 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35182/0 (socket says 172.21.15.138:35182) 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 -- 172.21.15.138:0/1151819524 learned_addr learned my addr 172.21.15.138:0/1151819524 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34104b00 0x7fca3419af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:29.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33400640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca341a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 -- 172.21.15.138:0/1151819524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34104b00 msgr2=0x7fca3419af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34104b00 0x7fca3419af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 -- 172.21.15.138:0/1151819524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 msgr2=0x7fca341a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca341a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca32000640 1 -- 172.21.15.138:0/1151819524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca341a1000 con 0x7fca3410a510 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.069+0000 7fca33400640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca341a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:29.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca32000640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9f0 tx=0x7fca2000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca2000ccc0 con 0x7fca3410a510 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca200108e0 con 0x7fca3410a510 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20018880 con 0x7fca3410a510 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca341a12f0 con 0x7fca3410a510 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.070+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca341a79b0 con 0x7fca3410a510 2024-08-20T01:10:29.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.071+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca34104ee0 con 0x7fca3410a510 2024-08-20T01:10:29.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.074+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fca20010430 con 0x7fca3410a510 2024-08-20T01:10:29.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.075+0000 7fca2b400640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 0x7fca180b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:29.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.075+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fca200e4ba0 con 0x7fca3410a510 2024-08-20T01:10:29.074 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.075+0000 7fca32a00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 0x7fca180b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:29.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.076+0000 7fca32a00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 0x7fca180b6320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca24005ab0 tx=0x7fca24005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:29.076 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.077+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fca200abb00 con 0x7fca3410a510 2024-08-20T01:10:29.107 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 5.02 MB/sec warmup 41 sec latency 7762.404 ms 2024-08-20T01:10:29.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.297+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca3402e340 con 0x7fca3410a510 2024-08-20T01:10:29.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.297+0000 7fca2b400640 1 -- 172.21.15.138:0/1151819524 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11938 (secure 0 0 0) 0x7fca200ab400 con 0x7fca3410a510 2024-08-20T01:10:29.297 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:10:29.297 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":29,"btime":"2024-08-20T01:10:21:636435+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:10:21.140831+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:10:29.297 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 29 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 msgr2=0x7fca180b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 0x7fca180b6320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fca24005ab0 tx=0x7fca24005a20 comp rx=0 tx=0).stop 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 msgr2=0x7fca3419b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fca2000e9f0 tx=0x7fca2000edf0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 shutdown_connections 2024-08-20T01:10:29.299 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca180b3e60 0x7fca180b6320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca3410ae90 0x7fca341a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca3410a510 0x7fca3419b490 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 --2- 172.21.15.138:0/1151819524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca34104b00 0x7fca3419af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 >> 172.21.15.138:0/1151819524 conn(0x7fca34100550 msgr2=0x7fca34106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 shutdown_connections 2024-08-20T01:10:29.300 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:29.300+0000 7fca33e00640 1 -- 172.21.15.138:0/1151819524 wait complete. 2024-08-20T01:10:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:29 smithi167 ceph-mon[21965]: pgmap v433: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-08-20T01:10:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:29 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1151819524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:29 smithi142 ceph-mon[21799]: pgmap v433: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-08-20T01:10:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:29 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1151819524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:30.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:29 smithi138 ceph-mon[18076]: pgmap v433: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 122 op/s 2024-08-20T01:10:30.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:29 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1151819524' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:30.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 4.90 MB/sec warmup 42 sec latency 8762.470 ms 2024-08-20T01:10:30.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id afa60021 --release --await' 2024-08-20T01:10:30.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id afa60021 --release --await 2024-08-20T01:10:30.673 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:31.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 34981 4.78 MB/sec warmup 43 sec latency 9762.602 ms 2024-08-20T01:10:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 -- 172.21.15.138:0/2653512473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 msgr2=0x7fe8b010ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/2653512473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b010ab00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe898009900 tx=0x7fe89802a410 comp rx=0 tx=0).stop 2024-08-20T01:10:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 -- 172.21.15.138:0/2653512473 shutdown_connections 2024-08-20T01:10:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/2653512473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 0x7fe8b0111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/2653512473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b010ab00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/2653512473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b0105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 -- 172.21.15.138:0/2653512473 >> 172.21.15.138:0/2653512473 conn(0x7fe8b0100720 msgr2=0x7fe8b0102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 -- 172.21.15.138:0/2653512473 shutdown_connections 2024-08-20T01:10:31.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.490+0000 7fe8b7c00640 1 -- 172.21.15.138:0/2653512473 wait complete. 2024-08-20T01:10:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 Processor -- start 2024-08-20T01:10:31.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 -- start start 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b019b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 0x7fe8b01a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0114d10 con 0x7fe8b0104cb0 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0114b90 con 0x7fe8b010b040 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b7c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b0114e90 con 0x7fe8b010a6c0 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.492+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35204/0 (socket says 172.21.15.138:35204) 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 -- 172.21.15.138:0/3909423442 learned_addr learned my addr 172.21.15.138:0/3909423442 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b019b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 0x7fe8b01a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 -- 172.21.15.138:0/3909423442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 msgr2=0x7fe8b019b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b019b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 -- 172.21.15.138:0/3909423442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 msgr2=0x7fe8b01a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 0x7fe8b01a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 -- 172.21.15.138:0/3909423442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8b01a1270 con 0x7fe8b0104cb0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.493+0000 7fe8b5200640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000ea60 tx=0x7fe8a000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.494+0000 7fe8ae000640 1 -- 172.21.15.138:0/3909423442 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000cde0 con 0x7fe8b0104cb0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.494+0000 7fe8ae000640 1 -- 172.21.15.138:0/3909423442 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a00035f0 con 0x7fe8b0104cb0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.494+0000 7fe8ae000640 1 -- 172.21.15.138:0/3909423442 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0010690 con 0x7fe8b0104cb0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.494+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8b01a1560 con 0x7fe8b0104cb0 2024-08-20T01:10:31.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.494+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8b01a7c20 con 0x7fe8b0104cb0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 Processor -- start 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 -- start start 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ae000640 1 -- 172.21.15.138:0/3909423442 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe8a00037a0 con 0x7fe8b0104cb0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ed060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c1f282670 con 0x561c1f2e1610 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561c1f256d70 con 0x561c1f2e0cf0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.497+0000 7fe8ba804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c1f246270 con 0x561c1f2ed5a0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:34106/0 (socket says 172.21.15.138:34106) 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 -- 172.21.15.138:0/4066730756 learned_addr learned my addr 172.21.15.138:0/4066730756 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51162/0 (socket says 172.21.15.138:51162) 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8b5200640 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 -- 172.21.15.138:0/4066730756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 msgr2=0x561c1f2e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 -- 172.21.15.138:0/4066730756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 msgr2=0x561c1f2ed060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ed060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.498+0000 7fe8a7e00640 1 -- 172.21.15.138:0/4066730756 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1f1e2870 con 0x561c1f2ed5a0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe898004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4059395644959b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8a6000640 1 -- 172.21.15.138:0/4066730756 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898035040 con 0x561c1f2ed5a0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8a6000640 1 -- 172.21.15.138:0/4066730756 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8980054f0 con 0x561c1f2ed5a0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8a6000640 1 -- 172.21.15.138:0/4066730756 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8980035f0 con 0x561c1f2ed5a0 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8ae000640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 0x7fe8940b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8ae000640 1 -- 172.21.15.138:0/3909423442 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe8a00e45f0 con 0x7fe8b0104cb0 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.499+0000 7fe8b5c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 0x7fe8940b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 -- 172.21.15.138:0/4066730756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 msgr2=0x561c1f2ef990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe898000c00 tx=0x7fe898004040 comp rx=0 tx=0).stop 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 -- 172.21.15.138:0/4066730756 shutdown_connections 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ef990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ed060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 --2- 172.21.15.138:0/4066730756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 -- 172.21.15.138:0/4066730756 >> 172.21.15.138:0/4066730756 conn(0x561c1f2deb20 msgr2=0x561c1f2dfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 -- 172.21.15.138:0/4066730756 shutdown_connections 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8b5c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 0x7fe8940b6280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8005fd0 tx=0x7fe8a8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.500+0000 7fe8ba804740 1 -- 172.21.15.138:0/4066730756 wait complete. 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 -- start start 2024-08-20T01:10:31.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 Processor -- start 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 0x561c1f2ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561c1f1ccff0 con 0x561c1f2e1610 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561c1f24a2b0 con 0x561c1f2ed5a0 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561c1f0a8bb0 con 0x561c1f2e0cf0 2024-08-20T01:10:31.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.501+0000 7fe8ba804740 10 client.0 _ll_register_callbacks cb 0x561c1ef51d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 0x561c1f2ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35234/0 (socket says 172.21.15.138:35234) 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 -- 172.21.15.138:0/2459748755 learned_addr learned my addr 172.21.15.138:0/2459748755 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8b5c00640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 -- 172.21.15.138:0/2459748755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 msgr2=0x561c1f2ecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 0x561c1f2ecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 -- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2ed5a0 msgr2=0x561c1f2ea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 -- 172.21.15.138:0/2459748755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1efa5b00 con 0x561c1f2e1610 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 0x561c1f2ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:31.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe8980049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590f6908213517e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.502+0000 7fe89ea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898035040 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe898004e50 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8980054f0 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe898009520 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe8ba804740 10 client.15753 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c1f01c960 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.503+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8980054f0 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe898055030 con 0x561c1f2e1610 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 10 client.15753 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 20 client.15753 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:31.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f2fc9e0 0x561c1f2fedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 4 client.15753 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c1f258cc0 con 0x561c1f2fc9e0 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f2fc9e0 0x561c1f2fedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f2fc9e0 0x561c1f2fedd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d648e87f405873 server_cookie=1842c28facdc482a in_seq=0 out_seq=0 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.504+0000 7fe89ea00640 10 client.15753 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.505+0000 7fe89ea00640 1 -- 172.21.15.138:0/2459748755 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561c1f258cc0 con 0x561c1f2fc9e0 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.505+0000 7fe89ea00640 10 client.15753 handle_command_reply: tid=0 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.505+0000 7fe8ba804740 1 client.15753 shutdown 2024-08-20T01:10:31.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.505+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 msgr2=0x561c1f2ea1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.505+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe898006010 tx=0x7fe8980049a0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 shutdown_connections 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f2fc9e0 0x561c1f2fedd0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2e1610 0x561c1f2ea1a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2459748755 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2e0cf0 0x561c1f2ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe89ea00640 0 client.15753 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 >> 172.21.15.138:0/2459748755 conn(0x561c1f2deb20 msgr2=0x561c1f2e46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 shutdown_connections 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 1 -- 172.21.15.138:0/2459748755 wait complete. 2024-08-20T01:10:31.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.506+0000 7fe8ba804740 20 client.15753 trim_cache size 0 max 16384 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.523+0000 7fe8ba804740 1 Processor -- start 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.523+0000 7fe8ba804740 1 -- start start 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f2e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01b45a0 con 0x561c1f2f9310 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01b1e10 con 0x561c1f2e0cf0 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8ba804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01afc60 con 0x561c1f2ed5a0 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51186/0 (socket says 172.21.15.138:51186) 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 -- 172.21.15.138:0/1151407322 learned_addr learned my addr 172.21.15.138:0/1151407322 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:31.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5c00640 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f2e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 -- 172.21.15.138:0/1151407322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 msgr2=0x561c1f2ebee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ebee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 -- 172.21.15.138:0/1151407322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 msgr2=0x561c1f2e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f2e73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.524+0000 7fe8b5200640 1 -- 172.21.15.138:0/1151407322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c1f2e5620 con 0x561c1f2e0cf0 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.525+0000 7fe8b5200640 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00e19f0 tx=0x7fe8a00e6ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d1cc6d5a4115b60 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.525+0000 7fe8a5600640 1 -- 172.21.15.138:0/1151407322 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00aa5c0 con 0x561c1f2e0cf0 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.525+0000 7fe8a5600640 1 -- 172.21.15.138:0/1151407322 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a00ad070 con 0x561c1f2e0cf0 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.525+0000 7fe8a5600640 1 -- 172.21.15.138:0/1151407322 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00b3f00 con 0x561c1f2e0cf0 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- 172.21.15.138:0/1151407322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 msgr2=0x561c1f2e8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00e19f0 tx=0x7fe8a00e6ab0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- 172.21.15.138:0/1151407322 shutdown_connections 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f2e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f2ebee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 --2- 172.21.15.138:0/1151407322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f2e8a40 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- 172.21.15.138:0/1151407322 >> 172.21.15.138:0/1151407322 conn(0x561c1f2deb20 msgr2=0x561c1f2eb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- 172.21.15.138:0/1151407322 shutdown_connections 2024-08-20T01:10:31.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- 172.21.15.138:0/1151407322 wait complete. 2024-08-20T01:10:31.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 -- start start 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.526+0000 7fe8ba804740 1 Processor -- start 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f300810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f300d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01b02d0 con 0x561c1f2f9310 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01b1740 con 0x561c1f2e0cf0 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe8b01b09a0 con 0x561c1f2ed5a0 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8ba804740 10 client.0 _ll_register_callbacks cb 0x7fe8a00ea060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8b5c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f300d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8a7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f300810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8b5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51200/0 (socket says 172.21.15.138:51200) 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.527+0000 7fe8b5200640 1 -- 172.21.15.138:0/2112905809 learned_addr learned my addr 172.21.15.138:0/2112905809 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 -- 172.21.15.138:0/2112905809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 msgr2=0x561c1f300810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f300810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 -- 172.21.15.138:0/2112905809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 msgr2=0x561c1f300d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f300d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 -- 172.21.15.138:0/2112905809 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c1f2e5d90 con 0x561c1f2e0cf0 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f300810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8b5200640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00a6ed0 tx=0x7fe8a0003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e62ec0a5761c1e96 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe89ea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:31.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a00af230 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe8a00a70e0 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8a00ad070 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8a00aae50 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8ba804740 10 client.25802 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.528+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c1f2e9120 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8a00ad070 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fe8a00e41e0 con 0x561c1f2e0cf0 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 10 client.25802 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 10 client.25802 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 20 client.25802 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f368560 0x561c1f36a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 4 client.25802 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "afa60021", "release": true, "await": true}] 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "afa60021", "release": true, "await": true}) -- 0x561c1f2efd70 con 0x561c1f368560 2024-08-20T01:10:31.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.529+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f368560 0x561c1f36a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:31.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.530+0000 7fe8a7e00640 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f368560 0x561c1f36a980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bc879f19e5c814b server_cookie=c2f06497dcf9f2bd in_seq=0 out_seq=0 2024-08-20T01:10:31.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.530+0000 7fe89ea00640 10 client.25802 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.540+0000 7fe89ea00640 1 -- 172.21.15.138:0/2112905809 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561c1f2efd70 con 0x561c1f368560 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.540+0000 7fe89ea00640 10 client.25802 handle_command_reply: tid=0 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.540+0000 7fe8ba804740 1 client.25802 shutdown 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 msgr2=0x561c1f302310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00a6ed0 tx=0x7fe8a0003cf0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 shutdown_connections 2024-08-20T01:10:31.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561c1f368560 0x561c1f36a980 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561c1f2f9310 0x561c1f300d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561c1f2ed5a0 0x561c1f300810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 --2- 172.21.15.138:0/2112905809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561c1f2e0cf0 0x561c1f302310 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe89ea00640 0 client.25802 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 >> 172.21.15.138:0/2112905809 conn(0x561c1f2deb20 msgr2=0x561c1f2e3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 shutdown_connections 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 1 -- 172.21.15.138:0/2112905809 wait complete. 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.541+0000 7fe8ba804740 20 client.25802 trim_cache size 0 max 16384 2024-08-20T01:10:31.541 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 29, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 12, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "afa60021": { 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 12, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:10:31.542 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 130.2, 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:31.543 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 msgr2=0x7fe8940b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 0x7fe8940b6280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8005fd0 tx=0x7fe8a8009500 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 msgr2=0x7fe8b019b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000ea60 tx=0x7fe8a000ef20 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 shutdown_connections 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe8940b3dc0 0x7fe8940b6280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe8b010b040 0x7fe8b01a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe8b010a6c0 0x7fe8b019b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 --2- 172.21.15.138:0/3909423442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe8b0104cb0 0x7fe8b019b1c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 >> 172.21.15.138:0/3909423442 conn(0x7fe8b0100720 msgr2=0x7fe8b0102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:31.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 shutdown_connections 2024-08-20T01:10:31.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:31.542+0000 7fe8b7c00640 1 -- 172.21.15.138:0/3909423442 wait complete. 2024-08-20T01:10:31.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:31 smithi138 ceph-mon[18076]: pgmap v434: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 121 op/s 2024-08-20T01:10:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:31 smithi167 ceph-mon[21965]: pgmap v434: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 121 op/s 2024-08-20T01:10:31.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:31 smithi142 ceph-mon[21799]: pgmap v434: 129 pgs: 129 active+clean; 38 MiB data, 458 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 793 KiB/s wr, 121 op/s 2024-08-20T01:10:32.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 36193 4.83 MB/sec warmup 44 sec latency 10189.958 ms 2024-08-20T01:10:32.515 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'afa60021', total seconds elapsed: 14.7 2024-08-20T01:10:33.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 39210 5.11 MB/sec warmup 45 sec latency 12.040 ms 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 -- 172.21.15.138:0/4043459364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf90104a20 msgr2=0x7fbf90104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 --2- 172.21.15.138:0/4043459364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf90104a20 0x7fbf90104e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf80009900 tx=0x7fbf8002a410 comp rx=0 tx=0).stop 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 -- 172.21.15.138:0/4043459364 shutdown_connections 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 --2- 172.21.15.138:0/4043459364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010ada0 0x7fbf901115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 --2- 172.21.15.138:0/4043459364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf9010a420 0x7fbf9010a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 --2- 172.21.15.138:0/4043459364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf90104a20 0x7fbf90104e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.361+0000 7fbf8f200640 1 -- 172.21.15.138:0/4043459364 >> 172.21.15.138:0/4043459364 conn(0x7fbf90100460 msgr2=0x7fbf90102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.362+0000 7fbf8f200640 1 -- 172.21.15.138:0/4043459364 shutdown_connections 2024-08-20T01:10:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.362+0000 7fbf8f200640 1 -- 172.21.15.138:0/4043459364 wait complete. 2024-08-20T01:10:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 Processor -- start 2024-08-20T01:10:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 -- start start 2024-08-20T01:10:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf90104a20 0x7fbf9019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90114ab0 con 0x7fbf9010a420 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90114930 con 0x7fbf9010ada0 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90114c30 con 0x7fbf90104a20 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35276/0 (socket says 172.21.15.138:35276) 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8d400640 1 -- 172.21.15.138:0/3054518388 learned_addr learned my addr 172.21.15.138:0/3054518388 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51222/0 (socket says 172.21.15.138:51222) 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.363+0000 7fbf8de00640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf90104a20 0x7fbf9019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 -- 172.21.15.138:0/3054518388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf90104a20 msgr2=0x7fbf9019afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf90104a20 0x7fbf9019afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 -- 172.21.15.138:0/3054518388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 msgr2=0x7fbf901a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 -- 172.21.15.138:0/3054518388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf901a1070 con 0x7fbf9010a420 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8e800640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00e9f0 tx=0x7fbf7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf86a00640 1 -- 172.21.15.138:0/3054518388 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c00cd80 con 0x7fbf9010a420 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf901a1360 con 0x7fbf9010a420 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf86a00640 1 -- 172.21.15.138:0/3054518388 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7c0054d0 con 0x7fbf9010a420 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf86a00640 1 -- 172.21.15.138:0/3054518388 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c010690 con 0x7fbf9010a420 2024-08-20T01:10:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.364+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf901a7a20 con 0x7fbf9010a420 2024-08-20T01:10:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.367+0000 7fbf97dd2740 1 Processor -- start 2024-08-20T01:10:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 -- start start 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0051bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 0x555ae005db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae005e080 0x555ae0060470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555adffc7210 con 0x555ae00517d0 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555adffb6710 con 0x555ae005e080 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555adffba750 con 0x555ae00520f0 2024-08-20T01:10:33.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf86a00640 1 -- 172.21.15.138:0/3054518388 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fbf7c004e90 con 0x7fbf9010a420 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0051bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 0x555ae005db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf8e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0051bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35290/0 (socket says 172.21.15.138:35290) 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf8e800640 1 -- 172.21.15.138:0/601624522 learned_addr learned my addr 172.21.15.138:0/601624522 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.368+0000 7fbf8d400640 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae005e080 0x555ae0060470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 -- 172.21.15.138:0/601624522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae005e080 msgr2=0x555ae0060470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae005e080 0x555ae0060470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 -- 172.21.15.138:0/601624522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 msgr2=0x555ae0051bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0051bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 -- 172.21.15.138:0/601624522 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adff82de0 con 0x555ae00520f0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf8de00640 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 0x555ae005db40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002aa00 tx=0x7fbf80005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f45c8777ff83ac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf7b400640 1 -- 172.21.15.138:0/601624522 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80035040 con 0x555ae00520f0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf7b400640 1 -- 172.21.15.138:0/601624522 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf800054f0 con 0x555ae00520f0 2024-08-20T01:10:33.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.369+0000 7fbf7b400640 1 -- 172.21.15.138:0/601624522 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf800035f0 con 0x555ae00520f0 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf86a00640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 0x7fbf740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf86a00640 1 -- 172.21.15.138:0/3054518388 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fbf7c0e4730 con 0x7fbf9010a420 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 -- 172.21.15.138:0/601624522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 msgr2=0x555ae005db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 0x555ae005db40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002aa00 tx=0x7fbf80005c00 comp rx=0 tx=0).stop 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf8e800640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 0x7fbf740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 -- 172.21.15.138:0/601624522 shutdown_connections 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae005e080 0x555ae0060470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00520f0 0x555ae005db40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/601624522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0051bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 -- 172.21.15.138:0/601624522 >> 172.21.15.138:0/601624522 conn(0x555ae004f600 msgr2=0x555ae00507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 -- 172.21.15.138:0/601624522 shutdown_connections 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.370+0000 7fbf97dd2740 1 -- 172.21.15.138:0/601624522 wait complete. 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf8e800640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 0x7fbf740b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009fd0 tx=0x7fbf88009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 -- start start 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 Processor -- start 2024-08-20T01:10:33.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00520f0 0x555ae005b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae005bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555adfeefab0 con 0x555ae00520f0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555adff5d140 con 0x555ae00517d0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555ae0060d00 con 0x555ae005e080 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf97dd2740 10 client.0 _ll_register_callbacks cb 0x555adff705d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.371+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51246/0 (socket says 172.21.15.138:51246) 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 -- 172.21.15.138:0/2033785373 learned_addr learned my addr 172.21.15.138:0/2033785373 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8d400640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00520f0 0x555ae005b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8e800640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae005bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 -- 172.21.15.138:0/2033785373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 msgr2=0x555ae005bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae005bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 -- 172.21.15.138:0/2033785373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00520f0 msgr2=0x555ae005b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00520f0 0x555ae005b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 -- 172.21.15.138:0/2033785373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adff5e170 con 0x555ae00517d0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf8de00640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002a930 tx=0x7fbf80004530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=464bf792b1a6d68d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf80053060 con 0x555ae00517d0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fbf800035f0 con 0x555ae00517d0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.372+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555adff8da30 con 0x555ae00517d0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf80035050 con 0x555ae00517d0 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 10 client.25814 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:33.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555adfd19790 con 0x555ae00517d0 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555adff8da30 con 0x555ae00517d0 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbf80031740 con 0x555ae00517d0 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 10 client.25814 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 10 client.25814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 20 client.25814 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae006de00 0x555ae00701f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 4 client.25814 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555adffc9160 con 0x555ae006de00 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.373+0000 7fbf8d400640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae006de00 0x555ae00701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.374+0000 7fbf8d400640 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae006de00 0x555ae00701f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acc7d1d5b225a6a2 server_cookie=d1cee3e684703bd4 in_seq=0 out_seq=0 2024-08-20T01:10:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.374+0000 7fbf6fe00640 10 client.25814 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.374+0000 7fbf6fe00640 1 -- 172.21.15.138:0/2033785373 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555adffc9160 con 0x555ae006de00 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.374+0000 7fbf6fe00640 10 client.25814 handle_command_reply: tid=0 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 client.25814 shutdown 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 msgr2=0x555ae005b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf8002a930 tx=0x7fbf80004530 comp rx=0 tx=0).stop 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 shutdown_connections 2024-08-20T01:10:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae006de00 0x555ae00701f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae005bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00520f0 0x555ae005b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/2033785373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00517d0 0x555ae005b030 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 >> 172.21.15.138:0/2033785373 conn(0x555ae004f600 msgr2=0x555ae00551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf6fe00640 0 client.25814 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 shutdown_connections 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 1 -- 172.21.15.138:0/2033785373 wait complete. 2024-08-20T01:10:33.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.375+0000 7fbf97dd2740 20 client.25814 trim_cache size 0 max 16384 2024-08-20T01:10:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 Processor -- start 2024-08-20T01:10:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 -- start start 2024-08-20T01:10:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00517d0 0x555ae005d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae005e080 0x555ae005b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901b4360 con 0x555ae005e080 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901b1bd0 con 0x555ae00520f0 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901afa20 con 0x555ae00517d0 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51252/0 (socket says 172.21.15.138:51252) 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 -- 172.21.15.138:0/1614850967 learned_addr learned my addr 172.21.15.138:0/1614850967 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8e800640 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae005e080 0x555ae005b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8de00640 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00517d0 0x555ae005d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 -- 172.21.15.138:0/1614850967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00517d0 msgr2=0x555ae005d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00517d0 0x555ae005d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 -- 172.21.15.138:0/1614850967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae005e080 msgr2=0x555ae005b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae005e080 0x555ae005b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 -- 172.21.15.138:0/1614850967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adfd92620 con 0x555ae00520f0 2024-08-20T01:10:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf8d400640 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0e1b90 tx=0x7fbf7c0e6c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfbd1f7e33cba7ef server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf7aa00640 1 -- 172.21.15.138:0/1614850967 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0aa940 con 0x555ae00520f0 2024-08-20T01:10:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf7aa00640 1 -- 172.21.15.138:0/1614850967 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7c0aab00 con 0x555ae00520f0 2024-08-20T01:10:33.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.392+0000 7fbf7aa00640 1 -- 172.21.15.138:0/1614850967 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c005180 con 0x555ae00520f0 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 -- 172.21.15.138:0/1614850967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 msgr2=0x555ae0055920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0e1b90 tx=0x7fbf7c0e6c10 comp rx=0 tx=0).stop 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 -- 172.21.15.138:0/1614850967 shutdown_connections 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae005e080 0x555ae005b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0055920 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/1614850967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae00517d0 0x555ae005d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 -- 172.21.15.138:0/1614850967 >> 172.21.15.138:0/1614850967 conn(0x555ae004f600 msgr2=0x555ae005b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 -- 172.21.15.138:0/1614850967 shutdown_connections 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.393+0000 7fbf97dd2740 1 -- 172.21.15.138:0/1614850967 wait complete. 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 -- start start 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 Processor -- start 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0065fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae00573c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901b0090 con 0x555ae00517d0 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901b1500 con 0x555ae00520f0 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbf901b0760 con 0x555ae005e080 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf97dd2740 10 client.0 _ll_register_callbacks cb 0x7fbf88009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf8d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51262/0 (socket says 172.21.15.138:51262) 2024-08-20T01:10:33.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf8d400640 1 -- 172.21.15.138:0/3565940488 learned_addr learned my addr 172.21.15.138:0/3565940488 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.394+0000 7fbf8de00640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0065fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8e800640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae00573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 -- 172.21.15.138:0/3565940488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 msgr2=0x555ae00573c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae00573c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 -- 172.21.15.138:0/3565940488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 msgr2=0x555ae0065fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0065fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 -- 172.21.15.138:0/3565940488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ae0057900 con 0x555ae00520f0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8e800640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae00573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8de00640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf8d400640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0a7070 tx=0x7fbf7c0048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a06269b2bbca33 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf6fe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0c4020 con 0x555ae00520f0 2024-08-20T01:10:33.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf7c0aab00 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fbf7c0e43c0 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbf7c0aa760 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf97dd2740 10 client.25820 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.395+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555ae00d7ee0 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf7c0aab00 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7fbf7c0b5050 con 0x555ae00520f0 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 10 client.25820 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 10 client.25820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 20 client.25820 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae00d8690 0x555ae00daab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 4 client.25820 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555adffc4c00 con 0x555ae00d8690 2024-08-20T01:10:33.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf8de00640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae00d8690 0x555ae00daab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf8de00640 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae00d8690 0x555ae00daab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c101fc134e93ecf5 server_cookie=8b753e974eedb73d in_seq=0 out_seq=0 2024-08-20T01:10:33.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.396+0000 7fbf6fe00640 10 client.25820 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf6fe00640 1 -- 172.21.15.138:0/3565940488 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555adffc4c00 con 0x555ae00d8690 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf6fe00640 10 client.25820 handle_command_reply: tid=0 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 client.25820 shutdown 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 msgr2=0x555ae0066500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0a7070 tx=0x7fbf7c0048f0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 shutdown_connections 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555ae00d8690 0x555ae00daab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555ae005e080 0x555ae00573c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf6fe00640 0 client.25820 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555ae00520f0 0x555ae0066500 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 --2- 172.21.15.138:0/3565940488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555ae00517d0 0x555ae0065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.432+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 >> 172.21.15.138:0/3565940488 conn(0x555ae004f600 msgr2=0x555ae004e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 shutdown_connections 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf97dd2740 1 -- 172.21.15.138:0/3565940488 wait complete. 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf97dd2740 20 client.25820 trim_cache size 0 max 16384 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:10:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 msgr2=0x7fbf740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 0x7fbf740b6200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009fd0 tx=0x7fbf88009500 comp rx=0 tx=0).stop 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 msgr2=0x7fbf9019b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00e9f0 tx=0x7fbf7c00edf0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.433+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 shutdown_connections 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbf740b3d40 0x7fbf740b6200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbf9010ada0 0x7fbf901a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbf9010a420 0x7fbf9019b510 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 --2- 172.21.15.138:0/3054518388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbf90104a20 0x7fbf9019afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 >> 172.21.15.138:0/3054518388 conn(0x7fbf90100460 msgr2=0x7fbf90102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 shutdown_connections 2024-08-20T01:10:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:33.434+0000 7fbf8f200640 1 -- 172.21.15.138:0/3054518388 wait complete. 2024-08-20T01:10:33.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:33 smithi167 ceph-mon[21965]: pgmap v435: 129 pgs: 129 active+clean; 38 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 345 KiB/s wr, 73 op/s 2024-08-20T01:10:33.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:33 smithi142 ceph-mon[21799]: pgmap v435: 129 pgs: 129 active+clean; 38 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 345 KiB/s wr, 73 op/s 2024-08-20T01:10:34.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:33 smithi138 ceph-mon[18076]: pgmap v435: 129 pgs: 129 active+clean; 38 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 345 KiB/s wr, 73 op/s 2024-08-20T01:10:34.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 41455 5.18 MB/sec warmup 46 sec latency 28.407 ms 2024-08-20T01:10:35.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 43861 5.37 MB/sec warmup 47 sec latency 22.074 ms 2024-08-20T01:10:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:35 smithi167 ceph-mon[21965]: pgmap v436: 129 pgs: 129 active+clean; 39 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 242 KiB/s wr, 45 op/s 2024-08-20T01:10:35.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:35 smithi142 ceph-mon[21799]: pgmap v436: 129 pgs: 129 active+clean; 39 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 242 KiB/s wr, 45 op/s 2024-08-20T01:10:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:35 smithi138 ceph-mon[18076]: pgmap v436: 129 pgs: 129 active+clean; 39 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 242 KiB/s wr, 45 op/s 2024-08-20T01:10:36.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 46704 5.62 MB/sec warmup 48 sec latency 12.066 ms 2024-08-20T01:10:37.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 49514 5.80 MB/sec warmup 49 sec latency 22.465 ms 2024-08-20T01:10:37.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:37 smithi167 ceph-mon[21965]: pgmap v437: 129 pgs: 129 active+clean; 43 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 530 KiB/s wr, 97 op/s 2024-08-20T01:10:37.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:37 smithi142 ceph-mon[21799]: pgmap v437: 129 pgs: 129 active+clean; 43 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 530 KiB/s wr, 97 op/s 2024-08-20T01:10:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:37 smithi138 ceph-mon[18076]: pgmap v437: 129 pgs: 129 active+clean; 43 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 530 KiB/s wr, 97 op/s 2024-08-20T01:10:38.108 INFO:tasks.workunit.client.0.smithi138.stdout: 1 51501 5.88 MB/sec warmup 50 sec latency 23.996 ms 2024-08-20T01:10:39.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 54017 6.04 MB/sec warmup 51 sec latency 23.346 ms 2024-08-20T01:10:39.718 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:39 smithi138 ceph-mon[18076]: pgmap v438: 129 pgs: 129 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 81 op/s 2024-08-20T01:10:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:39 smithi167 ceph-mon[21965]: pgmap v438: 129 pgs: 129 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 81 op/s 2024-08-20T01:10:39.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:39 smithi142 ceph-mon[21799]: pgmap v438: 129 pgs: 129 active+clean; 43 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 81 op/s 2024-08-20T01:10:40.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 56946 6.26 MB/sec warmup 52 sec latency 43.207 ms 2024-08-20T01:10:41.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 59361 6.32 MB/sec warmup 53 sec latency 21.386 ms 2024-08-20T01:10:41.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:41 smithi167 ceph-mon[21965]: pgmap v439: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 161 op/s 2024-08-20T01:10:41.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:41 smithi142 ceph-mon[21799]: pgmap v439: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 161 op/s 2024-08-20T01:10:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:41 smithi138 ceph-mon[18076]: pgmap v439: 129 pgs: 129 active+clean; 49 MiB data, 479 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.0 MiB/s wr, 161 op/s 2024-08-20T01:10:42.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 61754 6.50 MB/sec warmup 54 sec latency 27.641 ms 2024-08-20T01:10:43.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 64463 6.64 MB/sec warmup 55 sec latency 20.546 ms 2024-08-20T01:10:43.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:43 smithi167 ceph-mon[21965]: pgmap v440: 129 pgs: 129 active+clean; 51 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-08-20T01:10:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:43 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:10:43.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:43 smithi142 ceph-mon[21799]: pgmap v440: 129 pgs: 129 active+clean; 51 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-08-20T01:10:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:43 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:10:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:43 smithi138 ceph-mon[18076]: pgmap v440: 129 pgs: 129 active+clean; 51 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.3 MiB/s wr, 197 op/s 2024-08-20T01:10:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:43 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:10:44.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 67489 6.84 MB/sec warmup 56 sec latency 25.166 ms 2024-08-20T01:10:44.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:10:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:10:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:44 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:10:44.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:10:44.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:10:44.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:44 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:10:45.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:10:45.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:10:45.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:44 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:10:45.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 69009 6.83 MB/sec warmup 57 sec latency 40.973 ms 2024-08-20T01:10:45.891 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:45 smithi142 ceph-mon[21799]: pgmap v441: 129 pgs: 129 active+clean; 52 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-08-20T01:10:45.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:45 smithi167 ceph-mon[21965]: pgmap v441: 129 pgs: 129 active+clean; 52 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-08-20T01:10:46.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:45 smithi138 ceph-mon[18076]: pgmap v441: 129 pgs: 129 active+clean; 52 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.4 MiB/s wr, 209 op/s 2024-08-20T01:10:46.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 71396 6.95 MB/sec warmup 58 sec latency 11.039 ms 2024-08-20T01:10:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.555+0000 7f776a600640 1 -- 172.21.15.138:0/163773291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641029f0 msgr2=0x7f7764102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.555+0000 7f776a600640 1 --2- 172.21.15.138:0/163773291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641029f0 0x7f7764102dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7754009900 tx=0x7f775402a410 comp rx=0 tx=0).stop 2024-08-20T01:10:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 -- 172.21.15.138:0/163773291 shutdown_connections 2024-08-20T01:10:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 --2- 172.21.15.138:0/163773291 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 0x7f776410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 --2- 172.21.15.138:0/163773291 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641083f0 0x7f7764108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 --2- 172.21.15.138:0/163773291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641029f0 0x7f7764102dd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 -- 172.21.15.138:0/163773291 >> 172.21.15.138:0/163773291 conn(0x7f77640fe430 msgr2=0x7f7764100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 -- 172.21.15.138:0/163773291 shutdown_connections 2024-08-20T01:10:46.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.556+0000 7f776a600640 1 -- 172.21.15.138:0/163773291 wait complete. 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 Processor -- start 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 -- start start 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641029f0 0x7f776419b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 0x7f77641a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7764112a30 con 0x7f77641029f0 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77641128b0 con 0x7f77641083f0 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f776a600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7764112bb0 con 0x7f7764108d70 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f7763400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641029f0 0x7f776419b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f775be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f775be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39002/0 (socket says 172.21.15.138:39002) 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f775be00640 1 -- 172.21.15.138:0/3821155104 learned_addr learned my addr 172.21.15.138:0/3821155104 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.557+0000 7f7763e00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 0x7f77641a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 -- 172.21.15.138:0/3821155104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 msgr2=0x7f77641a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 0x7f77641a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 -- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641029f0 msgr2=0x7f776419b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641029f0 0x7f776419b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 -- 172.21.15.138:0/3821155104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77641a11f0 con 0x7f77641083f0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f775be00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f774c00ea60 tx=0x7f774c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f7760c00640 1 -- 172.21.15.138:0/3821155104 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c00cde0 con 0x7f77641083f0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f7760c00640 1 -- 172.21.15.138:0/3821155104 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774c0035f0 con 0x7f77641083f0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77641a14e0 con 0x7f77641083f0 2024-08-20T01:10:46.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f7760c00640 1 -- 172.21.15.138:0/3821155104 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c010690 con 0x7f77641083f0 2024-08-20T01:10:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.558+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77641a7ba0 con 0x7f77641083f0 2024-08-20T01:10:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.561+0000 7f776d5fa740 1 Processor -- start 2024-08-20T01:10:46.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 -- start start 2024-08-20T01:10:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7760c00640 1 -- 172.21.15.138:0/3821155104 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f774c0037a0 con 0x7f77641083f0 2024-08-20T01:10:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a5a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a66e80 0x55f029a69270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0299d0210 con 0x55f029a5a5d0 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f0299bf710 con 0x55f029a5aef0 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f776d5fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0299c3750 con 0x55f029a66e80 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7763e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a5a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7763400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7763400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39012/0 (socket says 172.21.15.138:39012) 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7763400640 1 -- 172.21.15.138:0/2810678634 learned_addr learned my addr 172.21.15.138:0/2810678634 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f7763e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a5a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53054/0 (socket says 172.21.15.138:53054) 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.562+0000 7f775be00640 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a66e80 0x55f029a69270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 -- 172.21.15.138:0/2810678634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a66e80 msgr2=0x55f029a69270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a66e80 0x55f029a69270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 -- 172.21.15.138:0/2810678634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 msgr2=0x55f029a5a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a5a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 -- 172.21.15.138:0/2810678634 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f02998bde0 con 0x55f029a5aef0 2024-08-20T01:10:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7763400640 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7754009900 tx=0x7f7754032bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3dfdf5d0e06f42c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7758c00640 1 -- 172.21.15.138:0/2810678634 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754040070 con 0x55f029a5aef0 2024-08-20T01:10:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7758c00640 1 -- 172.21.15.138:0/2810678634 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7754004030 con 0x55f029a5aef0 2024-08-20T01:10:46.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.563+0000 7f7758c00640 1 -- 172.21.15.138:0/2810678634 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7754035040 con 0x55f029a5aef0 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f7760c00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 0x7f77480b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 -- 172.21.15.138:0/2810678634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 msgr2=0x55f029a66940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7754009900 tx=0x7f7754032bd0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f7760c00640 1 -- 172.21.15.138:0/3821155104 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f774c0e4680 con 0x7f77641083f0 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f7763e00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 0x7f77480b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 -- 172.21.15.138:0/2810678634 shutdown_connections 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a66e80 0x55f029a69270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a5aef0 0x55f029a66940 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2810678634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a5a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 -- 172.21.15.138:0/2810678634 >> 172.21.15.138:0/2810678634 conn(0x55f029a583b0 msgr2=0x55f029a595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 -- 172.21.15.138:0/2810678634 shutdown_connections 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.564+0000 7f776d5fa740 1 -- 172.21.15.138:0/2810678634 wait complete. 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.565+0000 7f7763e00640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 0x7f77480b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f775c005fd0 tx=0x7f775c009600 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.565+0000 7f776d5fa740 1 -- start start 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.565+0000 7f776d5fa740 1 Processor -- start 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a66940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a64550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0298f8ab0 con 0x55f029a5aef0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f029966140 con 0x55f029a66e80 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f029a69b00 con 0x55f029a5a5d0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f776d5fa740 10 client.0 _ll_register_callbacks cb 0x55f0299795d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f775be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f775be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53060/0 (socket says 172.21.15.138:53060) 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f775be00640 1 -- 172.21.15.138:0/2358110832 learned_addr learned my addr 172.21.15.138:0/2358110832 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f7763e00640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a64550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.566+0000 7f7763400640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a66940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 -- 172.21.15.138:0/2358110832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 msgr2=0x55f029a66940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a66940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 -- 172.21.15.138:0/2358110832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 msgr2=0x55f029a63dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 -- 172.21.15.138:0/2358110832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f029967170 con 0x55f029a66e80 2024-08-20T01:10:46.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763400640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a66940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f775be00640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7763e00640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a64550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f775c011d30 tx=0x7f775c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d625df3a4830e290 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7751600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c004d20 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f775c003970 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.567+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f029996a30 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f775c031050 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 10 client.25832 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f029722790 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f029996a30 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f775c00f040 con 0x55f029a66e80 2024-08-20T01:10:46.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 10 client.25832 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 10 client.25832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 20 client.25832 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029a76dc0 0x55f029a791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 4 client.25832 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.568+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0299d2160 con 0x55f029a76dc0 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.569+0000 7f7763400640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029a76dc0 0x55f029a791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.569+0000 7f7763400640 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029a76dc0 0x55f029a791d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d56e23c90fae62 server_cookie=d6436bd5efaf1a21 in_seq=0 out_seq=0 2024-08-20T01:10:46.571 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.569+0000 7f7751600640 10 client.25832 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.570+0000 7f7751600640 1 -- 172.21.15.138:0/2358110832 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0299d2160 con 0x55f029a76dc0 2024-08-20T01:10:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.570+0000 7f7751600640 10 client.25832 handle_command_reply: tid=0 2024-08-20T01:10:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.570+0000 7f776d5fa740 1 client.25832 shutdown 2024-08-20T01:10:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 msgr2=0x55f029a64550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a64550 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f775c011d30 tx=0x7f775c004a40 comp rx=0 tx=0).stop 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 shutdown_connections 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029a76dc0 0x55f029a791d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a64550 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a63dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 --2- 172.21.15.138:0/2358110832 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a66940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f7751600640 0 client.25832 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 >> 172.21.15.138:0/2358110832 conn(0x55f029a583b0 msgr2=0x55f029a5dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 shutdown_connections 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 1 -- 172.21.15.138:0/2358110832 wait complete. 2024-08-20T01:10:46.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.571+0000 7f776d5fa740 20 client.25832 trim_cache size 0 max 16384 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 Processor -- start 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 -- start start 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77641b4580 con 0x55f029a5aef0 2024-08-20T01:10:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77641b1df0 con 0x55f029a66e80 2024-08-20T01:10:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f776d5fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77641afc40 con 0x55f029a5a5d0 2024-08-20T01:10:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f7763e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f775be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f7763e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39038/0 (socket says 172.21.15.138:39038) 2024-08-20T01:10:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.589+0000 7f7763e00640 1 -- 172.21.15.138:0/1922583524 learned_addr learned my addr 172.21.15.138:0/1922583524 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763400640 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 -- 172.21.15.138:0/1922583524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 msgr2=0x55f029a5e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a5e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 -- 172.21.15.138:0/1922583524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 msgr2=0x55f029a625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 -- 172.21.15.138:0/1922583524 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f029a64130 con 0x55f029a66e80 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f775be00640 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7763e00640 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f775c011980 tx=0x7f775c01ff40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa133d3a81e75e0e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7753e00640 1 -- 172.21.15.138:0/1922583524 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c004cb0 con 0x55f029a66e80 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7753e00640 1 -- 172.21.15.138:0/1922583524 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f775c024070 con 0x55f029a66e80 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.590+0000 7f7753e00640 1 -- 172.21.15.138:0/1922583524 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c028420 con 0x55f029a66e80 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- 172.21.15.138:0/1922583524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 msgr2=0x55f029a62b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f775c011980 tx=0x7f775c01ff40 comp rx=0 tx=0).stop 2024-08-20T01:10:46.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- 172.21.15.138:0/1922583524 shutdown_connections 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a62b20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5aef0 0x55f029a625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 --2- 172.21.15.138:0/1922583524 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5a5d0 0x55f029a5e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- 172.21.15.138:0/1922583524 >> 172.21.15.138:0/1922583524 conn(0x55f029a583b0 msgr2=0x55f029a5f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- 172.21.15.138:0/1922583524 shutdown_connections 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- 172.21.15.138:0/1922583524 wait complete. 2024-08-20T01:10:46.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.591+0000 7f776d5fa740 1 -- start start 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 Processor -- start 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a6f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5aef0 0x55f029a6f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f77641b02b0 con 0x55f029a5a5d0 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f77641b1720 con 0x55f029a66e80 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f77641b0980 con 0x55f029a5aef0 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f776d5fa740 10 client.0 _ll_register_callbacks cb 0x7f775c0119b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f7763e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f7763400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a6f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f7763e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39042/0 (socket says 172.21.15.138:39042) 2024-08-20T01:10:46.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f7763e00640 1 -- 172.21.15.138:0/4254001783 learned_addr learned my addr 172.21.15.138:0/4254001783 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.592+0000 7f775be00640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5aef0 0x55f029a6f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 -- 172.21.15.138:0/4254001783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5aef0 msgr2=0x55f029a6f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5aef0 0x55f029a6f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 -- 172.21.15.138:0/4254001783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 msgr2=0x55f029a6f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a6f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 -- 172.21.15.138:0/4254001783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f02979b620 con 0x55f029a66e80 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763400640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7763e00640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f775c020c50 tx=0x7f775c01fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2be362d20b4cb705 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7751600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c041070 con 0x55f029a66e80 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f775c020420 con 0x55f029a66e80 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f775c024070 con 0x55f029a66e80 2024-08-20T01:10:46.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.593+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f775c04d050 con 0x55f029a66e80 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 10 client.25838 fetch_fsmap learned FSMap version 29 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f029ae15e0 con 0x55f029a66e80 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f775c024070 con 0x55f029a66e80 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f775c00f070 con 0x55f029a66e80 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 10 client.25838 fetch_fsmap finished waiting for FSMap version 29 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 10 client.25838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 20 client.25838 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029ae1c50 0x55f029ae4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 4 client.25838 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0299cdc00 con 0x55f029ae1c50 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.594+0000 7f7763400640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029ae1c50 0x55f029ae4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.595+0000 7f7763400640 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029ae1c50 0x55f029ae4070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a249f99c441faec0 server_cookie=c710973c0832c806 in_seq=0 out_seq=0 2024-08-20T01:10:46.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.595+0000 7f7751600640 10 client.25838 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.676+0000 7f7751600640 1 -- 172.21.15.138:0/4254001783 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0299cdc00 con 0x55f029ae1c50 2024-08-20T01:10:46.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.676+0000 7f7751600640 10 client.25838 handle_command_reply: tid=0 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 client.25838 shutdown 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 msgr2=0x55f029a6a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f775c020c50 tx=0x7f775c01fd40 comp rx=0 tx=0).stop 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 shutdown_connections 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f029ae1c50 0x55f029ae4070 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f029a66e80 0x55f029a6a850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f029a5aef0 0x55f029a6f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f7751600640 0 client.25838 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 --2- 172.21.15.138:0/4254001783 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f029a5a5d0 0x55f029a6f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 >> 172.21.15.138:0/4254001783 conn(0x55f029a583b0 msgr2=0x55f029a62560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 shutdown_connections 2024-08-20T01:10:46.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 1 -- 172.21.15.138:0/4254001783 wait complete. 2024-08-20T01:10:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.677+0000 7f776d5fa740 20 client.25838 trim_cache size 0 max 16384 2024-08-20T01:10:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:10:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:10:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:10:46.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 msgr2=0x7f77480b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 0x7f77480b6200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f775c005fd0 tx=0x7f775c009600 comp rx=0 tx=0).stop 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 msgr2=0x7f776419b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f774c00ea60 tx=0x7f774c00ef20 comp rx=0 tx=0).stop 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 shutdown_connections 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77480b3d40 0x7f77480b6200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7764108d70 0x7f77641a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77641083f0 0x7f776419b690 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 --2- 172.21.15.138:0/3821155104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77641029f0 0x7f776419b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 >> 172.21.15.138:0/3821155104 conn(0x7f77640fe430 msgr2=0x7f77641049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 shutdown_connections 2024-08-20T01:10:46.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:46.678+0000 7f776a600640 1 -- 172.21.15.138:0/3821155104 wait complete. 2024-08-20T01:10:47.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 73902 7.04 MB/sec warmup 59 sec latency 25.609 ms 2024-08-20T01:10:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:47 smithi167 ceph-mon[21965]: pgmap v442: 129 pgs: 129 active+clean; 55 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-08-20T01:10:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:47 smithi142 ceph-mon[21799]: pgmap v442: 129 pgs: 129 active+clean; 55 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-08-20T01:10:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:47 smithi138 ceph-mon[18076]: pgmap v442: 129 pgs: 129 active+clean; 55 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 1.7 MiB/s wr, 243 op/s 2024-08-20T01:10:48.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 75467 7.06 MB/sec warmup 60 sec latency 15.541 ms 2024-08-20T01:10:48.909 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:48 smithi138 ceph-mon[18076]: pgmap v443: 129 pgs: 129 active+clean; 57 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-08-20T01:10:48.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:48 smithi167 ceph-mon[21965]: pgmap v443: 129 pgs: 129 active+clean; 57 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-08-20T01:10:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:48 smithi142 ceph-mon[21799]: pgmap v443: 129 pgs: 129 active+clean; 57 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.5 MiB/s wr, 215 op/s 2024-08-20T01:10:49.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 77126 7.06 MB/sec warmup 61 sec latency 380.512 ms 2024-08-20T01:10:50.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 78558 7.12 MB/sec warmup 62 sec latency 545.722 ms 2024-08-20T01:10:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:50 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:50.966 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:50 smithi167 ceph-mon[21965]: pgmap v444: 129 pgs: 129 active+clean; 60 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-08-20T01:10:50.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:50 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:50 smithi142 ceph-mon[21799]: pgmap v444: 129 pgs: 129 active+clean; 60 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-08-20T01:10:51.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:50 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:10:51.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:50 smithi138 ceph-mon[18076]: pgmap v444: 129 pgs: 129 active+clean; 60 MiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.8 MiB/s wr, 266 op/s 2024-08-20T01:10:51.123 INFO:tasks.workunit.client.0.smithi138.stdout: 1 80852 7.19 MB/sec warmup 63 sec latency 17.742 ms 2024-08-20T01:10:51.459 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.4 2024-08-20T01:10:51.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:10:51.906 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:52.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 82548 7.21 MB/sec warmup 64 sec latency 18.458 ms 2024-08-20T01:10:52.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 -- 172.21.15.138:0/4074440418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe240101d40 msgr2=0x7fe24010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 --2- 172.21.15.138:0/4074440418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe240101d40 0x7fe24010ea90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009900 tx=0x7fe22c02a480 comp rx=0 tx=0).stop 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 -- 172.21.15.138:0/4074440418 shutdown_connections 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 --2- 172.21.15.138:0/4074440418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe24010efd0 0x7fe2401113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 --2- 172.21.15.138:0/4074440418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe240101d40 0x7fe24010ea90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 --2- 172.21.15.138:0/4074440418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240101420 0x7fe240101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 -- 172.21.15.138:0/4074440418 >> 172.21.15.138:0/4074440418 conn(0x7fe2400fd2d0 msgr2=0x7fe2400ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.634+0000 7fe23f200640 1 -- 172.21.15.138:0/4074440418 shutdown_connections 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.635+0000 7fe23f200640 1 -- 172.21.15.138:0/4074440418 wait complete. 2024-08-20T01:10:52.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 Processor -- start 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 -- start start 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240101d40 0x7fe24010b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 0x7fe24010be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe240114b30 con 0x7fe240101420 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2401149b0 con 0x7fe24010efd0 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe240114cb0 con 0x7fe240101d40 2024-08-20T01:10:52.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.636+0000 7fe23de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34694/0 (socket says 172.21.15.138:34694) 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 -- 172.21.15.138:0/3688669753 learned_addr learned my addr 172.21.15.138:0/3688669753 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23e800640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 0x7fe24010be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 -- 172.21.15.138:0/3688669753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240101d40 msgr2=0x7fe24010b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240101d40 0x7fe24010b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 -- 172.21.15.138:0/3688669753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 msgr2=0x7fe24010be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 0x7fe24010be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 -- 172.21.15.138:0/3688669753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe24010c660 con 0x7fe240101420 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23e800640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 0x7fe24010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.637+0000 7fe23de00640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe23000e9f0 tx=0x7fe23000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.638+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe23000cd80 con 0x7fe240101420 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.638+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2300054d0 con 0x7fe240101420 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.638+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2401ab850 con 0x7fe240101420 2024-08-20T01:10:52.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.638+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe230010690 con 0x7fe240101420 2024-08-20T01:10:52.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.638+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2401abd90 con 0x7fe240101420 2024-08-20T01:10:52.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.639+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe240101800 con 0x7fe240101420 2024-08-20T01:10:52.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.642+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe230004e30 con 0x7fe240101420 2024-08-20T01:10:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.643+0000 7fe236a00640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 0x7fe2240b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:52.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.643+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe2300e49f0 con 0x7fe240101420 2024-08-20T01:10:52.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.643+0000 7fe23d400640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 0x7fe2240b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:52.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.644+0000 7fe23d400640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 0x7fe2240b6320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02a9a0 tx=0x7fe22c031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:52.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.646+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe230055e20 con 0x7fe240101420 2024-08-20T01:10:52.917 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.919+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe2401ac1f0 con 0x7fe240101420 2024-08-20T01:10:52.918 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.920+0000 7fe236a00640 1 -- 172.21.15.138:0/3688669753 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11936 (secure 0 0 0) 0x7fe2300ab950 con 0x7fe240101420 2024-08-20T01:10:52.919 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:10:52.919 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":30,"btime":"2024-08-20T01:10:49:648211+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:10:49.219429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:10:52.919 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 30 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.922+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 msgr2=0x7fe2240b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.922+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 0x7fe2240b6320 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe22c02a9a0 tx=0x7fe22c031400 comp rx=0 tx=0).stop 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 msgr2=0x7fe24010e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe23000e9f0 tx=0x7fe23000edf0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 shutdown_connections 2024-08-20T01:10:52.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2240b3e60 0x7fe2240b6320 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24010efd0 0x7fe24010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe240101d40 0x7fe24010b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 --2- 172.21.15.138:0/3688669753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe240101420 0x7fe24010e700 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 >> 172.21.15.138:0/3688669753 conn(0x7fe2400fd2d0 msgr2=0x7fe24010fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 shutdown_connections 2024-08-20T01:10:52.922 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:52.923+0000 7fe23f200640 1 -- 172.21.15.138:0/3688669753 wait complete. 2024-08-20T01:10:53.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 85422 7.38 MB/sec warmup 65 sec latency 11.435 ms 2024-08-20T01:10:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:53 smithi138 ceph-mon[18076]: pgmap v445: 129 pgs: 129 active+clean; 63 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-08-20T01:10:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:53 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3688669753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:53 smithi167 ceph-mon[21965]: pgmap v445: 129 pgs: 129 active+clean; 63 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-08-20T01:10:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:53 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3688669753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:53 smithi142 ceph-mon[21799]: pgmap v445: 129 pgs: 129 active+clean; 63 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.7 MiB/s wr, 259 op/s 2024-08-20T01:10:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:53 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3688669753' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:10:54.109 INFO:tasks.workunit.client.0.smithi138.stdout: 1 86297 7.30 MB/sec warmup 66 sec latency 233.990 ms 2024-08-20T01:10:54.253 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0' 2024-08-20T01:10:54.254 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 136.4 --await-for 45.0 2024-08-20T01:10:54.642 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:10:55.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 87050 7.24 MB/sec warmup 67 sec latency 832.967 ms 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 -- 172.21.15.138:0/929197733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 msgr2=0x7fda3c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 --2- 172.21.15.138:0/929197733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c10a960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fda28009900 tx=0x7fda2802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 -- 172.21.15.138:0/929197733 shutdown_connections 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 --2- 172.21.15.138:0/929197733 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c10aea0 0x7fda3c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 --2- 172.21.15.138:0/929197733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c10a960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 --2- 172.21.15.138:0/929197733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c104b10 0x7fda3c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.319 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.319+0000 7fda41200640 1 -- 172.21.15.138:0/929197733 >> 172.21.15.138:0/929197733 conn(0x7fda3c100540 msgr2=0x7fda3c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.320+0000 7fda41200640 1 -- 172.21.15.138:0/929197733 shutdown_connections 2024-08-20T01:10:55.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.320+0000 7fda41200640 1 -- 172.21.15.138:0/929197733 wait complete. 2024-08-20T01:10:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.321+0000 7fda41200640 1 Processor -- start 2024-08-20T01:10:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.321+0000 7fda41200640 1 -- start start 2024-08-20T01:10:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c10aea0 0x7fda3c1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c114b40 con 0x7fda3c10aea0 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1149c0 con 0x7fda3c10a520 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda41200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c114cc0 con 0x7fda3c104b10 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c10aea0 0x7fda3c1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:34370/0 (socket says 172.21.15.138:34370) 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 -- 172.21.15.138:0/2334008401 learned_addr learned my addr 172.21.15.138:0/2334008401 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:55.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 -- 172.21.15.138:0/2334008401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 msgr2=0x7fda3c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 -- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c10aea0 msgr2=0x7fda3c1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c10aea0 0x7fda3c1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.322+0000 7fda3b400640 1 -- 172.21.15.138:0/2334008401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda3c1a0fa0 con 0x7fda3c104b10 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.323+0000 7fda3b400640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e9f0 tx=0x7fda2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.323+0000 7fda33e00640 1 -- 172.21.15.138:0/2334008401 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00cd80 con 0x7fda3c104b10 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.323+0000 7fda33e00640 1 -- 172.21.15.138:0/2334008401 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda2c0108e0 con 0x7fda3c104b10 2024-08-20T01:10:55.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.323+0000 7fda33e00640 1 -- 172.21.15.138:0/2334008401 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c018980 con 0x7fda3c104b10 2024-08-20T01:10:55.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.324+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda3c1a1290 con 0x7fda3c104b10 2024-08-20T01:10:55.324 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.324+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda3c1a7950 con 0x7fda3c104b10 2024-08-20T01:10:55.325 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 Processor -- start 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda33e00640 1 -- 172.21.15.138:0/2334008401 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fda2c010430 con 0x7fda3c104b10 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 -- start start 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 0x55ffa034d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034def0 0x55ffa0362150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.327+0000 7fda45201740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffa02ed670 con 0x55ffa034def0 2024-08-20T01:10:55.326 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda45201740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ffa02c17c0 con 0x55ffa034d5d0 2024-08-20T01:10:55.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda45201740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ffa02b0cc0 con 0x55ffa0362690 2024-08-20T01:10:55.327 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034def0 0x55ffa0362150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:34376/0 (socket says 172.21.15.138:34376) 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3aa00640 1 -- 172.21.15.138:0/4158301745 learned_addr learned my addr 172.21.15.138:0/4158301745 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 0x55ffa034d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 -- 172.21.15.138:0/4158301745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 msgr2=0x55ffa0364a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 -- 172.21.15.138:0/4158301745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034def0 msgr2=0x55ffa0362150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034def0 0x55ffa0362150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3be00640 1 -- 172.21.15.138:0/4158301745 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa0244110 con 0x55ffa034d5d0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.328+0000 7fda3aa00640 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.329+0000 7fda3be00640 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 0x55ffa034d9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fda34009f00 tx=0x7fda34011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc067595e80d318a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.329+0000 7fda30c00640 1 -- 172.21.15.138:0/4158301745 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34004d20 con 0x55ffa034d5d0 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.329+0000 7fda33e00640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 0x7fda200b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.328 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.329+0000 7fda30c00640 1 -- 172.21.15.138:0/4158301745 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda34004030 con 0x55ffa034d5d0 2024-08-20T01:10:55.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.329+0000 7fda30c00640 1 -- 172.21.15.138:0/4158301745 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda34005280 con 0x55ffa034d5d0 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.330+0000 7fda45201740 1 -- 172.21.15.138:0/4158301745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 msgr2=0x55ffa034d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.330+0000 7fda45201740 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 0x55ffa034d9b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fda34009f00 tx=0x7fda34011810 comp rx=0 tx=0).stop 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.330+0000 7fda3b400640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 0x7fda200b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.330+0000 7fda33e00640 1 -- 172.21.15.138:0/2334008401 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fda2c0e48f0 con 0x7fda3c104b10 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 -- 172.21.15.138:0/4158301745 shutdown_connections 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa0362690 0x55ffa0364a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034def0 0x55ffa0362150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 --2- 172.21.15.138:0/4158301745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034d5d0 0x55ffa034d9b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 -- 172.21.15.138:0/4158301745 >> 172.21.15.138:0/4158301745 conn(0x55ffa034b400 msgr2=0x55ffa034c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 -- 172.21.15.138:0/4158301745 shutdown_connections 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 -- 172.21.15.138:0/4158301745 wait complete. 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda3b400640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 0x7fda200b6370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ffa034c2e0 tx=0x7fda2c0ea530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.331+0000 7fda45201740 1 -- start start 2024-08-20T01:10:55.331 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 Processor -- start 2024-08-20T01:10:55.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa03575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa0357b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ffa0256780 con 0x55ffa0362690 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ffa02b4d00 con 0x55ffa034def0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ffa01137a0 con 0x55ffa034d5d0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda45201740 10 client.0 _ll_register_callbacks cb 0x55ffa025e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45784/0 (socket says 172.21.15.138:45784) 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.332+0000 7fda3aa00640 1 -- 172.21.15.138:0/1605704996 learned_addr learned my addr 172.21.15.138:0/1605704996 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3be00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa03575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3be00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa0357b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 -- 172.21.15.138:0/1605704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 msgr2=0x55ffa03575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa03575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 -- 172.21.15.138:0/1605704996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 msgr2=0x55ffa0357b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa0357b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 -- 172.21.15.138:0/1605704996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa02639c0 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3be00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa0357b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda3aa00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fda2802a9e0 tx=0x7fda28004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c8b17e7cdacafb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda25600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28035040 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fda28005450 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.333+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda34004030 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda28005650 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 10 client.25856 fetch_fsmap learned FSMap version 30 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffa001fb90 con 0x55ffa034def0 2024-08-20T01:10:55.334 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda34004030 con 0x55ffa034def0 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fda28003b50 con 0x55ffa034def0 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 10 client.25856 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 10 client.25856 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 20 client.25856 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa037a9d0 0x55ffa037cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 4 client.25856 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa02c3710 con 0x55ffa037a9d0 2024-08-20T01:10:55.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.334+0000 7fda3be00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa037a9d0 0x55ffa037cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.335+0000 7fda3be00640 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa037a9d0 0x55ffa037cdc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3902e838f7e7812 server_cookie=3890fa1189dffae1 in_seq=0 out_seq=0 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.335+0000 7fda25600640 10 client.25856 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda25600640 1 -- 172.21.15.138:0/1605704996 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffa02c3710 con 0x55ffa037a9d0 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda25600640 10 client.25856 handle_command_reply: tid=0 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 client.25856 shutdown 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 msgr2=0x55ffa0361e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fda2802a9e0 tx=0x7fda28004420 comp rx=0 tx=0).stop 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 shutdown_connections 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa037a9d0 0x55ffa037cdc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa0357b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa0361e10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 --2- 172.21.15.138:0/1605704996 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa03575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda25600640 0 client.25856 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 >> 172.21.15.138:0/1605704996 conn(0x55ffa034b400 msgr2=0x55ffa0350f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.336+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 shutdown_connections 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.337+0000 7fda45201740 1 -- 172.21.15.138:0/1605704996 wait complete. 2024-08-20T01:10:55.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.337+0000 7fda45201740 20 client.25856 trim_cache size 0 max 16384 2024-08-20T01:10:55.354 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 Processor -- start 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 -- start start 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034d5d0 0x55ffa0354e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa0362690 0x55ffa034ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1afe90 con 0x55ffa034d5d0 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ad700 con 0x55ffa0362690 2024-08-20T01:10:55.355 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.356+0000 7fda45201740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ab590 con 0x55ffa034def0 2024-08-20T01:10:55.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:34394/0 (socket says 172.21.15.138:34394) 2024-08-20T01:10:55.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 -- 172.21.15.138:0/733459943 learned_addr learned my addr 172.21.15.138:0/733459943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:55.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3be00640 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034d5d0 0x55ffa0354e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3be00640 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa0362690 0x55ffa034ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 -- 172.21.15.138:0/733459943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa0362690 msgr2=0x55ffa034ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa0362690 0x55ffa034ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 -- 172.21.15.138:0/733459943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034d5d0 msgr2=0x55ffa0354e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034d5d0 0x55ffa0354e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 -- 172.21.15.138:0/733459943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa034c100 con 0x55ffa034def0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.357+0000 7fda3aa00640 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda28004650 tx=0x7fda280040b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00575386c9331b3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.358+0000 7fda27e00640 1 -- 172.21.15.138:0/733459943 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28059070 con 0x55ffa034def0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.358+0000 7fda27e00640 1 -- 172.21.15.138:0/733459943 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28041070 con 0x55ffa034def0 2024-08-20T01:10:55.357 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.358+0000 7fda27e00640 1 -- 172.21.15.138:0/733459943 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28035030 con 0x55ffa034def0 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 -- 172.21.15.138:0/733459943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 msgr2=0x55ffa035caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fda28004650 tx=0x7fda280040b0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 -- 172.21.15.138:0/733459943 shutdown_connections 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa0362690 0x55ffa034ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034def0 0x55ffa035caf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 --2- 172.21.15.138:0/733459943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa034d5d0 0x55ffa0354e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 -- 172.21.15.138:0/733459943 >> 172.21.15.138:0/733459943 conn(0x55ffa034b400 msgr2=0x55ffa034c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 -- 172.21.15.138:0/733459943 shutdown_connections 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.359+0000 7fda45201740 1 -- 172.21.15.138:0/733459943 wait complete. 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 -- start start 2024-08-20T01:10:55.359 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 Processor -- start 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa038e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa038b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1abbe0 con 0x55ffa0362690 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ad030 con 0x55ffa034def0 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fda3c1ac290 con 0x55ffa034d5d0 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.360+0000 7fda45201740 10 client.0 _ll_register_callbacks cb 0x7fda28009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45808/0 (socket says 172.21.15.138:45808) 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 -- 172.21.15.138:0/2417756884 learned_addr learned my addr 172.21.15.138:0/2417756884 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:55.360 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3be00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa038e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 -- 172.21.15.138:0/2417756884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 msgr2=0x55ffa038e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa038e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 -- 172.21.15.138:0/2417756884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 msgr2=0x55ffa038b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa038b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 -- 172.21.15.138:0/2417756884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa038b950 con 0x55ffa034def0 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3be00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa038e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda3aa00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda28031440 tx=0x7fda28031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc4e149303331ea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda25600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.361+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda28059040 con 0x55ffa034def0 2024-08-20T01:10:55.362 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fda280035f0 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda28041070 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fda2802aa70 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 10 client.25862 fetch_fsmap learned FSMap version 30 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55ffa03e5d70 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda28041070 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fda2805c030 con 0x55ffa034def0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 10 client.25862 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 10 client.25862 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.362+0000 7fda45201740 20 client.25862 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa03e6440 0x55ffa03e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda45201740 4 client.25862 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}] 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.4, "await_for": 45.0}) -- 0x55ffa02bf1b0 con 0x55ffa03e6440 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda3be00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa03e6440 0x55ffa03e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:55.363 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda3be00640 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa03e6440 0x55ffa03e8860 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bba0de538e6e01f server_cookie=78dfcee17790f5ea in_seq=0 out_seq=0 2024-08-20T01:10:55.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.363+0000 7fda25600640 10 client.25862 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:55.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda25600640 1 -- 172.21.15.138:0/2417756884 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ffa02bf1b0 con 0x55ffa03e6440 2024-08-20T01:10:55.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda25600640 10 client.25862 handle_command_reply: tid=0 2024-08-20T01:10:55.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 client.25862 shutdown 2024-08-20T01:10:55.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 msgr2=0x55ffa038e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fda28031440 tx=0x7fda28031770 comp rx=0 tx=0).stop 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 shutdown_connections 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ffa03e6440 0x55ffa03e8860 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ffa0362690 0x55ffa038b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ffa034def0 0x55ffa038e900 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 --2- 172.21.15.138:0/2417756884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ffa034d5d0 0x55ffa038e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda25600640 0 client.25862 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 >> 172.21.15.138:0/2417756884 conn(0x55ffa034b400 msgr2=0x55ffa035aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 shutdown_connections 2024-08-20T01:10:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.558+0000 7fda45201740 1 -- 172.21.15.138:0/2417756884 wait complete. 2024-08-20T01:10:55.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda45201740 20 client.25862 trim_cache size 0 max 16384 2024-08-20T01:10:55.561 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:10:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 30, 2024-08-20T01:10:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:10:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 14, 2024-08-20T01:10:55.561 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "f90e1cc8": { 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 14, 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:10:55.562 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 136.4, 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:10:55.563 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:55.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:55.565 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:10:55.565 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:10:55.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 msgr2=0x7fda200b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 0x7fda200b6370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55ffa034c2e0 tx=0x7fda2c0ea530 comp rx=0 tx=0).stop 2024-08-20T01:10:55.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 msgr2=0x7fda3c19aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:55.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00e9f0 tx=0x7fda2c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 shutdown_connections 2024-08-20T01:10:55.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fda200b3eb0 0x7fda200b6370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda3c10aea0 0x7fda3c1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda3c10a520 0x7fda3c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 --2- 172.21.15.138:0/2334008401 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fda3c104b10 0x7fda3c19aef0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.559+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 >> 172.21.15.138:0/2334008401 conn(0x7fda3c100540 msgr2=0x7fda3c102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.560+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 shutdown_connections 2024-08-20T01:10:55.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:10:55.560+0000 7fda41200640 1 -- 172.21.15.138:0/2334008401 wait complete. 2024-08-20T01:10:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:55 smithi167 ceph-mon[21965]: pgmap v446: 129 pgs: 129 active+clean; 64 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-08-20T01:10:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:55 smithi142 ceph-mon[21799]: pgmap v446: 129 pgs: 129 active+clean; 64 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-08-20T01:10:56.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:55 smithi138 ceph-mon[18076]: pgmap v446: 129 pgs: 129 active+clean; 64 MiB data, 552 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-08-20T01:10:56.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 7.24 MB/sec warmup 68 sec latency 679.286 ms 2024-08-20T01:10:56.556 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f90e1cc8', quiesce took 5.1 seconds. Will release after: 11.299999999999999 2024-08-20T01:10:57.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 7.14 MB/sec warmup 69 sec latency 1679.405 ms 2024-08-20T01:10:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:57 smithi167 ceph-mon[21965]: pgmap v447: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-08-20T01:10:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:57 smithi142 ceph-mon[21799]: pgmap v447: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-08-20T01:10:58.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:57 smithi138 ceph-mon[18076]: pgmap v447: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-08-20T01:10:58.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 7.03 MB/sec warmup 70 sec latency 2679.536 ms 2024-08-20T01:10:59.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.94 MB/sec warmup 71 sec latency 3679.660 ms 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 -- 172.21.15.138:0/3939750166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 msgr2=0x7f5318104ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 --2- 172.21.15.138:0/3939750166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f5318104ff0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5308009900 tx=0x7f530802a410 comp rx=0 tx=0).stop 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 -- 172.21.15.138:0/3939750166 shutdown_connections 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 --2- 172.21.15.138:0/3939750166 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 0x7f5318111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 --2- 172.21.15.138:0/3939750166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 0x7f531810aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 --2- 172.21.15.138:0/3939750166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f5318104ff0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.800 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 -- 172.21.15.138:0/3939750166 >> 172.21.15.138:0/3939750166 conn(0x7f5318100610 msgr2=0x7f5318102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.801+0000 7f531f000640 1 -- 172.21.15.138:0/3939750166 shutdown_connections 2024-08-20T01:10:59.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.802+0000 7f531f000640 1 -- 172.21.15.138:0/3939750166 wait complete. 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.802+0000 7f531f000640 1 Processor -- start 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 -- start start 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 0x7f531819b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 0x7f53181a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5318114c70 con 0x7f5318104c10 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5318114af0 con 0x7f531810a610 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f531f000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5318114df0 con 0x7f531810af90 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f5317e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f5317e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56116/0 (socket says 172.21.15.138:56116) 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f5317e00640 1 -- 172.21.15.138:0/2703957632 learned_addr learned my addr 172.21.15.138:0/2703957632 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.803+0000 7f530f400640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 0x7f531819b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f531d000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 0x7f53181a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 -- 172.21.15.138:0/2703957632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 msgr2=0x7f53181a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 0x7f53181a0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 -- 172.21.15.138:0/2703957632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 msgr2=0x7f531819b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 0x7f531819b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 -- 172.21.15.138:0/2703957632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53181a11e0 con 0x7f5318104c10 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5317e00640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5308005ab0 tx=0x7f5308003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5315600640 1 -- 172.21.15.138:0/2703957632 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530803f070 con 0x7f5318104c10 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53181a1470 con 0x7f5318104c10 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5315600640 1 -- 172.21.15.138:0/2703957632 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5308031b60 con 0x7f5318104c10 2024-08-20T01:10:59.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f5315600640 1 -- 172.21.15.138:0/2703957632 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308035030 con 0x7f5318104c10 2024-08-20T01:10:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.804+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53181a7b90 con 0x7f5318104c10 2024-08-20T01:10:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.807+0000 7f5321e9e740 1 Processor -- start 2024-08-20T01:10:59.805 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.807+0000 7f5321e9e740 1 -- start start 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fbcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fc8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fcb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c161f32210 con 0x55c161fc91e0 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c161f21710 con 0x55c161fbc930 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5321e9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c161f25750 con 0x55c161fbd250 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5315600640 1 -- 172.21.15.138:0/2703957632 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5308042070 con 0x7f5318104c10 2024-08-20T01:10:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f530f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fcb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fbcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f530f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fcb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56132/0 (socket says 172.21.15.138:56132) 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f530f400640 1 -- 172.21.15.138:0/4104569824 learned_addr learned my addr 172.21.15.138:0/4104569824 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5317e00640 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fc8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 -- 172.21.15.138:0/4104569824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 msgr2=0x55c161fc8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fc8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 -- 172.21.15.138:0/4104569824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 msgr2=0x55c161fcb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fcb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f531d000640 1 -- 172.21.15.138:0/4104569824 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c161eedde0 con 0x55c161fbc930 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.808+0000 7f5317e00640 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fc8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.809+0000 7f531d000640 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fbcd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5310009ed0 tx=0x7f53100107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16716b7a86fce6a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.809+0000 7f530d600640 1 -- 172.21.15.138:0/4104569824 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310003b70 con 0x55c161fbc930 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.809+0000 7f530d600640 1 -- 172.21.15.138:0/4104569824 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310015070 con 0x55c161fbc930 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.809+0000 7f530d600640 1 -- 172.21.15.138:0/4104569824 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310011730 con 0x55c161fbc930 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.809+0000 7f5315600640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 0x7f52fc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- 172.21.15.138:0/4104569824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 msgr2=0x55c161fbcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fbcd10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5310009ed0 tx=0x7f53100107a0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5315600640 1 -- 172.21.15.138:0/2703957632 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5308104b20 con 0x7f5318104c10 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5317e00640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 0x7f52fc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- 172.21.15.138:0/4104569824 shutdown_connections 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fcb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fc8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 --2- 172.21.15.138:0/4104569824 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fbcd10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- 172.21.15.138:0/4104569824 >> 172.21.15.138:0/4104569824 conn(0x55c161fba760 msgr2=0x55c161fbb900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- 172.21.15.138:0/4104569824 shutdown_connections 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- 172.21.15.138:0/4104569824 wait complete. 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5317e00640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 0x7f52fc0b6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55c161fbfab0 tx=0x7f5308031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.810+0000 7f5321e9e740 1 -- start start 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 Processor -- start 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbc930 0x55c161fc89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fc6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c161e5aab0 con 0x55c161fc91e0 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c161ec8140 con 0x55c161fbd250 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c161fcbec0 con 0x55c161fbc930 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f5321e9e740 10 client.0 _ll_register_callbacks cb 0x55c161edb5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f530f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f531d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbc930 0x55c161fc89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f530f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:58366/0 (socket says 172.21.15.138:58366) 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f530f400640 1 -- 172.21.15.138:0/2675433213 learned_addr learned my addr 172.21.15.138:0/2675433213 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.811+0000 7f531d000640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fc6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 -- 172.21.15.138:0/2675433213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbc930 msgr2=0x55c161fc89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbc930 0x55c161fc89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 -- 172.21.15.138:0/2675433213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 msgr2=0x55c161fc6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fc6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 -- 172.21.15.138:0/2675433213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c161ec9170 con 0x55c161fbd250 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f530f400640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5300005bc0 tx=0x7f5300012bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8713ca12ac2236f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f5306000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:10:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530000f920 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5310015070 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.812+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f530000fac0 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f530000fcc0 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 10 client.25874 fetch_fsmap learned FSMap version 30 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c161c84790 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310015070 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f530001c420 con 0x55c161fbd250 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 10 client.25874 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 20 client.25874 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c161fd8ed0 0x55c161fdb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 4 client.25874 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:10:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.813+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c161f34160 con 0x55c161fd8ed0 2024-08-20T01:10:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.814+0000 7f531d000640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c161fd8ed0 0x55c161fdb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.814+0000 7f531d000640 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c161fd8ed0 0x55c161fdb2c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9406e1cc53171a server_cookie=1508dda29dfa8fb3 in_seq=0 out_seq=0 2024-08-20T01:10:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.814+0000 7f5306000640 10 client.25874 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5306000640 1 -- 172.21.15.138:0/2675433213 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c161f34160 con 0x55c161fd8ed0 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5306000640 10 client.25874 handle_command_reply: tid=0 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 client.25874 shutdown 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 msgr2=0x55c161fc5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5300005bc0 tx=0x7f5300012bd0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 shutdown_connections 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c161fd8ed0 0x55c161fdb2c0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fc6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5306000640 0 client.25874 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbd250 0x55c161fc5e30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 --2- 172.21.15.138:0/2675433213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbc930 0x55c161fc89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 >> 172.21.15.138:0/2675433213 conn(0x55c161fba760 msgr2=0x55c161fc0300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 shutdown_connections 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 1 -- 172.21.15.138:0/2675433213 wait complete. 2024-08-20T01:10:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.815+0000 7f5321e9e740 20 client.25874 trim_cache size 0 max 16384 2024-08-20T01:10:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 Processor -- start 2024-08-20T01:10:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 -- start start 2024-08-20T01:10:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fe0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fe10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b4530 con 0x55c161fc91e0 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b1da0 con 0x55c161fbc930 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f5321e9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53181afbf0 con 0x55c161fbd250 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f531d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fe10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f531d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f531d000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:58380/0 (socket says 172.21.15.138:58380) 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f530f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fe0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.831+0000 7f531d000640 1 -- 172.21.15.138:0/3380504937 learned_addr learned my addr 172.21.15.138:0/3380504937 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 -- 172.21.15.138:0/3380504937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 msgr2=0x55c161fe0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fe0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 -- 172.21.15.138:0/3380504937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 msgr2=0x55c161fe10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fe10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 -- 172.21.15.138:0/3380504937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c161cfd620 con 0x55c161fbc930 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f531d000640 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5310010ca0 tx=0x7f53100107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57d109625a3b7ea6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f530cc00640 1 -- 172.21.15.138:0/3380504937 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310025040 con 0x55c161fbc930 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f530cc00640 1 -- 172.21.15.138:0/3380504937 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310015070 con 0x55c161fbc930 2024-08-20T01:10:59.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.832+0000 7f530cc00640 1 -- 172.21.15.138:0/3380504937 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5310011c50 con 0x55c161fbc930 2024-08-20T01:10:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- 172.21.15.138:0/3380504937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 msgr2=0x55c161fc1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5310010ca0 tx=0x7f53100107a0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- 172.21.15.138:0/3380504937 shutdown_connections 2024-08-20T01:10:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fc91e0 0x55c161fe10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fbd250 0x55c161fe0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3380504937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fc1360 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- 172.21.15.138:0/3380504937 >> 172.21.15.138:0/3380504937 conn(0x55c161fba760 msgr2=0x55c161fc0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- 172.21.15.138:0/3380504937 shutdown_connections 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- 172.21.15.138:0/3380504937 wait complete. 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 -- start start 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.833+0000 7f5321e9e740 1 Processor -- start 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fd1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 0x55c161fcd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b0260 con 0x55c161fbd250 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b16d0 con 0x55c161fbc930 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b0930 con 0x55c161fc91e0 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f5321e9e740 10 client.0 _ll_register_callbacks cb 0x7f5300009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56158/0 (socket says 172.21.15.138:56158) 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 -- 172.21.15.138:0/3371879122 learned_addr learned my addr 172.21.15.138:0/3371879122 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f531d000640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fd1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f531d000640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 0x55c161fcd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 -- 172.21.15.138:0/3371879122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 msgr2=0x55c161fcd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 0x55c161fcd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 -- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 msgr2=0x55c161fd1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fd1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.834+0000 7f530f400640 1 -- 172.21.15.138:0/3371879122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c161fcd6f0 con 0x55c161fbd250 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f531d000640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 0x55c161fcd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f530f400640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f530001ca80 tx=0x7f530001c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e78d5b5c73d4f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5306000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530002f060 con 0x55c161fbd250 2024-08-20T01:10:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f530001cd10 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5310015070 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f530000fe60 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5321e9e740 10 client.15828 fetch_fsmap learned FSMap version 30 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c16204c220 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.835+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5310015070 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f5300003930 con 0x55c161fbd250 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 10 client.15828 fetch_fsmap finished waiting for FSMap version 30 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 10 client.15828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 20 client.15828 populate_metadata read hostname 'smithi138' 2024-08-20T01:10:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c16204ca00 0x55c16204ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:10:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 4 client.15828 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:10:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c161f2fc00 con 0x55c16204ca00 2024-08-20T01:10:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.836+0000 7f531d000640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c16204ca00 0x55c16204ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:10:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.837+0000 7f531d000640 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c16204ca00 0x55c16204ee20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e34571023b9d29 server_cookie=28d73878701ca986 in_seq=0 out_seq=0 2024-08-20T01:10:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.837+0000 7f5306000640 10 client.15828 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.886+0000 7f5306000640 1 -- 172.21.15.138:0/3371879122 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c161f2fc00 con 0x55c16204ca00 2024-08-20T01:10:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.886+0000 7f5306000640 10 client.15828 handle_command_reply: tid=0 2024-08-20T01:10:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.886+0000 7f5321e9e740 1 client.15828 shutdown 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 msgr2=0x55c161fccb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f530001ca80 tx=0x7f530001c650 comp rx=0 tx=0).stop 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 shutdown_connections 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c16204ca00 0x55c16204ee20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c161fc91e0 0x55c161fcd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c161fbd250 0x55c161fccb70 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 --2- 172.21.15.138:0/3371879122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c161fbc930 0x55c161fd1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5306000640 0 client.15828 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 >> 172.21.15.138:0/3371879122 conn(0x55c161fba760 msgr2=0x55c161fc7460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 shutdown_connections 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 1 -- 172.21.15.138:0/3371879122 wait complete. 2024-08-20T01:10:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.887+0000 7f5321e9e740 20 client.15828 trim_cache size 0 max 16384 2024-08-20T01:10:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:10:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:10:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:10:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:10:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 msgr2=0x7f52fc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 0x7f52fc0b6200 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55c161fbfab0 tx=0x7f5308031400 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 msgr2=0x7f531819b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f5308005ab0 tx=0x7f5308003c10 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 shutdown_connections 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f52fc0b3d40 0x7f52fc0b6200 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f531810af90 0x7f53181a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f531810a610 0x7f531819b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 --2- 172.21.15.138:0/2703957632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5318104c10 0x7f531819b140 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 >> 172.21.15.138:0/2703957632 conn(0x7f5318100610 msgr2=0x7f5318102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 shutdown_connections 2024-08-20T01:10:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:10:59.888+0000 7f531f000640 1 -- 172.21.15.138:0/2703957632 wait complete. 2024-08-20T01:10:59.959 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:10:59 smithi138 ceph-mon[18076]: pgmap v448: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-08-20T01:10:59.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:10:59 smithi167 ceph-mon[21965]: pgmap v448: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-08-20T01:10:59.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:10:59 smithi142 ceph-mon[21799]: pgmap v448: 129 pgs: 129 active+clean; 68 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 255 op/s 2024-08-20T01:11:00.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.84 MB/sec warmup 72 sec latency 4679.823 ms 2024-08-20T01:11:01.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.75 MB/sec warmup 73 sec latency 5679.950 ms 2024-08-20T01:11:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:01 smithi167 ceph-mon[21965]: pgmap v449: 129 pgs: 129 active+clean; 66 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-08-20T01:11:01.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:01 smithi142 ceph-mon[21799]: pgmap v449: 129 pgs: 129 active+clean; 66 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-08-20T01:11:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:01 smithi138 ceph-mon[18076]: pgmap v449: 129 pgs: 129 active+clean; 66 MiB data, 536 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 246 op/s 2024-08-20T01:11:02.110 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.65 MB/sec warmup 74 sec latency 6680.053 ms 2024-08-20T01:11:03.111 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.57 MB/sec warmup 75 sec latency 7680.168 ms 2024-08-20T01:11:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:03 smithi167 ceph-mon[21965]: pgmap v450: 129 pgs: 129 active+clean; 66 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-08-20T01:11:03.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:03 smithi142 ceph-mon[21799]: pgmap v450: 129 pgs: 129 active+clean; 66 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-08-20T01:11:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:03 smithi138 ceph-mon[18076]: pgmap v450: 129 pgs: 129 active+clean; 66 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.5 MiB/s wr, 200 op/s 2024-08-20T01:11:04.118 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.48 MB/sec warmup 76 sec latency 8687.738 ms 2024-08-20T01:11:05.118 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.39 MB/sec warmup 77 sec latency 9687.858 ms 2024-08-20T01:11:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:05 smithi167 ceph-mon[21965]: pgmap v451: 129 pgs: 129 active+clean; 67 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-20T01:11:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:05 smithi142 ceph-mon[21799]: pgmap v451: 129 pgs: 129 active+clean; 67 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-20T01:11:06.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:05 smithi138 ceph-mon[18076]: pgmap v451: 129 pgs: 129 active+clean; 67 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-08-20T01:11:06.118 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.31 MB/sec warmup 78 sec latency 10688.005 ms 2024-08-20T01:11:06.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:06 smithi167 ceph-mon[21965]: pgmap v452: 129 pgs: 129 active+clean; 70 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-20T01:11:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:06 smithi142 ceph-mon[21799]: pgmap v452: 129 pgs: 129 active+clean; 70 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-20T01:11:07.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:06 smithi138 ceph-mon[18076]: pgmap v452: 129 pgs: 129 active+clean; 70 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-08-20T01:11:07.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.23 MB/sec warmup 79 sec latency 11688.126 ms 2024-08-20T01:11:07.856 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:11:08.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.15 MB/sec warmup 80 sec latency 12688.185 ms 2024-08-20T01:11:08.235 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 -- 172.21.15.138:0/898939236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c10f0c0 msgr2=0x7f0f5c1114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 --2- 172.21.15.138:0/898939236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1114b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009ca0 tx=0x7f0f5402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 -- 172.21.15.138:0/898939236 shutdown_connections 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 --2- 172.21.15.138:0/898939236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1114b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 --2- 172.21.15.138:0/898939236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c075b10 0x7f0f5c10eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 --2- 172.21.15.138:0/898939236 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c0751f0 0x7f0f5c0755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 -- 172.21.15.138:0/898939236 >> 172.21.15.138:0/898939236 conn(0x7f0f5c0fe500 msgr2=0x7f0f5c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 -- 172.21.15.138:0/898939236 shutdown_connections 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.037+0000 7f0f63c00640 1 -- 172.21.15.138:0/898939236 wait complete. 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.038+0000 7f0f63c00640 1 Processor -- start 2024-08-20T01:11:09.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.038+0000 7f0f63c00640 1 -- start start 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.038+0000 7f0f63c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c0751f0 0x7f0f5c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f63c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f63c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f63c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c114cf0 con 0x7f0f5c075b10 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f63c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c114b70 con 0x7f0f5c0751f0 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f63c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c114e70 con 0x7f0f5c10f0c0 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59512/0 (socket says 172.21.15.138:59512) 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 -- 172.21.15.138:0/2638544061 learned_addr learned my addr 172.21.15.138:0/2638544061 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f61c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f61200640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c0751f0 0x7f0f5c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 -- 172.21.15.138:0/2638544061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c10f0c0 msgr2=0x7f0f5c1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:09.068 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 -- 172.21.15.138:0/2638544061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c0751f0 msgr2=0x7f0f5c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c0751f0 0x7f0f5c19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.039+0000 7f0f5be00640 1 -- 172.21.15.138:0/2638544061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c1a1250 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f5be00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9f0 tx=0x7f0f4800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800cd80 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f480108e0 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c1a1540 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48018980 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.040+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c1a7dd0 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.041+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c076700 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.045+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0f48010430 con 0x7f0f5c075b10 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.047+0000 7f0f59600640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 0x7f0f440b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:09.069 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.048+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f0f480e4650 con 0x7f0f5c075b10 2024-08-20T01:11:09.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.048+0000 7f0f61200640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 0x7f0f440b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:09.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.048+0000 7f0f61200640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 0x7f0f440b6280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009bd0 tx=0x7f0f50005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:09.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.052+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0f480aaf60 con 0x7f0f5c075b10 2024-08-20T01:11:09.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.08 MB/sec warmup 81 sec latency 13688.258 ms 2024-08-20T01:11:09.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.380+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f5c1a7fa0 con 0x7f0f5c075b10 2024-08-20T01:11:09.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.381+0000 7f0f59600640 1 -- 172.21.15.138:0/2638544061 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11936 (secure 0 0 0) 0x7f0f480af4b0 con 0x7f0f5c075b10 2024-08-20T01:11:09.381 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:11:09.381 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 30 2024-08-20T01:11:09.381 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":30,"btime":"2024-08-20T01:10:49:648211+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:10:49.219429+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:11:09.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.384+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 msgr2=0x7f0f440b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:09.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.384+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 0x7f0f440b6280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0f50009bd0 tx=0x7f0f50005f90 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.384+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 msgr2=0x7f0f5c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.384+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9f0 tx=0x7f0f4800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 shutdown_connections 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0f440b3dc0 0x7f0f440b6280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0f5c10f0c0 0x7f0f5c1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f5c075b10 0x7f0f5c19b6e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 --2- 172.21.15.138:0/2638544061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f5c0751f0 0x7f0f5c19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:09.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 >> 172.21.15.138:0/2638544061 conn(0x7f0f5c0fe500 msgr2=0x7f0f5c111260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:09.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 shutdown_connections 2024-08-20T01:11:09.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:09.385+0000 7f0f63c00640 1 -- 172.21.15.138:0/2638544061 wait complete. 2024-08-20T01:11:09.845 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:09 smithi138 ceph-mon[18076]: pgmap v453: 129 pgs: 129 active+clean; 69 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 475 KiB/s wr, 79 op/s 2024-08-20T01:11:09.845 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:09 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2638544061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:09 smithi167 ceph-mon[21965]: pgmap v453: 129 pgs: 129 active+clean; 69 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 475 KiB/s wr, 79 op/s 2024-08-20T01:11:09.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2638544061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:09 smithi142 ceph-mon[21799]: pgmap v453: 129 pgs: 129 active+clean; 69 MiB data, 541 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 475 KiB/s wr, 79 op/s 2024-08-20T01:11:09.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:09 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2638544061' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:10.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 6.00 MB/sec warmup 82 sec latency 14688.303 ms 2024-08-20T01:11:10.526 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id f90e1cc8 --release --await' 2024-08-20T01:11:10.526 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id f90e1cc8 --release --await 2024-08-20T01:11:10.739 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:10 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:11:10.739 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:10 smithi138 ceph-mon[18076]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-20T01:11:10.913 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:11:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:10 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:11:10.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:10 smithi167 ceph-mon[21965]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-20T01:11:10.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:10 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:11:10.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:10 smithi142 ceph-mon[21799]: pgmap v454: 129 pgs: 129 active+clean; 75 MiB data, 529 MiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 1.1 MiB/s wr, 94 op/s 2024-08-20T01:11:11.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88093 5.93 MB/sec warmup 83 sec latency 15688.348 ms 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 -- 172.21.15.138:0/2222856145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 msgr2=0x7f53c4105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 --2- 172.21.15.138:0/2222856145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c4105080 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f53b8009900 tx=0x7f53b802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 -- 172.21.15.138:0/2222856145 shutdown_connections 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 --2- 172.21.15.138:0/2222856145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410b030 0x7f53c4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 --2- 172.21.15.138:0/2222856145 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410a6b0 0x7f53c410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 --2- 172.21.15.138:0/2222856145 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c4105080 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 -- 172.21.15.138:0/2222856145 >> 172.21.15.138:0/2222856145 conn(0x7f53c4100730 msgr2=0x7f53c4102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 -- 172.21.15.138:0/2222856145 shutdown_connections 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.656+0000 7f53cb800640 1 -- 172.21.15.138:0/2222856145 wait complete. 2024-08-20T01:11:11.682 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.657+0000 7f53cb800640 1 Processor -- start 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 -- start start 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410a6b0 0x7f53c419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410b030 0x7f53c41a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4114cf0 con 0x7f53c4104ca0 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4114b70 con 0x7f53c410a6b0 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53cb800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c4114e70 con 0x7f53c410b030 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53c8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53c8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59538/0 (socket says 172.21.15.138:59538) 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53c8e00640 1 -- 172.21.15.138:0/2319027382 learned_addr learned my addr 172.21.15.138:0/2319027382 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.658+0000 7f53c3e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410a6b0 0x7f53c419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c9800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410b030 0x7f53c41a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.683 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 -- 172.21.15.138:0/2319027382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410b030 msgr2=0x7f53c41a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410b030 0x7f53c41a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 -- 172.21.15.138:0/2319027382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410a6b0 msgr2=0x7f53c419b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410a6b0 0x7f53c419b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 -- 172.21.15.138:0/2319027382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c41a1230 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c8e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005940 tx=0x7f53b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c1600640 1 -- 172.21.15.138:0/2319027382 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b803f030 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.659+0000 7f53c1600640 1 -- 172.21.15.138:0/2319027382 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b8003aa0 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.660+0000 7f53c1600640 1 -- 172.21.15.138:0/2319027382 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b8035070 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.660+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c41a14c0 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.660+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c41a7be0 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 Processor -- start 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 -- start start 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53c1600640 1 -- 172.21.15.138:0/2319027382 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f53b8042030 con 0x7f53c4104ca0 2024-08-20T01:11:11.684 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb00c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb00e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594fafa1670 con 0x5594fb000610 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594faf75d70 con 0x5594fb00c5a0 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.663+0000 7f53ce428740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594faf65270 con 0x5594fafffcf0 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54302/0 (socket says 172.21.15.138:54302) 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 -- 172.21.15.138:0/3363706148 learned_addr learned my addr 172.21.15.138:0/3363706148 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c8e00640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb00c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c3e00640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb00e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 -- 172.21.15.138:0/3363706148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 msgr2=0x5594fb00e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb00e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.685 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 -- 172.21.15.138:0/3363706148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 msgr2=0x5594fb00c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb00c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 -- 172.21.15.138:0/3363706148 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594faf01870 con 0x5594fafffcf0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c3e00640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb00e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.664+0000 7f53c9800640 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53bc010370 tx=0x7f53bc010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d5c267ac4558754 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53c1600640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 0x7f53ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53b6000640 1 -- 172.21.15.138:0/3363706148 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc016070 con 0x5594fafffcf0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53b6000640 1 -- 172.21.15.138:0/3363706148 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc004d20 con 0x5594fafffcf0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53b6000640 1 -- 172.21.15.138:0/3363706148 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0053c0 con 0x5594fafffcf0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53c1600640 1 -- 172.21.15.138:0/2319027382 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f53b81049e0 con 0x7f53c4104ca0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.665+0000 7f53c8e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 0x7f53ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- 172.21.15.138:0/3363706148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 msgr2=0x5594fb0000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f53bc010370 tx=0x7f53bc010830 comp rx=0 tx=0).stop 2024-08-20T01:11:11.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53c8e00640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 0x7f53ac0b6200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5594faffea00 tx=0x7f53b810a530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- 172.21.15.138:0/3363706148 shutdown_connections 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb00e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb00c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 --2- 172.21.15.138:0/3363706148 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb0000d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- 172.21.15.138:0/3363706148 >> 172.21.15.138:0/3363706148 conn(0x5594faffdb20 msgr2=0x5594faffecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- 172.21.15.138:0/3363706148 shutdown_connections 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- 172.21.15.138:0/3363706148 wait complete. 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 -- start start 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.666+0000 7f53ce428740 1 Processor -- start 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb000610 0x5594fb009220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb009760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594faeebff0 con 0x5594fafffcf0 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5594faf692b0 con 0x5594fb00c5a0 2024-08-20T01:11:11.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594fadc7bb0 con 0x5594fb000610 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53ce428740 10 client.0 _ll_register_callbacks cb 0x5594fac70d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59566/0 (socket says 172.21.15.138:59566) 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53c9800640 1 -- 172.21.15.138:0/896135437 learned_addr learned my addr 172.21.15.138:0/896135437 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53c3e00640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb000610 0x5594fb009220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.667+0000 7f53c9800640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb009760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 -- 172.21.15.138:0/896135437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb000610 msgr2=0x5594fb009220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb000610 0x5594fb009220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 -- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 msgr2=0x5594fb009760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb009760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 -- 172.21.15.138:0/896135437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594facc4b00 con 0x5594fafffcf0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53c9800640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005cc0 tx=0x7f53bc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d72e4562e2b0af server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc016070 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f53bc004240 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.668+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53bc004d20 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f53bc004ef0 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 10 client.15846 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5594fad3b960 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53bc004d20 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f53bc00f040 con 0x5594fafffcf0 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 10 client.15846 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 10 client.15846 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 20 client.15846 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb01bac0 0x5594fb01deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 4 client.15846 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:11.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.669+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594faf77cc0 con 0x5594fb01bac0 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.670+0000 7f53c3e00640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb01bac0 0x5594fb01deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.670+0000 7f53c3e00640 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb01bac0 0x5594fb01deb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c93806dfd5c2b3b server_cookie=61706318ad366b5 in_seq=0 out_seq=0 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.670+0000 7f53aaa00640 10 client.15846 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53aaa00640 1 -- 172.21.15.138:0/896135437 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594faf77cc0 con 0x5594fb01bac0 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53aaa00640 10 client.15846 handle_command_reply: tid=0 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 client.15846 shutdown 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 msgr2=0x5594fb00bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f53bc005cc0 tx=0x7f53bc005c50 comp rx=0 tx=0).stop 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 shutdown_connections 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb01bac0 0x5594fb01deb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb009760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb000610 0x5594fb009220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.690 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 --2- 172.21.15.138:0/896135437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fafffcf0 0x5594fb00bdd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53aaa00640 0 client.15846 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:11.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 >> 172.21.15.138:0/896135437 conn(0x5594faffdb20 msgr2=0x5594fb0036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 shutdown_connections 2024-08-20T01:11:11.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.671+0000 7f53ce428740 1 -- 172.21.15.138:0/896135437 wait complete. 2024-08-20T01:11:11.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.672+0000 7f53ce428740 20 client.15846 trim_cache size 0 max 16384 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.689+0000 7f53ce428740 1 Processor -- start 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.689+0000 7f53ce428740 1 -- start start 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fafffcf0 0x5594fb0181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 0x5594fb0085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41b45e0 con 0x5594fb000610 2024-08-20T01:11:11.694 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41b1e50 con 0x5594fafffcf0 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53ce428740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41afca0 con 0x5594fb00c5a0 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53c9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fafffcf0 0x5594fb0181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53c3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59582/0 (socket says 172.21.15.138:59582) 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53c3e00640 1 -- 172.21.15.138:0/615177332 learned_addr learned my addr 172.21.15.138:0/615177332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.690+0000 7f53c9800640 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 0x5594fb0085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 -- 172.21.15.138:0/615177332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 msgr2=0x5594fb0085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 0x5594fb0085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 -- 172.21.15.138:0/615177332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fafffcf0 msgr2=0x5594fb0181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fafffcf0 0x5594fb0181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 -- 172.21.15.138:0/615177332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594faffe800 con 0x5594fb000610 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c9800640 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 0x5594fb0085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:11:11.695 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53c3e00640 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009d20 tx=0x7f53b0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2785f3f0f8b790e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53b5600640 1 -- 172.21.15.138:0/615177332 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b0022020 con 0x5594fb000610 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53b5600640 1 -- 172.21.15.138:0/615177332 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b00042f0 con 0x5594fb000610 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.691+0000 7f53b5600640 1 -- 172.21.15.138:0/615177332 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b001e900 con 0x5594fb000610 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 -- 172.21.15.138:0/615177332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 msgr2=0x5594fb008080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009d20 tx=0x7f53b0005f90 comp rx=0 tx=0).stop 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 -- 172.21.15.138:0/615177332 shutdown_connections 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fb00c5a0 0x5594fb0085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb000610 0x5594fb008080 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 --2- 172.21.15.138:0/615177332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fafffcf0 0x5594fb0181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 -- 172.21.15.138:0/615177332 >> 172.21.15.138:0/615177332 conn(0x5594faffdb20 msgr2=0x5594fb007970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 -- 172.21.15.138:0/615177332 shutdown_connections 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.692+0000 7f53ce428740 1 -- 172.21.15.138:0/615177332 wait complete. 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 -- start start 2024-08-20T01:11:11.696 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 Processor -- start 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb010630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb010b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41b0310 con 0x5594fb0834a0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41b1780 con 0x5594fb00c5a0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f53c41b09e0 con 0x5594fafffcf0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.693+0000 7f53ce428740 10 client.0 _ll_register_callbacks cb 0x7f53b810a400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb010b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59594/0 (socket says 172.21.15.138:59594) 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 -- 172.21.15.138:0/756973016 learned_addr learned my addr 172.21.15.138:0/756973016 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb010630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 -- 172.21.15.138:0/756973016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 msgr2=0x5594fb010630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.697 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb010630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 -- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 msgr2=0x5594fb010b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb010b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 -- 172.21.15.138:0/756973016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594fb085dd0 con 0x5594fb0834a0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53c9800640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f53bc015040 tx=0x7f53bc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d09dcc850f02ae server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.694+0000 7f53aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53bc016070 con 0x5594fb0834a0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f53bc005810 con 0x5594fb0834a0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53b00042f0 con 0x5594fb0834a0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f53bc005a30 con 0x5594fb0834a0 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53ce428740 10 client.15852 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:11.698 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5594fb0870e0 con 0x5594fb0834a0 2024-08-20T01:11:11.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53b00042f0 con 0x5594fb0834a0 2024-08-20T01:11:11.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f53bc0040e0 con 0x5594fb0834a0 2024-08-20T01:11:11.699 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.695+0000 7f53ce428740 10 client.15852 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53ce428740 10 client.15852 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53ce428740 20 client.15852 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb087770 0x5594fb089b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53ce428740 4 client.15852 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f90e1cc8", "release": true, "await": true}] 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f90e1cc8", "release": true, "await": true}) -- 0x5594faf73760 con 0x5594fb087770 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53c3e00640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb087770 0x5594fb089b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53c3e00640 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb087770 0x5594fb089b90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e24e827f177c6ffb server_cookie=1405da2ac855679f in_seq=0 out_seq=0 2024-08-20T01:11:11.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.696+0000 7f53aaa00640 10 client.15852 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:11.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.707+0000 7f53aaa00640 1 -- 172.21.15.138:0/756973016 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5594faf73760 con 0x5594fb087770 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.707+0000 7f53aaa00640 10 client.15852 handle_command_reply: tid=0 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.707+0000 7f53ce428740 1 client.15852 shutdown 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.707+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 msgr2=0x5594fb085890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.707+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f53bc015040 tx=0x7f53bc004ac0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 shutdown_connections 2024-08-20T01:11:11.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594fb087770 0x5594fb089b90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594fb0834a0 0x5594fb085890 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594fb00c5a0 0x5594fb010b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 --2- 172.21.15.138:0/756973016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594fafffcf0 0x5594fb010630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 >> 172.21.15.138:0/756973016 conn(0x5594faffdb20 msgr2=0x5594fb008190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53aaa00640 0 client.15852 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 shutdown_connections 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 1 -- 172.21.15.138:0/756973016 wait complete. 2024-08-20T01:11:11.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.708+0000 7f53ce428740 20 client.15852 trim_cache size 0 max 16384 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 31, 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 16, 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "f90e1cc8": { 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 16, 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:11:11.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 136.4, 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:11.711 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.709+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 msgr2=0x7f53ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.709+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 0x7f53ac0b6200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5594faffea00 tx=0x7f53b810a530 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.709+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 msgr2=0x7f53c419b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.709+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f53b8005940 tx=0x7f53b8004270 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 shutdown_connections 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f53ac0b3d40 0x7f53ac0b6200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f53c410b030 0x7f53c41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53c410a6b0 0x7f53c419b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 --2- 172.21.15.138:0/2319027382 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53c4104ca0 0x7f53c419b180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 >> 172.21.15.138:0/2319027382 conn(0x7f53c4100730 msgr2=0x7f53c4102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 shutdown_connections 2024-08-20T01:11:11.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:11.710+0000 7f53cb800640 1 -- 172.21.15.138:0/2319027382 wait complete. 2024-08-20T01:11:12.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 88419 5.88 MB/sec warmup 84 sec latency 16272.064 ms 2024-08-20T01:11:12.993 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f90e1cc8', total seconds elapsed: 21.5 2024-08-20T01:11:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 -- 172.21.15.138:0/3758132312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 msgr2=0x7fe0fc1113c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 --2- 172.21.15.138:0/3758132312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1113c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4009900 tx=0x7fe0f402a400 comp rx=0 tx=0).stop 2024-08-20T01:11:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 -- 172.21.15.138:0/3758132312 shutdown_connections 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 --2- 172.21.15.138:0/3758132312 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1113c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 --2- 172.21.15.138:0/3758132312 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 --2- 172.21.15.138:0/3758132312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc0755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 -- 172.21.15.138:0/3758132312 >> 172.21.15.138:0/3758132312 conn(0x7fe0fc0fe430 msgr2=0x7fe0fc100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 -- 172.21.15.138:0/3758132312 shutdown_connections 2024-08-20T01:11:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.038+0000 7fe101a00640 1 -- 172.21.15.138:0/3758132312 wait complete. 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 Processor -- start 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 -- start start 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc114bd0 con 0x7fe0fc10efd0 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc114a50 con 0x7fe0fc075210 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.039+0000 7fe101a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc114d50 con 0x7fe0fc075b30 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57466/0 (socket says 172.21.15.138:57466) 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 -- 172.21.15.138:0/795943306 learned_addr learned my addr 172.21.15.138:0/795943306 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0fb400640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 -- 172.21.15.138:0/795943306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 msgr2=0x7fe0fc19b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc19b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 -- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 msgr2=0x7fe0fc1a0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1a0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 -- 172.21.15.138:0/795943306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0fc1a1130 con 0x7fe0fc075210 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0f2000640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0faa00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00ea60 tx=0x7fe0ec00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.040+0000 7fe0f3e00640 1 -- 172.21.15.138:0/795943306 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00cde0 con 0x7fe0fc075210 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.041+0000 7fe0f3e00640 1 -- 172.21.15.138:0/795943306 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec0035f0 con 0x7fe0fc075210 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.041+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0fc1a1420 con 0x7fe0fc075210 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.041+0000 7fe0f3e00640 1 -- 172.21.15.138:0/795943306 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec010690 con 0x7fe0fc075210 2024-08-20T01:11:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.041+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0fc1a7cc0 con 0x7fe0fc075210 2024-08-20T01:11:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.044+0000 7fe104a79740 1 Processor -- start 2024-08-20T01:11:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.044+0000 7fe104a79740 1 -- start start 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac8930 0x56083dac8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac9250 0x56083dad4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe0f3e00640 1 -- 172.21.15.138:0/795943306 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe0ec0037a0 con 0x7fe0fc075210 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56083da3e210 con 0x56083dac8930 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56083da2d710 con 0x56083dac9250 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe104a79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56083da31750 con 0x56083dad51e0 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54344/0 (socket says 172.21.15.138:54344) 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe0f2000640 1 -- 172.21.15.138:0/2220626194 learned_addr learned my addr 172.21.15.138:0/2220626194 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.045+0000 7fe0fb400640 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac8930 0x56083dac8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 -- 172.21.15.138:0/2220626194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac9250 msgr2=0x56083dad4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac9250 0x56083dad4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 -- 172.21.15.138:0/2220626194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac8930 msgr2=0x56083dac8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac8930 0x56083dac8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 -- 172.21.15.138:0/2220626194 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083d9f9de0 con 0x56083dad51e0 2024-08-20T01:11:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0f2000640 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4011850 tx=0x7fe0e4011d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79b55dfafd8660a4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0ebe00640 1 -- 172.21.15.138:0/2220626194 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4011220 con 0x56083dad51e0 2024-08-20T01:11:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0ebe00640 1 -- 172.21.15.138:0/2220626194 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e4003970 con 0x56083dad51e0 2024-08-20T01:11:13.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.046+0000 7fe0ebe00640 1 -- 172.21.15.138:0/2220626194 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40043e0 con 0x56083dad51e0 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe0f3e00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 0x7fe0e00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe0f3e00640 1 -- 172.21.15.138:0/795943306 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe0ec0e45f0 con 0x7fe0fc075210 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe0fb400640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 0x7fe0e00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 -- 172.21.15.138:0/2220626194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 msgr2=0x56083dad75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe0e4011850 tx=0x7fe0e4011d10 comp rx=0 tx=0).stop 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 -- 172.21.15.138:0/2220626194 shutdown_connections 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad75d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac9250 0x56083dad4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 --2- 172.21.15.138:0/2220626194 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac8930 0x56083dac8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 -- 172.21.15.138:0/2220626194 >> 172.21.15.138:0/2220626194 conn(0x56083dac6760 msgr2=0x56083dac7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.047+0000 7fe104a79740 1 -- 172.21.15.138:0/2220626194 shutdown_connections 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.048+0000 7fe104a79740 1 -- 172.21.15.138:0/2220626194 wait complete. 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.048+0000 7fe0fb400640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 0x7fe0e00b6280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4006010 tx=0x7fe0f4005a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.048+0000 7fe104a79740 1 -- start start 2024-08-20T01:11:13.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.048+0000 7fe104a79740 1 Processor -- start 2024-08-20T01:11:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dad1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56083d966ab0 con 0x56083dac9250 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56083d9d4140 con 0x56083dac8930 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56083dad7ec0 con 0x56083dad51e0 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe104a79740 10 client.0 _ll_register_callbacks cb 0x56083d9e75d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57482/0 (socket says 172.21.15.138:57482) 2024-08-20T01:11:13.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 -- 172.21.15.138:0/1079890302 learned_addr learned my addr 172.21.15.138:0/1079890302 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0fb400640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dad1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 -- 172.21.15.138:0/1079890302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 msgr2=0x56083dad23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 -- 172.21.15.138:0/1079890302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 msgr2=0x56083dad1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dad1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.049+0000 7fe0faa00640 1 -- 172.21.15.138:0/1079890302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083d9d5170 con 0x56083dac8930 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009920 tx=0x7fe0ec003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c121afd090b7da5d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0aae50 con 0x56083dac8930 2024-08-20T01:11:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe0ec0a70e0 con 0x56083dac8930 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0e4003970 con 0x56083dac8930 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec0af450 con 0x56083dac8930 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe104a79740 10 client.25913 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56083d790790 con 0x56083dac8930 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.050+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0e4003970 con 0x56083dac8930 2024-08-20T01:11:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe0ec0e1c10 con 0x56083dac8930 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 10 client.25913 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 10 client.25913 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 20 client.25913 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083dae4ed0 0x56083dae72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 4 client.25913 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56083da40160 con 0x56083dae4ed0 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.051+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083dae4ed0 0x56083dae72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.052+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083dae4ed0 0x56083dae72c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2102a019e2d9c5c server_cookie=3bc4c7e6792d7cbc in_seq=0 out_seq=0 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.052+0000 7fe0e8c00640 10 client.25913 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.052+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1079890302 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56083da40160 con 0x56083dae4ed0 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.052+0000 7fe0e8c00640 10 client.25913 handle_command_reply: tid=0 2024-08-20T01:11:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.052+0000 7fe104a79740 1 client.25913 shutdown 2024-08-20T01:11:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 msgr2=0x56083dad4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec009920 tx=0x7fe0ec003cf0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 shutdown_connections 2024-08-20T01:11:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083dae4ed0 0x56083dae72c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dad23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dad1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 --2- 172.21.15.138:0/1079890302 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dad4a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe0e8c00640 0 client.25913 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 >> 172.21.15.138:0/1079890302 conn(0x56083dac6760 msgr2=0x56083dacc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 shutdown_connections 2024-08-20T01:11:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 1 -- 172.21.15.138:0/1079890302 wait complete. 2024-08-20T01:11:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.053+0000 7fe104a79740 20 client.25913 trim_cache size 0 max 16384 2024-08-20T01:11:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.071+0000 7fe104a79740 1 Processor -- start 2024-08-20T01:11:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.071+0000 7fe104a79740 1 -- start start 2024-08-20T01:11:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dad51e0 0x56083daed0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b4500 con 0x56083dac9250 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b1d70 con 0x56083dad51e0 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe104a79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1afbc0 con 0x56083dac8930 2024-08-20T01:11:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0f2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59632/0 (socket says 172.21.15.138:59632) 2024-08-20T01:11:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54364/0 (socket says 172.21.15.138:54364) 2024-08-20T01:11:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0f2000640 1 -- 172.21.15.138:0/95283434 learned_addr learned my addr 172.21.15.138:0/95283434 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.072+0000 7fe0fb400640 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dad51e0 0x56083daed0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 -- 172.21.15.138:0/95283434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dad51e0 msgr2=0x56083daed0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dad51e0 0x56083daed0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 -- 172.21.15.138:0/95283434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 msgr2=0x56083daecb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 -- 172.21.15.138:0/95283434 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56083dac9080 con 0x56083dac8930 2024-08-20T01:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0f2000640 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0faa00640 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00d010 tx=0x7fe0ec003e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f557e3eae50cf0ba server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0eb400640 1 -- 172.21.15.138:0/95283434 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0aae50 con 0x56083dac8930 2024-08-20T01:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0eb400640 1 -- 172.21.15.138:0/95283434 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec0ad030 con 0x56083dac8930 2024-08-20T01:11:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.073+0000 7fe0eb400640 1 -- 172.21.15.138:0/95283434 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0a6a60 con 0x56083dac8930 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 -- 172.21.15.138:0/95283434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 msgr2=0x56083dac7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00d010 tx=0x7fe0ec003e00 comp rx=0 tx=0).stop 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 -- 172.21.15.138:0/95283434 shutdown_connections 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dad51e0 0x56083daed0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083daecb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 --2- 172.21.15.138:0/95283434 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dac8930 0x56083dac7d10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 -- 172.21.15.138:0/95283434 >> 172.21.15.138:0/95283434 conn(0x56083dac6760 msgr2=0x56083dac7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 -- 172.21.15.138:0/95283434 shutdown_connections 2024-08-20T01:11:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.074+0000 7fe104a79740 1 -- 172.21.15.138:0/95283434 wait complete. 2024-08-20T01:11:13.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 -- start start 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 Processor -- start 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dadb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dadbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b0230 con 0x56083dac9250 2024-08-20T01:11:13.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b16a0 con 0x56083dac8930 2024-08-20T01:11:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe0fc1b0900 con 0x56083dad51e0 2024-08-20T01:11:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.075+0000 7fe104a79740 10 client.0 _ll_register_callbacks cb 0x7fe0ec0aac20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57500/0 (socket says 172.21.15.138:57500) 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 -- 172.21.15.138:0/1150537225 learned_addr learned my addr 172.21.15.138:0/1150537225 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dadb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0fb400640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dadbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 -- 172.21.15.138:0/1150537225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 msgr2=0x56083dadbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dadbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 -- 172.21.15.138:0/1150537225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 msgr2=0x56083dadb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dadb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 -- 172.21.15.138:0/1150537225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56083d809620 con 0x56083dac8930 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0fb400640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dadbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0faa00640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000c00 tx=0x7fe0ec003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0203e31c92906d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0e8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:13.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec0aae50 con 0x56083dac8930 2024-08-20T01:11:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0ec0ad030 con 0x56083dac8930 2024-08-20T01:11:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.076+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fe0ec0e4220 con 0x56083dac8930 2024-08-20T01:11:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe0ec0be050 con 0x56083dac8930 2024-08-20T01:11:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 10 client.25919 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:13.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56083db585f0 con 0x56083dac8930 2024-08-20T01:11:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0ec0ad030 con 0x56083dac8930 2024-08-20T01:11:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fe0ec0a68e0 con 0x56083dac8930 2024-08-20T01:11:13.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 10 client.25919 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 10 client.25919 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 20 client.25919 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083db58a50 0x56083db5ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 4 client.25919 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56083da3bc00 con 0x56083db58a50 2024-08-20T01:11:13.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.077+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083db58a50 0x56083db5ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.078+0000 7fe0f2000640 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083db58a50 0x56083db5ae70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aac0351fb6db1c7 server_cookie=e5fb413f8d0818e5 in_seq=0 out_seq=0 2024-08-20T01:11:13.096 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.078+0000 7fe0e8c00640 10 client.25919 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.089+0000 7fe0e8c00640 1 -- 172.21.15.138:0/1150537225 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56083da3bc00 con 0x56083db58a50 2024-08-20T01:11:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.089+0000 7fe0e8c00640 10 client.25919 handle_command_reply: tid=0 2024-08-20T01:11:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.089+0000 7fe104a79740 1 client.25919 shutdown 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 msgr2=0x56083dadb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec000c00 tx=0x7fe0ec003cf0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 shutdown_connections 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56083db58a50 0x56083db5ae70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56083dad51e0 0x56083dadbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56083dac9250 0x56083dadb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe0e8c00640 0 client.25919 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 --2- 172.21.15.138:0/1150537225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56083dac8930 0x56083dadb100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 >> 172.21.15.138:0/1150537225 conn(0x56083dac6760 msgr2=0x56083dac4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 shutdown_connections 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 1 -- 172.21.15.138:0/1150537225 wait complete. 2024-08-20T01:11:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.090+0000 7fe104a79740 20 client.25919 trim_cache size 0 max 16384 2024-08-20T01:11:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:11:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:11:13.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:11:13.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:11:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 msgr2=0x7fe0e00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 0x7fe0e00b6280 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe0f4006010 tx=0x7fe0f4005a40 comp rx=0 tx=0).stop 2024-08-20T01:11:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 msgr2=0x7fe0fc19b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec00ea60 tx=0x7fe0ec00ef20 comp rx=0 tx=0).stop 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 shutdown_connections 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe0e00b3dc0 0x7fe0e00b6280 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0fc10efd0 0x7fe0fc1a0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe0fc075b30 0x7fe0fc19b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 --2- 172.21.15.138:0/795943306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe0fc075210 0x7fe0fc19b090 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 >> 172.21.15.138:0/795943306 conn(0x7fe0fc0fe430 msgr2=0x7fe0fc1111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 shutdown_connections 2024-08-20T01:11:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:13.091+0000 7fe101a00640 1 -- 172.21.15.138:0/795943306 wait complete. 2024-08-20T01:11:13.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 89114 5.87 MB/sec warmup 85 sec latency 64.233 ms 2024-08-20T01:11:13.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:13 smithi167 ceph-mon[21965]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-20T01:11:13.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:13 smithi142 ceph-mon[21799]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-20T01:11:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:13 smithi138 ceph-mon[18076]: pgmap v455: 129 pgs: 129 active+clean; 75 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 1.2 MiB/s wr, 76 op/s 2024-08-20T01:11:14.119 INFO:tasks.workunit.client.0.smithi138.stdout: 1 91334 5.90 MB/sec warmup 86 sec latency 12.300 ms 2024-08-20T01:11:15.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 92112 5.90 MB/sec warmup 87 sec latency 696.017 ms 2024-08-20T01:11:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:15 smithi138 ceph-mon[18076]: pgmap v456: 129 pgs: 129 active+clean; 74 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-08-20T01:11:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:15 smithi167 ceph-mon[21965]: pgmap v456: 129 pgs: 129 active+clean; 74 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-08-20T01:11:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:15 smithi142 ceph-mon[21799]: pgmap v456: 129 pgs: 129 active+clean; 74 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 1.6 MiB/s wr, 93 op/s 2024-08-20T01:11:16.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 92112 5.84 MB/sec warmup 88 sec latency 1696.066 ms 2024-08-20T01:11:17.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 92706 5.82 MB/sec warmup 89 sec latency 2136.193 ms 2024-08-20T01:11:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:17 smithi167 ceph-mon[21965]: pgmap v457: 129 pgs: 129 active+clean; 76 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-08-20T01:11:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:17 smithi142 ceph-mon[21799]: pgmap v457: 129 pgs: 129 active+clean; 76 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-08-20T01:11:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:17 smithi138 ceph-mon[18076]: pgmap v457: 129 pgs: 129 active+clean; 76 MiB data, 526 MiB used, 1.0 TiB / 1.0 TiB avail; 553 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-08-20T01:11:18.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 95260 5.90 MB/sec warmup 90 sec latency 36.462 ms 2024-08-20T01:11:19.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 97037 5.93 MB/sec warmup 91 sec latency 17.078 ms 2024-08-20T01:11:19.779 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:19 smithi138 ceph-mon[18076]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 126 op/s 2024-08-20T01:11:19.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:19 smithi167 ceph-mon[21965]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 126 op/s 2024-08-20T01:11:19.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:19 smithi142 ceph-mon[21799]: pgmap v458: 129 pgs: 129 active+clean; 75 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 2.2 MiB/s wr, 126 op/s 2024-08-20T01:11:20.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 97993 5.90 MB/sec warmup 92 sec latency 652.925 ms 2024-08-20T01:11:21.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 98000 5.84 MB/sec warmup 93 sec latency 1330.242 ms 2024-08-20T01:11:21.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:21 smithi167 ceph-mon[21965]: pgmap v459: 129 pgs: 129 active+clean; 73 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 178 op/s 2024-08-20T01:11:21.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:21 smithi142 ceph-mon[21799]: pgmap v459: 129 pgs: 129 active+clean; 73 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 178 op/s 2024-08-20T01:11:22.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:21 smithi138 ceph-mon[18076]: pgmap v459: 129 pgs: 129 active+clean; 73 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 178 op/s 2024-08-20T01:11:22.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 99937 5.92 MB/sec warmup 94 sec latency 321.462 ms 2024-08-20T01:11:23.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 102993 6.04 MB/sec warmup 95 sec latency 13.087 ms 2024-08-20T01:11:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:23 smithi167 ceph-mon[21965]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-08-20T01:11:23.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:23 smithi142 ceph-mon[21799]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-08-20T01:11:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:23 smithi138 ceph-mon[18076]: pgmap v460: 129 pgs: 129 active+clean; 75 MiB data, 538 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.5 MiB/s wr, 204 op/s 2024-08-20T01:11:24.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 104502 6.05 MB/sec warmup 96 sec latency 24.032 ms 2024-08-20T01:11:25.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 105891 6.06 MB/sec warmup 97 sec latency 404.383 ms 2024-08-20T01:11:25.913 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:25 smithi138 ceph-mon[18076]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T01:11:25.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:25 smithi167 ceph-mon[21965]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T01:11:25.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:25 smithi142 ceph-mon[21799]: pgmap v461: 129 pgs: 129 active+clean; 76 MiB data, 586 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 246 op/s 2024-08-20T01:11:26.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 106526 6.05 MB/sec warmup 98 sec latency 966.659 ms 2024-08-20T01:11:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 -- 172.21.15.138:0/804794769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58108200 msgr2=0x7f1a58108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 --2- 172.21.15.138:0/804794769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58108200 0x7f1a58108640 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009900 tx=0x7f1a4402a410 comp rx=0 tx=0).stop 2024-08-20T01:11:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 -- 172.21.15.138:0/804794769 shutdown_connections 2024-08-20T01:11:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 --2- 172.21.15.138:0/804794769 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 0x7f1a5810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 --2- 172.21.15.138:0/804794769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58108200 0x7f1a58108640 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 --2- 172.21.15.138:0/804794769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58102800 0x7f1a58102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 -- 172.21.15.138:0/804794769 >> 172.21.15.138:0/804794769 conn(0x7f1a580fe280 msgr2=0x7f1a581006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 -- 172.21.15.138:0/804794769 shutdown_connections 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.223+0000 7f1a5e800640 1 -- 172.21.15.138:0/804794769 wait complete. 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 Processor -- start 2024-08-20T01:11:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 -- start start 2024-08-20T01:11:26.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 0x7f1a58075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.224+0000 7f1a5e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58112890 con 0x7f1a58102800 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58112710 con 0x7f1a58108200 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a58112a10 con 0x7f1a58108b80 2024-08-20T01:11:26.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60164/0 (socket says 172.21.15.138:60164) 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 -- 172.21.15.138:0/472256773 learned_addr learned my addr 172.21.15.138:0/472256773 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37396/0 (socket says 172.21.15.138:37396) 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5de00640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 0x7f1a58075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 -- 172.21.15.138:0/472256773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 msgr2=0x7f1a58075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 0x7f1a58075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 -- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 msgr2=0x7f1a580751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 -- 172.21.15.138:0/472256773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a581a7200 con 0x7f1a58108200 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5d400640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.225+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.226+0000 7f1a56000640 1 -- 172.21.15.138:0/472256773 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4403f070 con 0x7f1a58108200 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.226+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a581a7490 con 0x7f1a58108200 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.226+0000 7f1a56000640 1 -- 172.21.15.138:0/472256773 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a44031d60 con 0x7f1a58108200 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.226+0000 7f1a56000640 1 -- 172.21.15.138:0/472256773 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44035030 con 0x7f1a58108200 2024-08-20T01:11:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.226+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a581a79d0 con 0x7f1a58108200 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.229+0000 7f1a56000640 1 -- 172.21.15.138:0/472256773 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1a44042070 con 0x7f1a58108200 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.229+0000 7f1a62c34740 1 Processor -- start 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 -- start start 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776afce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 0x55d776aff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d776a66210 con 0x55d776af0b10 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d776a55710 con 0x55d776afd3c0 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a62c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d776a59750 con 0x55d776af1430 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a5de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a5de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37406/0 (socket says 172.21.15.138:37406) 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a5ca00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 0x55d776aff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a5de00640 1 -- 172.21.15.138:0/2316170332 learned_addr learned my addr 172.21.15.138:0/2316170332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.230+0000 7f1a5d400640 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776afce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 -- 172.21.15.138:0/2316170332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 msgr2=0x55d776afce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776afce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 -- 172.21.15.138:0/2316170332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 msgr2=0x55d776aff7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 0x55d776aff7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 -- 172.21.15.138:0/2316170332 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d776a21de0 con 0x55d776af0b10 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 0x55d776aff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a56000640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 0x7f1a400b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a56000640 1 -- 172.21.15.138:0/472256773 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1a44104ae0 con 0x7f1a58108200 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5de00640 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009f70 tx=0x7f1a50010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744f4ed93447d6ec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a5d400640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 0x7f1a400b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a4a600640 1 -- 172.21.15.138:0/2316170332 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50016070 con 0x55d776af0b10 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.231+0000 7f1a4a600640 1 -- 172.21.15.138:0/2316170332 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50004d20 con 0x55d776af0b10 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a4a600640 1 -- 172.21.15.138:0/2316170332 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50012810 con 0x55d776af0b10 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 -- 172.21.15.138:0/2316170332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 msgr2=0x55d776af0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009f70 tx=0x7f1a50010800 comp rx=0 tx=0).stop 2024-08-20T01:11:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a5d400640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 0x7f1a400b6200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55d776af3c90 tx=0x7f1a4c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 -- 172.21.15.138:0/2316170332 shutdown_connections 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776afd3c0 0x55d776aff7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776afce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 --2- 172.21.15.138:0/2316170332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af0b10 0x55d776af0ef0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 -- 172.21.15.138:0/2316170332 >> 172.21.15.138:0/2316170332 conn(0x55d776aee940 msgr2=0x55d776aefae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 -- 172.21.15.138:0/2316170332 shutdown_connections 2024-08-20T01:11:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.232+0000 7f1a62c34740 1 -- 172.21.15.138:0/2316170332 wait complete. 2024-08-20T01:11:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 -- start start 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 Processor -- start 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776af9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776afa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d77698eab0 con 0x55d776afd3c0 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d7769fc140 con 0x55d776af0b10 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d776b000f0 con 0x55d776af1430 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.233+0000 7f1a62c34740 10 client.0 _ll_register_callbacks cb 0x55d776a0f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60186/0 (socket says 172.21.15.138:60186) 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5d400640 1 -- 172.21.15.138:0/3656350998 learned_addr learned my addr 172.21.15.138:0/3656350998 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776af9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5de00640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776afa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 -- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 msgr2=0x55d776afcb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 -- 172.21.15.138:0/3656350998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 msgr2=0x55d776afa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776afa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 -- 172.21.15.138:0/3656350998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7769fd170 con 0x55d776af1430 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5d400640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776af9f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a44101ee0 tx=0x7f1a44031f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe8ab9759104d558 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a44049070 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.234+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a50004d20 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1a440c7a40 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a440c7310 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 10 client.25885 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d7767b8790 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50004d20 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1a4403b4f0 con 0x55d776af1430 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 10 client.25885 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 20 client.25885 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b0ce90 0x55d776b0f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 4 client.25885 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.235+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d776a68160 con 0x55d776b0ce90 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.236+0000 7f1a5d400640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b0ce90 0x55d776b0f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.236+0000 7f1a5d400640 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b0ce90 0x55d776b0f2c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ef7eca511c9c23 server_cookie=f7ee2431875f9b42 in_seq=0 out_seq=0 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.236+0000 7f1a3ea00640 10 client.25885 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a3ea00640 1 -- 172.21.15.138:0/3656350998 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d776a68160 con 0x55d776b0ce90 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a3ea00640 10 client.25885 handle_command_reply: tid=0 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 client.25885 shutdown 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 msgr2=0x55d776af9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776af9f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1a44101ee0 tx=0x7f1a44031f10 comp rx=0 tx=0).stop 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 shutdown_connections 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b0ce90 0x55d776b0f2c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776afa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af1430 0x55d776af9f80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 --2- 172.21.15.138:0/3656350998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776afcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a3ea00640 0 client.25885 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 >> 172.21.15.138:0/3656350998 conn(0x55d776aee940 msgr2=0x55d776af44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 shutdown_connections 2024-08-20T01:11:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 1 -- 172.21.15.138:0/3656350998 wait complete. 2024-08-20T01:11:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.237+0000 7f1a62c34740 20 client.25885 trim_cache size 0 max 16384 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 Processor -- start 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 -- start start 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 0x55d776af5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af1430 0x55d776b14ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b4400 con 0x55d776afd3c0 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b1c70 con 0x55d776af1430 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.252+0000 7f1a62c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581afac0 con 0x55d776af0b10 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 0x55d776af5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37438/0 (socket says 172.21.15.138:37438) 2024-08-20T01:11:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 -- 172.21.15.138:0/1363878301 learned_addr learned my addr 172.21.15.138:0/1363878301 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af1430 0x55d776b14ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 -- 172.21.15.138:0/1363878301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 msgr2=0x55d776af5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 0x55d776af5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 -- 172.21.15.138:0/1363878301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af1430 msgr2=0x55d776b14ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af1430 0x55d776b14ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 -- 172.21.15.138:0/1363878301 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d776831620 con 0x55d776afd3c0 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5d400640 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 0x55d776af5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.253+0000 7f1a5de00640 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1a50011010 tx=0x7f1a50009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2124a0b9530b1cb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a49c00640 1 -- 172.21.15.138:0/1363878301 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5002a070 con 0x55d776afd3c0 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a49c00640 1 -- 172.21.15.138:0/1363878301 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50004d40 con 0x55d776afd3c0 2024-08-20T01:11:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a49c00640 1 -- 172.21.15.138:0/1363878301 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50016070 con 0x55d776afd3c0 2024-08-20T01:11:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 -- 172.21.15.138:0/1363878301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 msgr2=0x55d776b150e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1a50011010 tx=0x7f1a50009820 comp rx=0 tx=0).stop 2024-08-20T01:11:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 -- 172.21.15.138:0/1363878301 shutdown_connections 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776afd3c0 0x55d776b150e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af1430 0x55d776b14ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1363878301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776af0b10 0x55d776af5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 -- 172.21.15.138:0/1363878301 >> 172.21.15.138:0/1363878301 conn(0x55d776aee940 msgr2=0x55d776af4e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 -- 172.21.15.138:0/1363878301 shutdown_connections 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.254+0000 7f1a62c34740 1 -- 172.21.15.138:0/1363878301 wait complete. 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 -- start start 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 Processor -- start 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 0x55d776b01a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776afd3c0 0x55d776b76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b0130 con 0x55d776af1430 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b15a0 con 0x55d776af0b10 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1a581b0800 con 0x55d776afd3c0 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a62c34740 10 client.0 _ll_register_callbacks cb 0x7f1a44108320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a5ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 0x55d776b01a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a5de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776afd3c0 0x55d776b76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a5d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60206/0 (socket says 172.21.15.138:60206) 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.255+0000 7f1a5d400640 1 -- 172.21.15.138:0/1707471859 learned_addr learned my addr 172.21.15.138:0/1707471859 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 -- 172.21.15.138:0/1707471859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776afd3c0 msgr2=0x55d776b76b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776afd3c0 0x55d776b76b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 -- 172.21.15.138:0/1707471859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 msgr2=0x55d776b01a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 0x55d776b01a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 -- 172.21.15.138:0/1707471859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d776b770b0 con 0x55d776af0b10 2024-08-20T01:11:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 0x55d776b01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a5d400640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009530 tx=0x7f1a4c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d19303f8cf8af870 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a3ea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c004020 con 0x55d776af0b10 2024-08-20T01:11:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.256+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a50004d40 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1a4c0041c0 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1a4c03b050 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 10 client.25937 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d776b78290 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a50004d40 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1a4c003a00 con 0x55d776af0b10 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 10 client.25937 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 10 client.25937 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 20 client.25937 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b78a40 0x55d776b7ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 4 client.25937 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.257+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d776af9d20 con 0x55d776b78a40 2024-08-20T01:11:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.258+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b78a40 0x55d776b7ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.258+0000 7f1a5ca00640 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b78a40 0x55d776b7ae60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b63a7f9b9c383d8f server_cookie=e7d9070693581b55 in_seq=0 out_seq=0 2024-08-20T01:11:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.258+0000 7f1a3ea00640 10 client.25937 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.317+0000 7f1a3ea00640 1 -- 172.21.15.138:0/1707471859 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d776af9d20 con 0x55d776b78a40 2024-08-20T01:11:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.317+0000 7f1a3ea00640 10 client.25937 handle_command_reply: tid=0 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.317+0000 7f1a62c34740 1 client.25937 shutdown 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 msgr2=0x55d776b01530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009530 tx=0x7f1a4c017000 comp rx=0 tx=0).stop 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 shutdown_connections 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d776b78a40 0x55d776b7ae60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d776afd3c0 0x55d776b76b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d776af1430 0x55d776b01a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a3ea00640 0 client.25937 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 --2- 172.21.15.138:0/1707471859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d776af0b10 0x55d776b01530 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 >> 172.21.15.138:0/1707471859 conn(0x55d776aee940 msgr2=0x55d776afb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 shutdown_connections 2024-08-20T01:11:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 1 -- 172.21.15.138:0/1707471859 wait complete. 2024-08-20T01:11:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.318+0000 7f1a62c34740 20 client.25937 trim_cache size 0 max 16384 2024-08-20T01:11:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:11:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:11:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:11:26.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 msgr2=0x7f1a400b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 0x7f1a400b6200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55d776af3c90 tx=0x7f1a4c005f90 comp rx=0 tx=0).stop 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 msgr2=0x7f1a58075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a44005ab0 tx=0x7f1a44003c80 comp rx=0 tx=0).stop 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 shutdown_connections 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1a400b3d40 0x7f1a400b6200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1a58108b80 0x7f1a58075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a58108200 0x7f1a58075710 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 --2- 172.21.15.138:0/472256773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a58102800 0x7f1a580751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 >> 172.21.15.138:0/472256773 conn(0x7f1a580fe280 msgr2=0x7f1a58100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 shutdown_connections 2024-08-20T01:11:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:26.319+0000 7f1a5e800640 1 -- 172.21.15.138:0/472256773 wait complete. 2024-08-20T01:11:26.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:26 smithi167 ceph-mon[21965]: pgmap v462: 129 pgs: 129 active+clean; 72 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 248 op/s 2024-08-20T01:11:26.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:26 smithi142 ceph-mon[21799]: pgmap v462: 129 pgs: 129 active+clean; 72 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 248 op/s 2024-08-20T01:11:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:26 smithi138 ceph-mon[18076]: pgmap v462: 129 pgs: 129 active+clean; 72 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 248 op/s 2024-08-20T01:11:27.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 107629 6.03 MB/sec warmup 99 sec latency 136.902 ms 2024-08-20T01:11:28.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 110297 6.13 MB/sec warmup 100 sec latency 10.419 ms 2024-08-20T01:11:29.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 112401 6.15 MB/sec warmup 101 sec latency 16.262 ms 2024-08-20T01:11:29.882 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:29 smithi138 ceph-mon[18076]: pgmap v463: 129 pgs: 129 active+clean; 73 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:11:29.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:29 smithi167 ceph-mon[21965]: pgmap v463: 129 pgs: 129 active+clean; 73 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:11:29.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:29 smithi142 ceph-mon[21799]: pgmap v463: 129 pgs: 129 active+clean; 73 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:11:30.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 113983 6.20 MB/sec warmup 102 sec latency 126.205 ms 2024-08-20T01:11:31.121 INFO:tasks.workunit.client.0.smithi138.stdout: 1 116869 6.28 MB/sec warmup 103 sec latency 16.924 ms 2024-08-20T01:11:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:31 smithi167 ceph-mon[21965]: pgmap v464: 129 pgs: 129 active+clean; 75 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-08-20T01:11:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:31 smithi142 ceph-mon[21799]: pgmap v464: 129 pgs: 129 active+clean; 75 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-08-20T01:11:32.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:31 smithi138 ceph-mon[18076]: pgmap v464: 129 pgs: 129 active+clean; 75 MiB data, 629 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 304 op/s 2024-08-20T01:11:32.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 118351 6.29 MB/sec warmup 104 sec latency 12.821 ms 2024-08-20T01:11:33.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 120992 6.39 MB/sec warmup 105 sec latency 11.435 ms 2024-08-20T01:11:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:33 smithi167 ceph-mon[21965]: pgmap v465: 129 pgs: 129 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-08-20T01:11:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:33 smithi142 ceph-mon[21799]: pgmap v465: 129 pgs: 129 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-08-20T01:11:34.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:33 smithi138 ceph-mon[18076]: pgmap v465: 129 pgs: 129 active+clean; 76 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 263 op/s 2024-08-20T01:11:34.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 123100 6.40 MB/sec warmup 106 sec latency 13.896 ms 2024-08-20T01:11:35.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 124642 6.45 MB/sec warmup 107 sec latency 185.362 ms 2024-08-20T01:11:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:35 smithi167 ceph-mon[21965]: pgmap v466: 129 pgs: 129 active+clean; 78 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-20T01:11:35.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:35 smithi142 ceph-mon[21799]: pgmap v466: 129 pgs: 129 active+clean; 78 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-20T01:11:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:35 smithi138 ceph-mon[18076]: pgmap v466: 129 pgs: 129 active+clean; 78 MiB data, 666 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.2 MiB/s wr, 252 op/s 2024-08-20T01:11:36.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 127521 6.52 MB/sec warmup 108 sec latency 16.769 ms 2024-08-20T01:11:37.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 128934 6.53 MB/sec warmup 109 sec latency 14.346 ms 2024-08-20T01:11:37.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:37 smithi167 ceph-mon[21965]: pgmap v467: 129 pgs: 129 active+clean; 82 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-20T01:11:37.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:37 smithi142 ceph-mon[21799]: pgmap v467: 129 pgs: 129 active+clean; 82 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-20T01:11:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:37 smithi138 ceph-mon[18076]: pgmap v467: 129 pgs: 129 active+clean; 82 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.2 MiB/s wr, 265 op/s 2024-08-20T01:11:38.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 131632 6.62 MB/sec warmup 110 sec latency 14.033 ms 2024-08-20T01:11:39.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 133563 6.62 MB/sec warmup 111 sec latency 37.870 ms 2024-08-20T01:11:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 -- 172.21.15.138:0/1959565360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 msgr2=0x7f705410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 --2- 172.21.15.138:0/1959565360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705410aa50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f703c009900 tx=0x7f703c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 -- 172.21.15.138:0/1959565360 shutdown_connections 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 --2- 172.21.15.138:0/1959565360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 0x7f7054111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 --2- 172.21.15.138:0/1959565360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705410aa50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 --2- 172.21.15.138:0/1959565360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 0x7f7054104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 -- 172.21.15.138:0/1959565360 >> 172.21.15.138:0/1959565360 conn(0x7f7054100610 msgr2=0x7f7054102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 -- 172.21.15.138:0/1959565360 shutdown_connections 2024-08-20T01:11:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.447+0000 7f7059a00640 1 -- 172.21.15.138:0/1959565360 wait complete. 2024-08-20T01:11:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.448+0000 7f7059a00640 1 Processor -- start 2024-08-20T01:11:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 -- start start 2024-08-20T01:11:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 0x7f705419b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 0x7f70541a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7054114ca0 con 0x7f7054104c10 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7054114b20 con 0x7f705410af90 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f7059a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7054114e20 con 0x7f705410a610 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.449+0000 7f704be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57852/0 (socket says 172.21.15.138:57852) 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 -- 172.21.15.138:0/4017826902 learned_addr learned my addr 172.21.15.138:0/4017826902 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f7052a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 0x7f705419b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f7053400640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 0x7f70541a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 -- 172.21.15.138:0/4017826902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 msgr2=0x7f70541a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 0x7f70541a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 -- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 msgr2=0x7f705419b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 0x7f705419b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 -- 172.21.15.138:0/4017826902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70541a11c0 con 0x7f705410a610 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.450+0000 7f704be00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f703c005ab0 tx=0x7f703c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.451+0000 7f704b400640 1 -- 172.21.15.138:0/4017826902 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c03f070 con 0x7f705410a610 2024-08-20T01:11:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.451+0000 7f704b400640 1 -- 172.21.15.138:0/4017826902 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703c031b60 con 0x7f705410a610 2024-08-20T01:11:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.451+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70541a1450 con 0x7f705410a610 2024-08-20T01:11:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.451+0000 7f704b400640 1 -- 172.21.15.138:0/4017826902 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703c035030 con 0x7f705410a610 2024-08-20T01:11:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.451+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70540772c0 con 0x7f705410a610 2024-08-20T01:11:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.454+0000 7f704b400640 1 -- 172.21.15.138:0/4017826902 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f703c042070 con 0x7f705410a610 2024-08-20T01:11:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 Processor -- start 2024-08-20T01:11:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 -- start start 2024-08-20T01:11:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e3dc0f210 con 0x558e3dca5e80 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558e3dbfe710 con 0x558e3dc995d0 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.455+0000 7f705c9f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e3dc02750 con 0x558e3dc99ef0 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42136/0 (socket says 172.21.15.138:42136) 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 -- 172.21.15.138:0/4178413791 learned_addr learned my addr 172.21.15.138:0/4178413791 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7052a00640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f704be00640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 -- 172.21.15.138:0/4178413791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 msgr2=0x558e3dca5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 -- 172.21.15.138:0/4178413791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 msgr2=0x558e3dca8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 -- 172.21.15.138:0/4178413791 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3dbcade0 con 0x558e3dc995d0 2024-08-20T01:11:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7052a00640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f704be00640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:11:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.456+0000 7f7053400640 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f704c009aa0 tx=0x7f704c009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c80883062da8b2f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f704b400640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 0x7f70380b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f7043e00640 1 -- 172.21.15.138:0/4178413791 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c023070 con 0x558e3dc995d0 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f7043e00640 1 -- 172.21.15.138:0/4178413791 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704c003e20 con 0x558e3dc995d0 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f7043e00640 1 -- 172.21.15.138:0/4178413791 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c004790 con 0x558e3dc995d0 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f704b400640 1 -- 172.21.15.138:0/4017826902 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f703c104810 con 0x7f705410a610 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.457+0000 7f7052a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 0x7f70380b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- 172.21.15.138:0/4178413791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 msgr2=0x558e3dc999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f704c009aa0 tx=0x7f704c009f60 comp rx=0 tx=0).stop 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f7052a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 0x7f70380b6280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558e3dc9c750 tx=0x7f7044005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- 172.21.15.138:0/4178413791 shutdown_connections 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 --2- 172.21.15.138:0/4178413791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dc999b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- 172.21.15.138:0/4178413791 >> 172.21.15.138:0/4178413791 conn(0x558e3dc973b0 msgr2=0x558e3dc985a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- 172.21.15.138:0/4178413791 shutdown_connections 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- 172.21.15.138:0/4178413791 wait complete. 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.458+0000 7f705c9f1740 1 -- start start 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 Processor -- start 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc995d0 0x558e3dca2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558e3db37ab0 con 0x558e3dc99ef0 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558e3dba5140 con 0x558e3dca5e80 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558e3dca8b60 con 0x558e3dc995d0 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f705c9f1740 10 client.0 _ll_register_callbacks cb 0x558e3dbb85d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f7053400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f7053400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42140/0 (socket says 172.21.15.138:42140) 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.459+0000 7f7053400640 1 -- 172.21.15.138:0/3844648441 learned_addr learned my addr 172.21.15.138:0/3844648441 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f704be00640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7052a00640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc995d0 0x558e3dca2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 -- 172.21.15.138:0/3844648441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc995d0 msgr2=0x558e3dca2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc995d0 0x558e3dca2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 -- 172.21.15.138:0/3844648441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 msgr2=0x558e3dca3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 -- 172.21.15.138:0/3844648441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3dba6170 con 0x558e3dca5e80 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7053400640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f704c014e10 tx=0x7f704c004870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f86254911db675d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7040c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c02d060 con 0x558e3dca5e80 2024-08-20T01:11:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f704c004a40 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.460+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558e3dbd5a30 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f704c034050 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 10 client.25949 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e3d961790 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558e3dbd5a30 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f704c005300 con 0x558e3dca5e80 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 10 client.25949 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 10 client.25949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 20 client.25949 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dcb5c60 0x558e3dcb8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 4 client.25949 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.461+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e3dc11160 con 0x558e3dcb5c60 2024-08-20T01:11:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.462+0000 7f7052a00640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dcb5c60 0x558e3dcb8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.462+0000 7f7052a00640 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dcb5c60 0x558e3dcb8050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56aa3f2c2bc28a5 server_cookie=d0767fcc65a9fcef in_seq=0 out_seq=0 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.462+0000 7f7040c00640 10 client.25949 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f7040c00640 1 -- 172.21.15.138:0/3844648441 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e3dc11160 con 0x558e3dcb5c60 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f7040c00640 10 client.25949 handle_command_reply: tid=0 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 client.25949 shutdown 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 msgr2=0x558e3dca3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f704c014e10 tx=0x7f704c004870 comp rx=0 tx=0).stop 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 shutdown_connections 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dcb5c60 0x558e3dcb8050 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dca5e80 0x558e3dca3840 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 --2- 172.21.15.138:0/3844648441 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc995d0 0x558e3dca2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f7040c00640 0 client.25949 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 >> 172.21.15.138:0/3844648441 conn(0x558e3dc973b0 msgr2=0x558e3dc9cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 shutdown_connections 2024-08-20T01:11:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.463+0000 7f705c9f1740 1 -- 172.21.15.138:0/3844648441 wait complete. 2024-08-20T01:11:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.464+0000 7f705c9f1740 20 client.25949 trim_cache size 0 max 16384 2024-08-20T01:11:39.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.479+0000 7f705c9f1740 1 Processor -- start 2024-08-20T01:11:39.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 -- start start 2024-08-20T01:11:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70541b0110 con 0x558e3dca5e80 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70541ad980 con 0x558e3dc995d0 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f705c9f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70541ab7d0 con 0x558e3dc99ef0 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7052a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7052a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42154/0 (socket says 172.21.15.138:42154) 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7052a00640 1 -- 172.21.15.138:0/2026354012 learned_addr learned my addr 172.21.15.138:0/2026354012 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7053400640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f704be00640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7052a00640 1 -- 172.21.15.138:0/2026354012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 msgr2=0x558e3dca22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.480+0000 7f7052a00640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7052a00640 1 -- 172.21.15.138:0/2026354012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 msgr2=0x558e3dca27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7052a00640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7052a00640 1 -- 172.21.15.138:0/2026354012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e3d9da620 con 0x558e3dc995d0 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7053400640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7052a00640 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558e3dcb85f0 tx=0x7f704401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4befadb3995ff1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7043400640 1 -- 172.21.15.138:0/2026354012 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704401fc00 con 0x558e3dc995d0 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7043400640 1 -- 172.21.15.138:0/2026354012 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704401fda0 con 0x558e3dc995d0 2024-08-20T01:11:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.481+0000 7f7043400640 1 -- 172.21.15.138:0/2026354012 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704402b420 con 0x558e3dc995d0 2024-08-20T01:11:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- 172.21.15.138:0/2026354012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 msgr2=0x558e3dca35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x558e3dcb85f0 tx=0x7f704401a000 comp rx=0 tx=0).stop 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- 172.21.15.138:0/2026354012 shutdown_connections 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dca5e80 0x558e3dca27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dc99ef0 0x558e3dca22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 --2- 172.21.15.138:0/2026354012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dca35b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- 172.21.15.138:0/2026354012 >> 172.21.15.138:0/2026354012 conn(0x558e3dc973b0 msgr2=0x558e3dca4c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- 172.21.15.138:0/2026354012 shutdown_connections 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- 172.21.15.138:0/2026354012 wait complete. 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.482+0000 7f705c9f1740 1 -- start start 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 Processor -- start 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 0x558e3dca92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f70541abe40 con 0x558e3dc99ef0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f70541ad2b0 con 0x558e3dc995d0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f70541ac510 con 0x558e3dca5e80 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f705c9f1740 10 client.0 _ll_register_callbacks cb 0x7f704c021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f7052a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f7053400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 0x558e3dca92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f704be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f7052a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42158/0 (socket says 172.21.15.138:42158) 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.483+0000 7f7052a00640 1 -- 172.21.15.138:0/1304054267 learned_addr learned my addr 172.21.15.138:0/1304054267 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 -- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 msgr2=0x558e3dcae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 -- 172.21.15.138:0/1304054267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 msgr2=0x558e3dca8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:39.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 -- 172.21.15.138:0/1304054267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e3dca9b20 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7052a00640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7053400640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 0x558e3dca92f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f704c0057a0 tx=0x7f704c009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0c40e376bf5450d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7040c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c03c070 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f704401fda0 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f704c00bbd0 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f704c048050 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f705c9f1740 10 client.25909 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.484+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558e3dd1f650 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f704401fda0 con 0x558e3dca5e80 2024-08-20T01:11:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f704c0048e0 con 0x558e3dca5e80 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 10 client.25909 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 20 client.25909 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dd1fcb0 0x558e3dd220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 4 client.25909 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558e3dc0cc00 con 0x558e3dd1fcb0 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.485+0000 7f7052a00640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dd1fcb0 0x558e3dd220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.486+0000 7f7052a00640 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dd1fcb0 0x558e3dd220d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8071bc8d545ef70d server_cookie=9e999d5a73ef39e3 in_seq=0 out_seq=0 2024-08-20T01:11:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.486+0000 7f7040c00640 10 client.25909 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f7040c00640 1 -- 172.21.15.138:0/1304054267 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558e3dc0cc00 con 0x558e3dd1fcb0 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f7040c00640 10 client.25909 handle_command_reply: tid=0 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 client.25909 shutdown 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 msgr2=0x558e3dca92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 0x558e3dca92f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f704c0057a0 tx=0x7f704c009970 comp rx=0 tx=0).stop 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 shutdown_connections 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558e3dd1fcb0 0x558e3dd220d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558e3dca5e80 0x558e3dca92f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558e3dc99ef0 0x558e3dca8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 --2- 172.21.15.138:0/1304054267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558e3dc995d0 0x558e3dcae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f7040c00640 0 client.25909 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 >> 172.21.15.138:0/1304054267 conn(0x558e3dc973b0 msgr2=0x558e3dc9cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 shutdown_connections 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.561+0000 7f705c9f1740 1 -- 172.21.15.138:0/1304054267 wait complete. 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f705c9f1740 20 client.25909 trim_cache size 0 max 16384 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:11:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 msgr2=0x7f70380b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 0x7f70380b6280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x558e3dc9c750 tx=0x7f7044005f90 comp rx=0 tx=0).stop 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 msgr2=0x7f705419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f703c005ab0 tx=0x7f703c003c10 comp rx=0 tx=0).stop 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 shutdown_connections 2024-08-20T01:11:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70380b3dc0 0x7f70380b6280 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f705410af90 0x7f70541a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f705410a610 0x7f705419b660 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 --2- 172.21.15.138:0/4017826902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7054104c10 0x7f705419b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.562+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 >> 172.21.15.138:0/4017826902 conn(0x7f7054100610 msgr2=0x7f7054102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.563+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 shutdown_connections 2024-08-20T01:11:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:39.563+0000 7f7059a00640 1 -- 172.21.15.138:0/4017826902 wait complete. 2024-08-20T01:11:39.922 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:39 smithi138 ceph-mon[18076]: pgmap v468: 129 pgs: 129 active+clean; 83 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 253 op/s 2024-08-20T01:11:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:39 smithi167 ceph-mon[21965]: pgmap v468: 129 pgs: 129 active+clean; 83 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 253 op/s 2024-08-20T01:11:39.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:39 smithi142 ceph-mon[21799]: pgmap v468: 129 pgs: 129 active+clean; 83 MiB data, 677 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.0 MiB/s wr, 253 op/s 2024-08-20T01:11:40.122 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135140 6.67 MB/sec warmup 112 sec latency 227.627 ms 2024-08-20T01:11:41.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135142 6.61 MB/sec warmup 113 sec latency 999.894 ms 2024-08-20T01:11:41.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:41 smithi167 ceph-mon[21965]: pgmap v469: 129 pgs: 129 active+clean; 87 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-08-20T01:11:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:41 smithi142 ceph-mon[21799]: pgmap v469: 129 pgs: 129 active+clean; 87 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-08-20T01:11:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:41 smithi138 ceph-mon[18076]: pgmap v469: 129 pgs: 129 active+clean; 87 MiB data, 717 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-08-20T01:11:42.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135142 6.55 MB/sec warmup 114 sec latency 1999.968 ms 2024-08-20T01:11:43.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135142 6.50 MB/sec warmup 115 sec latency 3000.041 ms 2024-08-20T01:11:43.451 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-20T01:11:43.451 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-08-20T01:11:43.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:11:43.768 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:43 smithi138 ceph-mon[18076]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-08-20T01:11:43.832 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:11:43.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:43 smithi167 ceph-mon[21965]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-08-20T01:11:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:43 smithi142 ceph-mon[21799]: pgmap v470: 129 pgs: 129 active+clean; 109 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 269 op/s 2024-08-20T01:11:44.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135142 6.44 MB/sec warmup 116 sec latency 4000.088 ms 2024-08-20T01:11:44.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.775+0000 7f8e70200640 1 -- 172.21.15.138:0/317632099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e680751f0 msgr2=0x7f8e680755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:44.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.775+0000 7f8e70200640 1 --2- 172.21.15.138:0/317632099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e680751f0 0x7f8e680755d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009860 tx=0x7f8e5c02a340 comp rx=0 tx=0).stop 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 -- 172.21.15.138:0/317632099 shutdown_connections 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 --2- 172.21.15.138:0/317632099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 0x7f8e681114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 --2- 172.21.15.138:0/317632099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e68075b10 0x7f8e6810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 --2- 172.21.15.138:0/317632099 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e680751f0 0x7f8e680755d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 -- 172.21.15.138:0/317632099 >> 172.21.15.138:0/317632099 conn(0x7f8e680fe500 msgr2=0x7f8e68100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 -- 172.21.15.138:0/317632099 shutdown_connections 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.776+0000 7f8e70200640 1 -- 172.21.15.138:0/317632099 wait complete. 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 Processor -- start 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 -- start start 2024-08-20T01:11:44.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 0x7f8e6819b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 0x7f8e681a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68114ca0 con 0x7f8e6810f0c0 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68114b20 con 0x7f8e680751f0 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.777+0000 7f8e70200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8e68114e20 con 0x7f8e68075b10 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42174/0 (socket says 172.21.15.138:42174) 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 0x7f8e6819b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 -- 172.21.15.138:0/3611016322 learned_addr learned my addr 172.21.15.138:0/3611016322 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6e200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 0x7f8e681a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:44.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 -- 172.21.15.138:0/3611016322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 msgr2=0x7f8e6819b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 0x7f8e6819b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 -- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 msgr2=0x7f8e681a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 0x7f8e681a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6d800640 1 -- 172.21.15.138:0/3611016322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e681a1300 con 0x7f8e680751f0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.778+0000 7f8e6ce00640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 0x7f8e6819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e6d800640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c035040 tx=0x7f8e5c032d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c037040 con 0x7f8e680751f0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e5c041070 con 0x7f8e680751f0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e681a1590 con 0x7f8e680751f0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c003850 con 0x7f8e680751f0 2024-08-20T01:11:44.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.779+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e681a7cb0 con 0x7f8e680751f0 2024-08-20T01:11:44.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.780+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e680755d0 con 0x7f8e680751f0 2024-08-20T01:11:44.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.812+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8e5c0039f0 con 0x7f8e680751f0 2024-08-20T01:11:44.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.813+0000 7f8e66000640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 0x7f8e500b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:44.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.813+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8e5c104690 con 0x7f8e680751f0 2024-08-20T01:11:44.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.814+0000 7f8e6ce00640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 0x7f8e500b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:44.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.814+0000 7f8e6ce00640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 0x7f8e500b6280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8e540096c0 tx=0x7f8e54009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:44.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:44.822+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8e5c0cb780 con 0x7f8e680751f0 2024-08-20T01:11:44.880 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:44 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:11:44.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:44 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:11:44.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:44 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:11:45.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.060+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e681a8020 con 0x7f8e680751f0 2024-08-20T01:11:45.059 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.061+0000 7f8e66000640 1 -- 172.21.15.138:0/3611016322 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11935 (secure 0 0 0) 0x7f8e5c0cb080 con 0x7f8e680751f0 2024-08-20T01:11:45.060 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:11:45.060 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":31,"btime":"2024-08-20T01:11:09:621725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:11:09.220799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:11:45.060 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 31 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 msgr2=0x7f8e500b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 0x7f8e500b6280 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8e540096c0 tx=0x7f8e54009260 comp rx=0 tx=0).stop 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 msgr2=0x7f8e6819b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c035040 tx=0x7f8e5c032d50 comp rx=0 tx=0).stop 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 shutdown_connections 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8e500b3dc0 0x7f8e500b6280 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8e6810f0c0 0x7f8e681a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8e68075b10 0x7f8e6819b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:45.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 --2- 172.21.15.138:0/3611016322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e680751f0 0x7f8e6819b250 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:45.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 >> 172.21.15.138:0/3611016322 conn(0x7f8e680fe500 msgr2=0x7f8e680ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:45.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 shutdown_connections 2024-08-20T01:11:45.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:45.064+0000 7f8e70200640 1 -- 172.21.15.138:0/3611016322 wait complete. 2024-08-20T01:11:45.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135146 6.39 MB/sec warmup 117 sec latency 4907.135 ms 2024-08-20T01:11:45.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:45 smithi138 ceph-mon[18076]: pgmap v471: 129 pgs: 129 active+clean; 115 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-08-20T01:11:45.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3611016322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:45.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:11:45.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:11:45.708 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:45 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:11:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:45 smithi167 ceph-mon[21965]: pgmap v471: 129 pgs: 129 active+clean; 115 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-08-20T01:11:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:45 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3611016322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:11:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:11:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:45 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:11:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:45 smithi142 ceph-mon[21799]: pgmap v471: 129 pgs: 129 active+clean; 115 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.5 MiB/s wr, 320 op/s 2024-08-20T01:11:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:45 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3611016322' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:11:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:11:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:11:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:45 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:11:46.124 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-08-20T01:11:46.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-08-20T01:11:46.133 INFO:tasks.workunit.client.0.smithi138.stdout: 1 135146 6.33 MB/sec warmup 118 sec latency 1091.161 ms 2024-08-20T01:11:46.546 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:11:47.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136524 6.33 MB/sec warmup 119 sec latency 1206.778 ms 2024-08-20T01:11:47.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 -- 172.21.15.138:0/2189624091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d8104cb0 msgr2=0x7f67d8105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 --2- 172.21.15.138:0/2189624091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d8104cb0 0x7f67d8105090 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009900 tx=0x7f67cc02a410 comp rx=0 tx=0).stop 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 -- 172.21.15.138:0/2189624091 shutdown_connections 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 --2- 172.21.15.138:0/2189624091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810b040 0x7f67d8111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 --2- 172.21.15.138:0/2189624091 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d810a6c0 0x7f67d810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 --2- 172.21.15.138:0/2189624091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d8104cb0 0x7f67d8105090 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 -- 172.21.15.138:0/2189624091 >> 172.21.15.138:0/2189624091 conn(0x7f67d8100720 msgr2=0x7f67d8102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 -- 172.21.15.138:0/2189624091 shutdown_connections 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.155+0000 7f67df800640 1 -- 172.21.15.138:0/2189624091 wait complete. 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.156+0000 7f67df800640 1 Processor -- start 2024-08-20T01:11:47.155 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.156+0000 7f67df800640 1 -- start start 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 0x7f67d819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 0x7f67d819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 0x7f67d81a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8114d10 con 0x7f67d810a6c0 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8114b90 con 0x7f67d810b040 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67df800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8114e90 con 0x7f67d8104cb0 2024-08-20T01:11:47.156 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67dd800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 0x7f67d81a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 0x7f67d819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67dd800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 0x7f67d81a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42194/0 (socket says 172.21.15.138:42194) 2024-08-20T01:11:47.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67dd800640 1 -- 172.21.15.138:0/2550324193 learned_addr learned my addr 172.21.15.138:0/2550324193 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:47.157 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67dce00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 0x7f67d819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 -- 172.21.15.138:0/2550324193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 msgr2=0x7f67d819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 0x7f67d819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 -- 172.21.15.138:0/2550324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 msgr2=0x7f67d81a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 0x7f67d81a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.157+0000 7f67d7e00640 1 -- 172.21.15.138:0/2550324193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81a1230 con 0x7f67d810a6c0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67dce00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 0x7f67d819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67d7e00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 0x7f67d819b6c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67d5600640 1 -- 172.21.15.138:0/2550324193 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ccc0 con 0x7f67d810a6c0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67d5600640 1 -- 172.21.15.138:0/2550324193 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c40108e0 con 0x7f67d810a6c0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67d5600640 1 -- 172.21.15.138:0/2550324193 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4018880 con 0x7f67d810a6c0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d81a1520 con 0x7f67d810a6c0 2024-08-20T01:11:47.158 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.158+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d81a7be0 con 0x7f67d810a6c0 2024-08-20T01:11:47.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.161+0000 7f67e2249740 1 Processor -- start 2024-08-20T01:11:47.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 -- start start 2024-08-20T01:11:47.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c226f0 0x558d14c24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d14b8b600 con 0x558d14c226f0 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d14b7ab00 con 0x558d14c16760 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67e2249740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d14b7eb40 con 0x558d14c15e40 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67dd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67dd800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57930/0 (socket says 172.21.15.138:57930) 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67dd800640 1 -- 172.21.15.138:0/3306438394 learned_addr learned my addr 172.21.15.138:0/3306438394 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.162+0000 7f67d5600640 1 -- 172.21.15.138:0/2550324193 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f67c4010430 con 0x7f67d810a6c0 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 -- 172.21.15.138:0/3306438394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 msgr2=0x558d14c221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 -- 172.21.15.138:0/3306438394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c226f0 msgr2=0x558d14c24ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c226f0 0x558d14c24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 -- 172.21.15.138:0/3306438394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d14b20ee0 con 0x558d14c15e40 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67dd800640 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f67d0011570 tx=0x7f67d0011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=febfb4fabd0a9900 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67ca000640 1 -- 172.21.15.138:0/3306438394 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004b20 con 0x558d14c15e40 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67ca000640 1 -- 172.21.15.138:0/3306438394 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d0004cc0 con 0x558d14c15e40 2024-08-20T01:11:47.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.163+0000 7f67ca000640 1 -- 172.21.15.138:0/3306438394 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0005490 con 0x558d14c15e40 2024-08-20T01:11:47.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 -- 172.21.15.138:0/3306438394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 msgr2=0x558d14c16220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f67d0011570 tx=0x7f67d0011a30 comp rx=0 tx=0).stop 2024-08-20T01:11:47.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 -- 172.21.15.138:0/3306438394 shutdown_connections 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c226f0 0x558d14c24ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 --2- 172.21.15.138:0/3306438394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c15e40 0x558d14c16220 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 -- 172.21.15.138:0/3306438394 >> 172.21.15.138:0/3306438394 conn(0x558d14c13c70 msgr2=0x558d14c14e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67d5600640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 0x7f67c00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.164+0000 7f67e2249740 1 -- 172.21.15.138:0/3306438394 shutdown_connections 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.165+0000 7f67d5600640 1 -- 172.21.15.138:0/2550324193 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f67c40e4690 con 0x7f67d810a6c0 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.165+0000 7f67e2249740 1 -- 172.21.15.138:0/3306438394 wait complete. 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.165+0000 7f67e2249740 1 -- start start 2024-08-20T01:11:47.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.165+0000 7f67dce00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 0x7f67c00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 Processor -- start 2024-08-20T01:11:47.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67dce00640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 0x7f67c00b6280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005ab0 tx=0x7f67cc005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c1f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c200d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558d149dd7b0 con 0x558d14c15e40 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558d14b15dd0 con 0x558d14c16760 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558d14c25370 con 0x558d14c226f0 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.166+0000 7f67e2249740 10 client.0 _ll_register_callbacks cb 0x558d148870a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42202/0 (socket says 172.21.15.138:42202) 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c1f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 -- 172.21.15.138:0/3398976161 learned_addr learned my addr 172.21.15.138:0/3398976161 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67dd800640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 -- 172.21.15.138:0/3398976161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 msgr2=0x558d14c200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 -- 172.21.15.138:0/3398976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 msgr2=0x558d14c1f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c1f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 -- 172.21.15.138:0/3398976161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d148e9ba0 con 0x558d14c16760 2024-08-20T01:11:47.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67d7e00640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67c40aaf30 tx=0x7f67c4010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbeb32f96d15dab1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.167+0000 7f67bea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40af4f0 con 0x558d14c16760 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f67c40e42c0 con 0x558d14c16760 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67d0004cc0 con 0x558d14c16760 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67c40b3fa0 con 0x558d14c16760 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 10 client.25973 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:47.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d14c30ed0 con 0x558d14c16760 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d0004cc0 con 0x558d14c16760 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f67c40a6b10 con 0x558d14c16760 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 10 client.25973 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 10 client.25973 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 20 client.25973 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c32220 0x558d14c34610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 4 client.25973 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.168+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d14b8d550 con 0x558d14c32220 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.169+0000 7f67dce00640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c32220 0x558d14c34610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.169+0000 7f67dce00640 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c32220 0x558d14c34610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7793b52926a2d6 server_cookie=9bfa587ab1a6f018 in_seq=0 out_seq=0 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.169+0000 7f67bea00640 10 client.25973 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.170+0000 7f67bea00640 1 -- 172.21.15.138:0/3398976161 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558d14b8d550 con 0x558d14c32220 2024-08-20T01:11:47.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.170+0000 7f67bea00640 10 client.25973 handle_command_reply: tid=0 2024-08-20T01:11:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.170+0000 7f67e2249740 1 client.25973 shutdown 2024-08-20T01:11:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.170+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 msgr2=0x558d14c1fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.170+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f67c40aaf30 tx=0x7f67c4010860 comp rx=0 tx=0).stop 2024-08-20T01:11:47.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 shutdown_connections 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c32220 0x558d14c34610 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c16760 0x558d14c1fb90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 --2- 172.21.15.138:0/3398976161 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c1f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67bea00640 0 client.25973 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 >> 172.21.15.138:0/3398976161 conn(0x558d14c13c70 msgr2=0x558d14c19810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 shutdown_connections 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 1 -- 172.21.15.138:0/3398976161 wait complete. 2024-08-20T01:11:47.172 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.171+0000 7f67e2249740 20 client.25973 trim_cache size 0 max 16384 2024-08-20T01:11:47.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.193+0000 7f67e2249740 1 Processor -- start 2024-08-20T01:11:47.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.193+0000 7f67e2249740 1 -- start start 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c226f0 0x558d14c39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81b4680 con 0x558d14c15e40 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81b1ef0 con 0x558d14c226f0 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67e2249740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81afd40 con 0x558d14c16760 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67dd800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c226f0 0x558d14c39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67dce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50618/0 (socket says 172.21.15.138:50618) 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57948/0 (socket says 172.21.15.138:57948) 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67dce00640 1 -- 172.21.15.138:0/4128200542 learned_addr learned my addr 172.21.15.138:0/4128200542 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:47.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 -- 172.21.15.138:0/4128200542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c226f0 msgr2=0x558d14c39ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c226f0 0x558d14c39ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 -- 172.21.15.138:0/4128200542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 msgr2=0x558d14c19f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.194+0000 7f67d7e00640 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67d7e00640 1 -- 172.21.15.138:0/4128200542 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d14c34c70 con 0x558d14c16760 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67dce00640 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:47.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67d7e00640 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9c0 tx=0x7f67c40a9c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e78c4e473aa0b52b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67c9600640 1 -- 172.21.15.138:0/4128200542 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40e42c0 con 0x558d14c16760 2024-08-20T01:11:47.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67c9600640 1 -- 172.21.15.138:0/4128200542 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c40e3070 con 0x558d14c16760 2024-08-20T01:11:47.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.195+0000 7f67c9600640 1 -- 172.21.15.138:0/4128200542 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40a6910 con 0x558d14c16760 2024-08-20T01:11:47.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- 172.21.15.138:0/4128200542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 msgr2=0x558d14c1fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9c0 tx=0x7f67c40a9c70 comp rx=0 tx=0).stop 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- 172.21.15.138:0/4128200542 shutdown_connections 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c226f0 0x558d14c39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c16760 0x558d14c1fd30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 --2- 172.21.15.138:0/4128200542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c15e40 0x558d14c19f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- 172.21.15.138:0/4128200542 >> 172.21.15.138:0/4128200542 conn(0x558d14c13c70 msgr2=0x558d14c1f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- 172.21.15.138:0/4128200542 shutdown_connections 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- 172.21.15.138:0/4128200542 wait complete. 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.196+0000 7f67e2249740 1 -- start start 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 Processor -- start 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c16760 0x558d14c2ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c27650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81b03b0 con 0x558d14c16760 2024-08-20T01:11:47.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81b1820 con 0x558d14c15e40 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81b0a80 con 0x558d14c226f0 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67e2249740 10 client.0 _ll_register_callbacks cb 0x7f67c40aa710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67dce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42214/0 (socket says 172.21.15.138:42214) 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67dce00640 1 -- 172.21.15.138:0/621784502 learned_addr learned my addr 172.21.15.138:0/621784502 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67dd800640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c27650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.197+0000 7f67d7e00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c16760 0x558d14c2ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 -- 172.21.15.138:0/621784502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 msgr2=0x558d14c27650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c27650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 -- 172.21.15.138:0/621784502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c16760 msgr2=0x558d14c2ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c16760 0x558d14c2ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 -- 172.21.15.138:0/621784502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d14c27e80 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67dce00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67cc031440 tx=0x7f67cc009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a630c55d7f1a9d30 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67bea00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc035040 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f67cc0515a0 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.198+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67c40e3070 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f67cc061050 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 10 client.25979 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x558d14c9daa0 con 0x558d14c15e40 2024-08-20T01:11:47.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c40e3070 con 0x558d14c15e40 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f67cc044040 con 0x558d14c15e40 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 10 client.25979 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 10 client.25979 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 20 client.25979 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c9e270 0x558d14ca0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 4 client.25979 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.199+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x558d14b88ff0 con 0x558d14c9e270 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.200+0000 7f67d7e00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c9e270 0x558d14ca0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.200+0000 7f67bea00640 10 client.25979 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:47.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.200+0000 7f67d7e00640 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c9e270 0x558d14ca0690 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22ec2d34f38fd8a server_cookie=12f31624de99aac9 in_seq=0 out_seq=0 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67bea00640 1 -- 172.21.15.138:0/621784502 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558d14b88ff0 con 0x558d14c9e270 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67bea00640 10 client.25979 handle_command_reply: tid=0 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 client.25979 shutdown 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 msgr2=0x558d14c2a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67cc031440 tx=0x7f67cc009260 comp rx=0 tx=0).stop 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 shutdown_connections 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558d14c9e270 0x558d14ca0690 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558d14c226f0 0x558d14c27650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558d14c16760 0x558d14c2ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 --2- 172.21.15.138:0/621784502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d14c15e40 0x558d14c2a7e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67bea00640 0 client.25979 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 >> 172.21.15.138:0/621784502 conn(0x558d14c13c70 msgr2=0x558d14c1fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.260+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 shutdown_connections 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.261+0000 7f67e2249740 1 -- 172.21.15.138:0/621784502 wait complete. 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.261+0000 7f67e2249740 20 client.25979 trim_cache size 0 max 16384 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 31, 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:11:47.261 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 18, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "90be3842": { 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 18, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 139.8, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:11:47.262 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:11:47.263 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 msgr2=0x7f67c00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 0x7f67c00b6280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005ab0 tx=0x7f67cc005a20 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 msgr2=0x7f67d819b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 0x7f67d819b6c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e9f0 tx=0x7f67c400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 shutdown_connections 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f67c00b3dc0 0x7f67c00b6280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67d810b040 0x7f67d81a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67d810a6c0 0x7f67d819b6c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 --2- 172.21.15.138:0/2550324193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f67d8104cb0 0x7f67d819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 >> 172.21.15.138:0/2550324193 conn(0x7f67d8100720 msgr2=0x7f67d8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 shutdown_connections 2024-08-20T01:11:47.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:11:47.262+0000 7f67df800640 1 -- 172.21.15.138:0/2550324193 wait complete. 2024-08-20T01:11:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:47 smithi167 ceph-mon[21965]: pgmap v472: 129 pgs: 129 active+clean; 102 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 317 op/s 2024-08-20T01:11:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:47 smithi142 ceph-mon[21799]: pgmap v472: 129 pgs: 129 active+clean; 102 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 317 op/s 2024-08-20T01:11:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:47 smithi138 ceph-mon[18076]: pgmap v472: 129 pgs: 129 active+clean; 102 MiB data, 775 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.4 MiB/s wr, 317 op/s 2024-08-20T01:11:48.322 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '90be3842', quiesce took 4.9 seconds. Will release after: 14.950000000000001 2024-08-20T01:11:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:48 smithi142 ceph-mon[21799]: pgmap v473: 129 pgs: 129 active+clean; 99 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 262 op/s 2024-08-20T01:11:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:48 smithi138 ceph-mon[18076]: pgmap v473: 129 pgs: 129 active+clean; 99 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 262 op/s 2024-08-20T01:11:49.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 1 sec latency 1872.519 ms 2024-08-20T01:11:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:48 smithi167 ceph-mon[21965]: pgmap v473: 129 pgs: 129 active+clean; 99 MiB data, 778 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 262 op/s 2024-08-20T01:11:50.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 2 sec latency 2872.604 ms 2024-08-20T01:11:51.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 3 sec latency 3872.727 ms 2024-08-20T01:11:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:51 smithi167 ceph-mon[21965]: pgmap v474: 129 pgs: 129 active+clean; 86 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-08-20T01:11:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:51 smithi142 ceph-mon[21799]: pgmap v474: 129 pgs: 129 active+clean; 86 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-08-20T01:11:52.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:51 smithi138 ceph-mon[18076]: pgmap v474: 129 pgs: 129 active+clean; 86 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-08-20T01:11:52.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 4 sec latency 4872.831 ms 2024-08-20T01:11:52.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 -- 172.21.15.138:0/2595688722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054010a410 msgr2=0x7f054010a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 --2- 172.21.15.138:0/2595688722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054010a410 0x7f054010a850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f052c009900 tx=0x7f052c02a410 comp rx=0 tx=0).stop 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 -- 172.21.15.138:0/2595688722 shutdown_connections 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 --2- 172.21.15.138:0/2595688722 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 0x7f0540111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 --2- 172.21.15.138:0/2595688722 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f054010a410 0x7f054010a850 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 --2- 172.21.15.138:0/2595688722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0540104a10 0x7f0540104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 -- 172.21.15.138:0/2595688722 >> 172.21.15.138:0/2595688722 conn(0x7f0540100450 msgr2=0x7f0540102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 -- 172.21.15.138:0/2595688722 shutdown_connections 2024-08-20T01:11:52.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.686+0000 7f0545a00640 1 -- 172.21.15.138:0/2595688722 wait complete. 2024-08-20T01:11:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.687+0000 7f0545a00640 1 Processor -- start 2024-08-20T01:11:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.687+0000 7f0545a00640 1 -- start start 2024-08-20T01:11:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.687+0000 7f0545a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0540104a10 0x7f054019af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.687+0000 7f0545a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.687+0000 7f0545a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 0x7f05401a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f0545a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0540114a60 con 0x7f0540104a10 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f0545a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05401148e0 con 0x7f054010a410 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f0545a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0540114be0 con 0x7f054010ad90 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0540104a10 0x7f054019af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:58962/0 (socket says 172.21.15.138:58962) 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 -- 172.21.15.138:0/1274775050 learned_addr learned my addr 172.21.15.138:0/1274775050 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f0545000640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 0x7f05401a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 -- 172.21.15.138:0/1274775050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 msgr2=0x7f05401a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 0x7f05401a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 -- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0540104a10 msgr2=0x7f054019af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0540104a10 0x7f054019af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 -- 172.21.15.138:0/1274775050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05401a1020 con 0x7f054010a410 2024-08-20T01:11:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.688+0000 7f053f400640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f052c005ab0 tx=0x7f052c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.689+0000 7f053cc00640 1 -- 172.21.15.138:0/1274775050 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c03f070 con 0x7f054010a410 2024-08-20T01:11:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.689+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05401a12b0 con 0x7f054010a410 2024-08-20T01:11:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.689+0000 7f053cc00640 1 -- 172.21.15.138:0/1274775050 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f052c031b60 con 0x7f054010a410 2024-08-20T01:11:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.689+0000 7f053cc00640 1 -- 172.21.15.138:0/1274775050 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052c035030 con 0x7f054010a410 2024-08-20T01:11:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.689+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05401a79d0 con 0x7f054010a410 2024-08-20T01:11:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 Processor -- start 2024-08-20T01:11:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 -- start start 2024-08-20T01:11:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b676b10 0x55af7b676ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b6833c0 0x55af7b6857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af7b5ec210 con 0x55af7b676b10 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55af7b5db710 con 0x55af7b6833c0 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f054a059740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55af7b5df750 con 0x55af7b677430 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.692+0000 7f053cc00640 1 -- 172.21.15.138:0/1274775050 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f052c042070 con 0x7f054010a410 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f0545000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b676b10 0x55af7b676ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37442/0 (socket says 172.21.15.138:37442) 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 -- 172.21.15.138:0/2475966352 learned_addr learned my addr 172.21.15.138:0/2475966352 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053f400640 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b6833c0 0x55af7b6857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 -- 172.21.15.138:0/2475966352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b6833c0 msgr2=0x55af7b6857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b6833c0 0x55af7b6857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 -- 172.21.15.138:0/2475966352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b676b10 msgr2=0x55af7b676ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b676b10 0x55af7b676ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 -- 172.21.15.138:0/2475966352 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7b5a7de0 con 0x55af7b677430 2024-08-20T01:11:52.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.693+0000 7f053fe00640 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0530009cf0 tx=0x7f0530011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73c20e2c435a9d59 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f0535600640 1 -- 172.21.15.138:0/2475966352 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530004230 con 0x55af7b677430 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f0535600640 1 -- 172.21.15.138:0/2475966352 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05300043d0 con 0x55af7b677430 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f0535600640 1 -- 172.21.15.138:0/2475966352 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530019880 con 0x55af7b677430 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f053cc00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 0x7f05240b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f053cc00640 1 -- 172.21.15.138:0/1274775050 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f052c104a00 con 0x7f054010a410 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f054a059740 1 -- 172.21.15.138:0/2475966352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 msgr2=0x55af7b682e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f054a059740 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0530009cf0 tx=0x7f0530011810 comp rx=0 tx=0).stop 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.694+0000 7f0545000640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 0x7f05240b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 -- 172.21.15.138:0/2475966352 shutdown_connections 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b6833c0 0x55af7b6857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b682e80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 --2- 172.21.15.138:0/2475966352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b676b10 0x55af7b676ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 -- 172.21.15.138:0/2475966352 >> 172.21.15.138:0/2475966352 conn(0x55af7b674940 msgr2=0x55af7b675ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 -- 172.21.15.138:0/2475966352 shutdown_connections 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 -- 172.21.15.138:0/2475966352 wait complete. 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f0545000640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 0x7f05240b6280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0538005fd0 tx=0x7f0538009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 -- start start 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.695+0000 7f054a059740 1 Processor -- start 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 0x55af7b680040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b6833c0 0x55af7b6805a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55af7b514ab0 con 0x55af7b677430 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55af7b582140 con 0x55af7b676b10 2024-08-20T01:11:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55af7b6860a0 con 0x55af7b6833c0 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f054a059740 10 client.0 _ll_register_callbacks cb 0x55af7b5955d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f053fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f053fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:58976/0 (socket says 172.21.15.138:58976) 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f053f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 0x55af7b680040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f053fe00640 1 -- 172.21.15.138:0/1793927332 learned_addr learned my addr 172.21.15.138:0/1793927332 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.696+0000 7f0545000640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b6833c0 0x55af7b6805a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 -- 172.21.15.138:0/1793927332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b6833c0 msgr2=0x55af7b6805a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b6833c0 0x55af7b6805a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 -- 172.21.15.138:0/1793927332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 msgr2=0x55af7b680040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 0x55af7b680040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 -- 172.21.15.138:0/1793927332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7b583170 con 0x55af7b676b10 2024-08-20T01:11:52.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053f400640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 0x55af7b680040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f053fe00640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0530019520 tx=0x7f0530003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c221c0c063674f1e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f052a000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530027070 con 0x55af7b676b10 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f0530005240 con 0x55af7b676b10 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05300043d0 con 0x55af7b676b10 2024-08-20T01:11:52.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.697+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0530033050 con 0x55af7b676b10 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 10 client.25991 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af7b33e790 con 0x55af7b676b10 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05300043d0 con 0x55af7b676b10 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f0530004b70 con 0x55af7b676b10 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 10 client.25991 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:52.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 10 client.25991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 20 client.25991 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b692f00 0x55af7b6952f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 4 client.25991 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:11:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af7b5ee160 con 0x55af7b692f00 2024-08-20T01:11:52.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.698+0000 7f053f400640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b692f00 0x55af7b6952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.699+0000 7f053f400640 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b692f00 0x55af7b6952f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bd064842dad39fc server_cookie=160ef9858a03c95e in_seq=0 out_seq=0 2024-08-20T01:11:52.704 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.699+0000 7f052a000640 10 client.25991 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f052a000640 1 -- 172.21.15.138:0/1793927332 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af7b5ee160 con 0x55af7b692f00 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f052a000640 10 client.25991 handle_command_reply: tid=0 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 client.25991 shutdown 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 msgr2=0x55af7b682c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0530019520 tx=0x7f0530003820 comp rx=0 tx=0).stop 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 shutdown_connections 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b692f00 0x55af7b6952f0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b6833c0 0x55af7b6805a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f052a000640 0 client.25991 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b677430 0x55af7b680040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 --2- 172.21.15.138:0/1793927332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b682c00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 >> 172.21.15.138:0/1793927332 conn(0x55af7b674940 msgr2=0x55af7b67a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 shutdown_connections 2024-08-20T01:11:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 1 -- 172.21.15.138:0/1793927332 wait complete. 2024-08-20T01:11:52.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.700+0000 7f054a059740 20 client.25991 trim_cache size 0 max 16384 2024-08-20T01:11:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.715+0000 7f054a059740 1 Processor -- start 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.715+0000 7f054a059740 1 -- start start 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b69aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b69afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05401b4390 con 0x55af7b6833c0 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05401b1c00 con 0x55af7b676b10 2024-08-20T01:11:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f054a059740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05401afa50 con 0x55af7b677430 2024-08-20T01:11:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:58992/0 (socket says 172.21.15.138:58992) 2024-08-20T01:11:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 -- 172.21.15.138:0/125526124 learned_addr learned my addr 172.21.15.138:0/125526124 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f0545000640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b69afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053f400640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b69aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 -- 172.21.15.138:0/125526124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 msgr2=0x55af7b69aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b69aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 -- 172.21.15.138:0/125526124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 msgr2=0x55af7b69afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b69afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053fe00640 1 -- 172.21.15.138:0/125526124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af7b3b7620 con 0x55af7b676b10 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.716+0000 7f053f400640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b69aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:11:52.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.717+0000 7f053fe00640 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0530019590 tx=0x7f0530004560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1d23236f6787315 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.717+0000 7f0534c00640 1 -- 172.21.15.138:0/125526124 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530030070 con 0x55af7b676b10 2024-08-20T01:11:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.717+0000 7f0534c00640 1 -- 172.21.15.138:0/125526124 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530005470 con 0x55af7b676b10 2024-08-20T01:11:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.717+0000 7f0534c00640 1 -- 172.21.15.138:0/125526124 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530027070 con 0x55af7b676b10 2024-08-20T01:11:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- 172.21.15.138:0/125526124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 msgr2=0x55af7b67f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0530019590 tx=0x7f0530004560 comp rx=0 tx=0).stop 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- 172.21.15.138:0/125526124 shutdown_connections 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b69afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b69aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 --2- 172.21.15.138:0/125526124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b67f8b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- 172.21.15.138:0/125526124 >> 172.21.15.138:0/125526124 conn(0x55af7b674940 msgr2=0x55af7b681e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- 172.21.15.138:0/125526124 shutdown_connections 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- 172.21.15.138:0/125526124 wait complete. 2024-08-20T01:11:52.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 -- start start 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.718+0000 7f054a059740 1 Processor -- start 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b688ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b689050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f05401b00c0 con 0x55af7b6833c0 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f05401b1530 con 0x55af7b676b10 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f05401b0790 con 0x55af7b677430 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f054a059740 10 client.0 _ll_register_callbacks cb 0x7f0538009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f053fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b688ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43104/0 (socket says 172.21.15.138:43104) 2024-08-20T01:11:52.722 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 -- 172.21.15.138:0/3368254650 learned_addr learned my addr 172.21.15.138:0/3368254650 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f053f400640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b689050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 -- 172.21.15.138:0/3368254650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 msgr2=0x55af7b689050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b689050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 -- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 msgr2=0x55af7b688ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b688ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.719+0000 7f0545000640 1 -- 172.21.15.138:0/3368254650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af7b6f6160 con 0x55af7b6833c0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f0545000640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0538004240 tx=0x7f0538004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c112abb45d55fad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f052a000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053802e060 con 0x55af7b6833c0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f053800f9e0 con 0x55af7b6833c0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0530005470 con 0x55af7b6833c0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0538035050 con 0x55af7b6833c0 2024-08-20T01:11:52.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f054a059740 10 client.15945 fetch_fsmap learned FSMap version 31 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55af7b6fd580 con 0x55af7b6833c0 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.720+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0530005470 con 0x55af7b6833c0 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f053801e400 con 0x55af7b6833c0 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 10 client.15945 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 10 client.15945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 20 client.15945 populate_metadata read hostname 'smithi138' 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b6fdd00 0x55af7b700120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 4 client.15945 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af7b682f90 con 0x55af7b6fdd00 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.721+0000 7f053fe00640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b6fdd00 0x55af7b700120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.722+0000 7f053fe00640 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b6fdd00 0x55af7b700120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=799a6733a9733603 server_cookie=b1cc6ce780d4b4fc in_seq=0 out_seq=0 2024-08-20T01:11:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.722+0000 7f052a000640 10 client.15945 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.741+0000 7f052a000640 1 -- 172.21.15.138:0/3368254650 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af7b682f90 con 0x55af7b6fdd00 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.741+0000 7f052a000640 10 client.15945 handle_command_reply: tid=0 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.741+0000 7f054a059740 1 client.15945 shutdown 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.741+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 msgr2=0x55af7b6f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.741+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0538004240 tx=0x7f0538004a40 comp rx=0 tx=0).stop 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 shutdown_connections 2024-08-20T01:11:52.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55af7b6fdd00 0x55af7b700120 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55af7b6833c0 0x55af7b6f5a20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55af7b677430 0x55af7b689050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f052a000640 0 client.15945 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 --2- 172.21.15.138:0/3368254650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af7b676b10 0x55af7b688ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 >> 172.21.15.138:0/3368254650 conn(0x55af7b674940 msgr2=0x55af7b681670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 shutdown_connections 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 1 -- 172.21.15.138:0/3368254650 wait complete. 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.742+0000 7f054a059740 20 client.15945 trim_cache size 0 max 16384 2024-08-20T01:11:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:11:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:11:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:11:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:11:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 msgr2=0x7f05240b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 0x7f05240b6280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0538005fd0 tx=0x7f0538009500 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 msgr2=0x7f054019b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f052c005ab0 tx=0x7f052c003c10 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 shutdown_connections 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f05240b3dc0 0x7f05240b6280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f054010ad90 0x7f05401a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f054010a410 0x7f054019b4c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 --2- 172.21.15.138:0/1274775050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0540104a10 0x7f054019af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 >> 172.21.15.138:0/1274775050 conn(0x7f0540100450 msgr2=0x7f0540102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 shutdown_connections 2024-08-20T01:11:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:11:52.743+0000 7f0545a00640 1 -- 172.21.15.138:0/1274775050 wait complete. 2024-08-20T01:11:53.135 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 5 sec latency 5872.969 ms 2024-08-20T01:11:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:53 smithi167 ceph-mon[21965]: pgmap v475: 129 pgs: 129 active+clean; 84 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-08-20T01:11:53.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:53 smithi142 ceph-mon[21799]: pgmap v475: 129 pgs: 129 active+clean; 84 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-08-20T01:11:54.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:53 smithi138 ceph-mon[18076]: pgmap v475: 129 pgs: 129 active+clean; 84 MiB data, 792 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-08-20T01:11:54.134 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 6 sec latency 6873.110 ms 2024-08-20T01:11:55.135 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 7 sec latency 7873.247 ms 2024-08-20T01:11:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:55 smithi167 ceph-mon[21965]: pgmap v476: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:11:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:55 smithi142 ceph-mon[21799]: pgmap v476: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:11:56.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:55 smithi138 ceph-mon[18076]: pgmap v476: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:11:56.135 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 8 sec latency 8873.451 ms 2024-08-20T01:11:57.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 9 sec latency 9873.582 ms 2024-08-20T01:11:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:57 smithi167 ceph-mon[21965]: pgmap v477: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 318 KiB/s wr, 87 op/s 2024-08-20T01:11:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:57 smithi142 ceph-mon[21799]: pgmap v477: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 318 KiB/s wr, 87 op/s 2024-08-20T01:11:58.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:57 smithi138 ceph-mon[18076]: pgmap v477: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 318 KiB/s wr, 87 op/s 2024-08-20T01:11:58.135 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 10 sec latency 10873.711 ms 2024-08-20T01:11:59.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 11 sec latency 11873.808 ms 2024-08-20T01:11:59.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:11:59 smithi167 ceph-mon[21965]: pgmap v478: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 206 KiB/s wr, 59 op/s 2024-08-20T01:11:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:11:59 smithi142 ceph-mon[21799]: pgmap v478: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 206 KiB/s wr, 59 op/s 2024-08-20T01:12:00.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:11:59 smithi138 ceph-mon[18076]: pgmap v478: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 206 KiB/s wr, 59 op/s 2024-08-20T01:12:00.135 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 12 sec latency 12873.962 ms 2024-08-20T01:12:01.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 13 sec latency 13874.105 ms 2024-08-20T01:12:01.926 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:01 smithi138 ceph-mon[18076]: pgmap v479: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 129 KiB/s wr, 52 op/s 2024-08-20T01:12:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:01 smithi167 ceph-mon[21965]: pgmap v479: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 129 KiB/s wr, 52 op/s 2024-08-20T01:12:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:01 smithi142 ceph-mon[21799]: pgmap v479: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 129 KiB/s wr, 52 op/s 2024-08-20T01:12:02.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 14 sec latency 14874.175 ms 2024-08-20T01:12:03.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 15 sec latency 15874.241 ms 2024-08-20T01:12:03.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:12:03.658 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:03.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:03 smithi138 ceph-mon[18076]: pgmap v480: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-08-20T01:12:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:03 smithi167 ceph-mon[21965]: pgmap v480: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-08-20T01:12:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:03 smithi142 ceph-mon[21799]: pgmap v480: 129 pgs: 129 active+clean; 84 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 62 KiB/s wr, 26 op/s 2024-08-20T01:12:04.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 16 sec latency 16874.315 ms 2024-08-20T01:12:04.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 -- 172.21.15.138:0/2733559493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6938075b10 msgr2=0x7f693810eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:04.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 --2- 172.21.15.138:0/2733559493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6938075b10 0x7f693810eb80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6920009900 tx=0x7f692002a410 comp rx=0 tx=0).stop 2024-08-20T01:12:04.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 -- 172.21.15.138:0/2733559493 shutdown_connections 2024-08-20T01:12:04.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 --2- 172.21.15.138:0/2733559493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 0x7f69381114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 --2- 172.21.15.138:0/2733559493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6938075b10 0x7f693810eb80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 --2- 172.21.15.138:0/2733559493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f69380751f0 0x7f69380755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 -- 172.21.15.138:0/2733559493 >> 172.21.15.138:0/2733559493 conn(0x7f69380fe500 msgr2=0x7f6938100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:04.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 -- 172.21.15.138:0/2733559493 shutdown_connections 2024-08-20T01:12:04.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.277+0000 7f693dc00640 1 -- 172.21.15.138:0/2733559493 wait complete. 2024-08-20T01:12:04.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 Processor -- start 2024-08-20T01:12:04.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 -- start start 2024-08-20T01:12:04.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:04.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 0x7f693819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 0x7f69381a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6938114ca0 con 0x7f69380751f0 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6938114b20 con 0x7f693810f0c0 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.278+0000 7f693dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6938114e20 con 0x7f6938075b10 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f6936a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:04.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 0x7f693819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f6936a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54432/0 (socket says 172.21.15.138:54432) 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f6936a00640 1 -- 172.21.15.138:0/2391173321 learned_addr learned my addr 172.21.15.138:0/2391173321 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f6937400640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 0x7f69381a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 -- 172.21.15.138:0/2391173321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 msgr2=0x7f69381a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 0x7f69381a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 -- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 msgr2=0x7f693819b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f692fe00640 1 -- 172.21.15.138:0/2391173321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69381a12a0 con 0x7f6938075b10 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.279+0000 7f6936a00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f692fe00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 0x7f693819b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f692002a930 tx=0x7f6920031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692003f030 con 0x7f6938075b10 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69381a1530 con 0x7f6938075b10 2024-08-20T01:12:04.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6920004490 con 0x7f6938075b10 2024-08-20T01:12:04.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69381a7c50 con 0x7f6938075b10 2024-08-20T01:12:04.290 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.280+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920049060 con 0x7f6938075b10 2024-08-20T01:12:04.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.282+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69380755d0 con 0x7f6938075b10 2024-08-20T01:12:04.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.284+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6920042030 con 0x7f6938075b10 2024-08-20T01:12:04.291 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.285+0000 7f692f400640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 0x7f691c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.285+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f69201044d0 con 0x7f6938075b10 2024-08-20T01:12:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.286+0000 7f6936a00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 0x7f691c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.286+0000 7f6936a00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 0x7f691c0b6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6928011500 tx=0x7f6928005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:04.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.287+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f69200cade0 con 0x7f6938075b10 2024-08-20T01:12:04.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.514+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f693819c320 con 0x7f6938075b10 2024-08-20T01:12:04.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.515+0000 7f692f400640 1 -- 172.21.15.138:0/2391173321 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11935 (secure 0 0 0) 0x7f69200cf330 con 0x7f6938075b10 2024-08-20T01:12:04.514 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:12:04.515 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":31,"btime":"2024-08-20T01:11:09:621725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:11:09.220799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:12:04.515 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 31 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 msgr2=0x7f691c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 0x7f691c0b6200 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6928011500 tx=0x7f6928005f90 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 msgr2=0x7f693819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 0x7f693819b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f692002a930 tx=0x7f6920031a20 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 shutdown_connections 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f691c0b3d40 0x7f691c0b6200 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f693810f0c0 0x7f69381a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6938075b10 0x7f693819b730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 --2- 172.21.15.138:0/2391173321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69380751f0 0x7f693819b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.518+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 >> 172.21.15.138:0/2391173321 conn(0x7f69380fe500 msgr2=0x7f69380ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:04.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.519+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 shutdown_connections 2024-08-20T01:12:04.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:04.519+0000 7f693dc00640 1 -- 172.21.15.138:0/2391173321 wait complete. 2024-08-20T01:12:04.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2391173321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2391173321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2391173321' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:05.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 17 sec latency 17874.428 ms 2024-08-20T01:12:05.448 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 90be3842 --release --await' 2024-08-20T01:12:05.448 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 90be3842 --release --await 2024-08-20T01:12:05.832 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 -- 172.21.15.138:0/3449541780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c108d70 msgr2=0x7f8f2c10f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 --2- 172.21.15.138:0/3449541780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c108d70 0x7f8f2c10f570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ca0 tx=0x7f8f2402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 -- 172.21.15.138:0/3449541780 shutdown_connections 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 --2- 172.21.15.138:0/3449541780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c108d70 0x7f8f2c10f570 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 --2- 172.21.15.138:0/3449541780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 --2- 172.21.15.138:0/3449541780 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 -- 172.21.15.138:0/3449541780 >> 172.21.15.138:0/3449541780 conn(0x7f8f2c0fe430 msgr2=0x7f8f2c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 -- 172.21.15.138:0/3449541780 shutdown_connections 2024-08-20T01:12:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.874+0000 7f8f32600640 1 -- 172.21.15.138:0/3449541780 wait complete. 2024-08-20T01:12:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.875+0000 7f8f32600640 1 Processor -- start 2024-08-20T01:12:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.875+0000 7f8f32600640 1 -- start start 2024-08-20T01:12:05.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c19b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112a80 con 0x7f8f2c108d70 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112900 con 0x7f8f2c1083f0 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f32600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c112c00 con 0x7f8f2c1029f0 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54458/0 (socket says 172.21.15.138:54458) 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2be00640 1 -- 172.21.15.138:0/1793819861 learned_addr learned my addr 172.21.15.138:0/1793819861 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c19b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 -- 172.21.15.138:0/1793819861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c1083f0 msgr2=0x7f8f2c19b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c19b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 -- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 msgr2=0x7f8f2c1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2b400640 1 -- 172.21.15.138:0/1793819861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f2c1a1140 con 0x7f8f2c1029f0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.876+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c19b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00e9f0 tx=0x7f8f1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f23e00640 1 -- 172.21.15.138:0/1793819861 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00cd80 con 0x7f8f2c1029f0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f23e00640 1 -- 172.21.15.138:0/1793819861 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0035f0 con 0x7f8f2c1029f0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f2c1a1430 con 0x7f8f2c1029f0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f23e00640 1 -- 172.21.15.138:0/1793819861 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c010690 con 0x7f8f2c1029f0 2024-08-20T01:12:05.877 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.877+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2c1a7af0 con 0x7f8f2c1029f0 2024-08-20T01:12:05.878 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.880+0000 7f8f355d2740 1 Processor -- start 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.880+0000 7f8f355d2740 1 -- start start 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.880+0000 7f8f23e00640 1 -- 172.21.15.138:0/1793819861 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8f1c0037a0 con 0x7f8f2c1029f0 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.880+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1e9ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f355d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1e10210 con 0x55c3b1e9a930 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f355d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1dff710 con 0x55c3b1e9b250 2024-08-20T01:12:05.879 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f355d2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1e03750 con 0x55c3b1ea71e0 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1e9ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:48956/0 (socket says 172.21.15.138:48956) 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 -- 172.21.15.138:0/4228807011 learned_addr learned my addr 172.21.15.138:0/4228807011 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1e9ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54468/0 (socket says 172.21.15.138:54468) 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2b400640 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 -- 172.21.15.138:0/4228807011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 msgr2=0x55c3b1ea6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 -- 172.21.15.138:0/4228807011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 msgr2=0x55c3b1e9ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.881 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1e9ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.881+0000 7f8f2aa00640 1 -- 172.21.15.138:0/4228807011 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b1dcbde0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009c90 tx=0x7f8f18011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=318d67e12f08e02a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f20c00640 1 -- 172.21.15.138:0/4228807011 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1801e070 con 0x55c3b1ea71e0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f23e00640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 0x7f8f100b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f20c00640 1 -- 172.21.15.138:0/4228807011 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f18019ae0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f20c00640 1 -- 172.21.15.138:0/4228807011 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1800e940 con 0x55c3b1ea71e0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f23e00640 1 -- 172.21.15.138:0/1793819861 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8f1c0e46d0 con 0x7f8f2c1029f0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.882+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 0x7f8f100b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 -- 172.21.15.138:0/4228807011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 msgr2=0x55c3b1ea95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8f18009c90 tx=0x7f8f18011810 comp rx=0 tx=0).stop 2024-08-20T01:12:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 -- 172.21.15.138:0/4228807011 shutdown_connections 2024-08-20T01:12:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea95d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 --2- 172.21.15.138:0/4228807011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1e9ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 -- 172.21.15.138:0/4228807011 >> 172.21.15.138:0/4228807011 conn(0x55c3b1e98760 msgr2=0x55c3b1e99900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 0x7f8f100b6200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f24004af0 tx=0x7f8f24005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 -- 172.21.15.138:0/4228807011 shutdown_connections 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.883+0000 7f8f355d2740 1 -- 172.21.15.138:0/4228807011 wait complete. 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 -- start start 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 Processor -- start 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1d38ab0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1da6140 con 0x55c3b1e9a930 2024-08-20T01:12:05.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c3b1ea9ec0 con 0x55c3b1e9b250 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.884+0000 7f8f355d2740 10 client.0 _ll_register_callbacks cb 0x55c3b1db95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54476/0 (socket says 172.21.15.138:54476) 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 -- 172.21.15.138:0/1676013192 learned_addr learned my addr 172.21.15.138:0/1676013192 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 -- 172.21.15.138:0/1676013192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9b250 msgr2=0x55c3b1ea3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 -- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 msgr2=0x55c3b1ea6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 -- 172.21.15.138:0/1676013192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b1da7170 con 0x55c3b1ea71e0 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:05.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f24031a10 tx=0x7f8f2402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfed0fbf49985dea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.885+0000 7f8f15600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2402f390 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8f24004250 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f18019ae0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f24004450 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f355d2740 10 client.15963 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3b1b62790 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f18019ae0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.886+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8f24052070 con 0x55c3b1ea71e0 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 10 client.15963 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 20 client.15963 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:05.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1eb6e80 0x55c3b1eb92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 4 client.15963 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3b1e12160 con 0x55c3b1eb6e80 2024-08-20T01:12:05.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.887+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1eb6e80 0x55c3b1eb92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.888+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1eb6e80 0x55c3b1eb92e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2194a3bac4c06927 server_cookie=422781aad6c7053d in_seq=0 out_seq=0 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.888+0000 7f8f15600640 10 client.15963 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.888+0000 7f8f15600640 1 -- 172.21.15.138:0/1676013192 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c3b1e12160 con 0x55c3b1eb6e80 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.888+0000 7f8f15600640 10 client.15963 handle_command_reply: tid=0 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.888+0000 7f8f355d2740 1 client.15963 shutdown 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 msgr2=0x55c3b1ea4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8f24031a10 tx=0x7f8f2402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 shutdown_connections 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1eb6e80 0x55c3b1eb92e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ea4390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9b250 0x55c3b1ea3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1676013192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 >> 172.21.15.138:0/1676013192 conn(0x55c3b1e98760 msgr2=0x55c3b1e9e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f15600640 0 client.15963 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 shutdown_connections 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 1 -- 172.21.15.138:0/1676013192 wait complete. 2024-08-20T01:12:05.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.889+0000 7f8f355d2740 20 client.15963 trim_cache size 0 max 16384 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.904+0000 7f8f355d2740 1 Processor -- start 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.904+0000 7f8f355d2740 1 -- start start 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ebf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1b44e0 con 0x55c3b1e9b250 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1b1d50 con 0x55c3b1ea71e0 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f355d2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1afba0 con 0x55c3b1e9a930 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54486/0 (socket says 172.21.15.138:54486) 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2aa00640 1 -- 172.21.15.138:0/1465414083 learned_addr learned my addr 172.21.15.138:0/1465414083 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2be00640 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ebf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 -- 172.21.15.138:0/1465414083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1ea71e0 msgr2=0x55c3b1ebf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ebf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 -- 172.21.15.138:0/1465414083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 msgr2=0x55c3b1ebebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2b400640 1 -- 172.21.15.138:0/1465414083 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3b1e9b080 con 0x55c3b1e9a930 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.905+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f2b400640 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea36c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e1a70 tx=0x7f8f1c0e6b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f4ce5a905c091ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f17e00640 1 -- 172.21.15.138:0/1465414083 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c08be80 con 0x55c3b1e9a930 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f17e00640 1 -- 172.21.15.138:0/1465414083 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0aaa40 con 0x55c3b1e9a930 2024-08-20T01:12:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f17e00640 1 -- 172.21.15.138:0/1465414083 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0a71c0 con 0x55c3b1e9a930 2024-08-20T01:12:05.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f355d2740 1 -- 172.21.15.138:0/1465414083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 msgr2=0x55c3b1ea36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.906+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea36c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0e1a70 tx=0x7f8f1c0e6b20 comp rx=0 tx=0).stop 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 -- 172.21.15.138:0/1465414083 shutdown_connections 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1ea71e0 0x55c3b1ebf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9b250 0x55c3b1ebebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 --2- 172.21.15.138:0/1465414083 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1e9a930 0x55c3b1ea36c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 -- 172.21.15.138:0/1465414083 >> 172.21.15.138:0/1465414083 conn(0x55c3b1e98760 msgr2=0x55c3b1e99890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 -- 172.21.15.138:0/1465414083 shutdown_connections 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 -- 172.21.15.138:0/1465414083 wait complete. 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 -- start start 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.907+0000 7f8f355d2740 1 Processor -- start 2024-08-20T01:12:05.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1eac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1f1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1b0210 con 0x55c3b1e9a930 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1b1680 con 0x55c3b1e9b250 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8f2c1b08e0 con 0x55c3b1ea71e0 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f355d2740 10 client.0 _ll_register_callbacks cb 0x7f8f24009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54498/0 (socket says 172.21.15.138:54498) 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 -- 172.21.15.138:0/96887825 learned_addr learned my addr 172.21.15.138:0/96887825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1eac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2be00640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1f1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 -- 172.21.15.138:0/96887825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 msgr2=0x55c3b1f1fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1f1fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 -- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 msgr2=0x55c3b1eac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1eac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.908+0000 7f8f2b400640 1 -- 172.21.15.138:0/96887825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3b1bdb620 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f2b400640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c6a0 tx=0x7f8f1c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d263e395f1986 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0c4070 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8f1c0a71c0 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f1c0aaa40 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8f1c086030 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 10 client.15969 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3b1f21020 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f1c0aaa40 con 0x55c3b1e9a930 2024-08-20T01:12:05.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8f1c089020 con 0x55c3b1e9a930 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 10 client.15969 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 20 client.15969 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1f21710 0x55c3b1f23b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 4 client.15969 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.909+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c3b1e0dc00 con 0x55c3b1f21710 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.910+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1f21710 0x55c3b1f23b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.910+0000 7f8f2aa00640 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1f21710 0x55c3b1f23b30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=878177edebcdcb56 server_cookie=f2e8e04362172950 in_seq=0 out_seq=0 2024-08-20T01:12:05.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.910+0000 7f8f15600640 10 client.15969 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:05.926 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f15600640 1 -- 172.21.15.138:0/96887825 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c3b1e0dc00 con 0x55c3b1f21710 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f15600640 10 client.15969 handle_command_reply: tid=0 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 client.15969 shutdown 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 msgr2=0x55c3b1eac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00c6a0 tx=0x7f8f1c003c10 comp rx=0 tx=0).stop 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 shutdown_connections 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c3b1f21710 0x55c3b1f23b30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c3b1ea71e0 0x55c3b1f1fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3b1e9b250 0x55c3b1eac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 --2- 172.21.15.138:0/96887825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c3b1e9a930 0x55c3b1eac2c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f15600640 0 client.15969 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 >> 172.21.15.138:0/96887825 conn(0x55c3b1e98760 msgr2=0x55c3b1ea5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 shutdown_connections 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 1 -- 172.21.15.138:0/96887825 wait complete. 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.928+0000 7f8f355d2740 20 client.15969 trim_cache size 0 max 16384 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:12:05.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 msgr2=0x7f8f100b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 0x7f8f100b6200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f24004af0 tx=0x7f8f24005d80 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 msgr2=0x7f8f2c19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c19b0a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00e9f0 tx=0x7f8f1c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 shutdown_connections 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8f100b3d40 0x7f8f100b6200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f2c108d70 0x7f8f2c1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f2c1083f0 0x7f8f2c19b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 --2- 172.21.15.138:0/1793819861 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8f2c1029f0 0x7f8f2c19b0a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:05.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 >> 172.21.15.138:0/1793819861 conn(0x7f8f2c0fe430 msgr2=0x7f8f2c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 shutdown_connections 2024-08-20T01:12:05.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:05.929+0000 7f8f32600640 1 -- 172.21.15.138:0/1793819861 wait complete. 2024-08-20T01:12:05.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:05 smithi167 ceph-mon[21965]: pgmap v481: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-08-20T01:12:05.973 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:05 smithi138 ceph-mon[18076]: pgmap v481: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-08-20T01:12:05.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:05 smithi142 ceph-mon[21799]: pgmap v481: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 62 KiB/s wr, 31 op/s 2024-08-20T01:12:06.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 136818 0.00 MB/sec execute 18 sec latency 18874.518 ms 2024-08-20T01:12:06.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 -- 172.21.15.138:0/200584013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a410a6b0 msgr2=0x7f95a410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 --2- 172.21.15.138:0/200584013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a410a6b0 0x7f95a410aaf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f959002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 -- 172.21.15.138:0/200584013 shutdown_connections 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 --2- 172.21.15.138:0/200584013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a4111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 --2- 172.21.15.138:0/200584013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a410a6b0 0x7f95a410aaf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 --2- 172.21.15.138:0/200584013 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a4104ca0 0x7f95a4105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 -- 172.21.15.138:0/200584013 >> 172.21.15.138:0/200584013 conn(0x7f95a4100730 msgr2=0x7f95a4102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 -- 172.21.15.138:0/200584013 shutdown_connections 2024-08-20T01:12:06.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.460+0000 7f95a9a00640 1 -- 172.21.15.138:0/200584013 wait complete. 2024-08-20T01:12:06.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 Processor -- start 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 -- start start 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a410a6b0 0x7f95a419b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a41a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114cf0 con 0x7f95a4104ca0 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114b70 con 0x7f95a410b030 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.461+0000 7f95a9a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4114e70 con 0x7f95a410a6b0 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54512/0 (socket says 172.21.15.138:54512) 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 -- 172.21.15.138:0/1742082657 learned_addr learned my addr 172.21.15.138:0/1742082657 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2000640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a410a6b0 0x7f95a419b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a3400640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a41a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 -- 172.21.15.138:0/1742082657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a410a6b0 msgr2=0x7f95a419b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a410a6b0 0x7f95a419b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 -- 172.21.15.138:0/1742082657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 msgr2=0x7f95a41a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a41a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 -- 172.21.15.138:0/1742082657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41a12b0 con 0x7f95a4104ca0 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a3400640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.462+0000 7f95a2a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9f0 tx=0x7f959400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.463+0000 7f959b400640 1 -- 172.21.15.138:0/1742082657 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400cd80 con 0x7f95a4104ca0 2024-08-20T01:12:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.463+0000 7f959b400640 1 -- 172.21.15.138:0/1742082657 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95940108e0 con 0x7f95a4104ca0 2024-08-20T01:12:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.463+0000 7f959b400640 1 -- 172.21.15.138:0/1742082657 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594018980 con 0x7f95a4104ca0 2024-08-20T01:12:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.463+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41a15a0 con 0x7f95a4104ca0 2024-08-20T01:12:06.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.463+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a41a7c60 con 0x7f95a4104ca0 2024-08-20T01:12:06.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 Processor -- start 2024-08-20T01:12:06.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 -- start start 2024-08-20T01:12:06.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f959b400640 1 -- 172.21.15.138:0/1742082657 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9594010430 con 0x7f95a4104ca0 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d805060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d807990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56274d79a670 con 0x56274d8055a0 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56274d76ed70 con 0x56274d7f8cf0 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.466+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56274d75e270 con 0x56274d7f9610 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d805060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d807990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47492/0 (socket says 172.21.15.138:47492) 2024-08-20T01:12:06.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 -- 172.21.15.138:0/2013171808 learned_addr learned my addr 172.21.15.138:0/2013171808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 -- 172.21.15.138:0/2013171808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 msgr2=0x56274d805060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d805060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 -- 172.21.15.138:0/2013171808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 msgr2=0x56274d807990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d807990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a3400640 1 -- 172.21.15.138:0/2013171808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274d6fa870 con 0x56274d7f8cf0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.467+0000 7f95a2a00640 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d805060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f95a3400640 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fd0 tx=0x7f959c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f4b66232128f27 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f959b400640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 0x7f95880b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f958fe00640 1 -- 172.21.15.138:0/2013171808 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c011700 con 0x56274d7f8cf0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f958fe00640 1 -- 172.21.15.138:0/2013171808 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c011d50 con 0x56274d7f8cf0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f958fe00640 1 -- 172.21.15.138:0/2013171808 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00e3c0 con 0x56274d7f8cf0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f959b400640 1 -- 172.21.15.138:0/1742082657 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f95940e46a0 con 0x7f95a4104ca0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.468+0000 7f95a2a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 0x7f95880b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2013171808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 msgr2=0x56274d7f90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f959c009fd0 tx=0x7f959c010860 comp rx=0 tx=0).stop 2024-08-20T01:12:06.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95a2a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 0x7f95880b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95940e1b50 tx=0x7f95940ea550 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2013171808 shutdown_connections 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d807990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d805060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2013171808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f8cf0 0x56274d7f90d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2013171808 >> 172.21.15.138:0/2013171808 conn(0x56274d7f6b20 msgr2=0x56274d7f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2013171808 shutdown_connections 2024-08-20T01:12:06.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.469+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2013171808 wait complete. 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 -- start start 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 Processor -- start 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f8cf0 0x56274d804db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d802740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56274d6e4ff0 con 0x56274d7f8cf0 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56274d7622b0 con 0x56274d8055a0 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56274d5c0bb0 con 0x56274d7f9610 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.470+0000 7f95ac5c5740 10 client.0 _ll_register_callbacks cb 0x56274d469d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f8cf0 0x56274d804db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.472 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:49016/0 (socket says 172.21.15.138:49016) 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 -- 172.21.15.138:0/3983262819 learned_addr learned my addr 172.21.15.138:0/3983262819 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a3400640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d802740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 -- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 msgr2=0x56274d802740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d802740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 -- 172.21.15.138:0/3983262819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f8cf0 msgr2=0x56274d804db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f8cf0 0x56274d804db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 -- 172.21.15.138:0/3983262819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274d4bdb00 con 0x56274d7f9610 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f95a2000640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f95900314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25cae116bdd43e11 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.471+0000 7f958cc00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035040 con 0x56274d7f9610 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c011d50 con 0x56274d7f9610 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9590031790 con 0x56274d7f9610 2024-08-20T01:12:06.473 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f959004f050 con 0x56274d7f9610 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f95ac5c5740 10 client.25987 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56274d534960 con 0x56274d7f9610 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.472+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f959c011d50 con 0x56274d7f9610 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f9590004750 con 0x56274d7f9610 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 10 client.25987 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 10 client.25987 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 20 client.25987 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d814b80 0x56274d816f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 4 client.25987 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56274d770cc0 con 0x56274d814b80 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.473+0000 7f95a3400640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d814b80 0x56274d816f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.474+0000 7f95a3400640 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d814b80 0x56274d816f90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae2198d18724cd3 server_cookie=89d16309fc3717cd in_seq=0 out_seq=0 2024-08-20T01:12:06.474 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.474+0000 7f958cc00640 10 client.25987 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.474+0000 7f958cc00640 1 -- 172.21.15.138:0/3983262819 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56274d770cc0 con 0x56274d814b80 2024-08-20T01:12:06.475 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.474+0000 7f958cc00640 10 client.25987 handle_command_reply: tid=0 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 client.25987 shutdown 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 msgr2=0x56274d802200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9590009900 tx=0x7f95900314a0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 shutdown_connections 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d814b80 0x56274d816f90 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d802740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f9610 0x56274d802200 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3983262819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f8cf0 0x56274d804db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 >> 172.21.15.138:0/3983262819 conn(0x56274d7f6b20 msgr2=0x56274d7fc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f958cc00640 0 client.25987 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 shutdown_connections 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3983262819 wait complete. 2024-08-20T01:12:06.476 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.475+0000 7f95ac5c5740 20 client.25987 trim_cache size 0 max 16384 2024-08-20T01:12:06.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 Processor -- start 2024-08-20T01:12:06.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 -- start start 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d8111c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d80d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b4610 con 0x56274d7f9610 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b1e80 con 0x56274d8055a0 2024-08-20T01:12:06.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41afcd0 con 0x56274d7f8cf0 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95a2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95a2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54536/0 (socket says 172.21.15.138:54536) 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95a2000640 1 -- 172.21.15.138:0/3636009244 learned_addr learned my addr 172.21.15.138:0/3636009244 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95a3400640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d8111c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.493+0000 7f95a3400640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d80d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 -- 172.21.15.138:0/3636009244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 msgr2=0x56274d8111c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d8111c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 -- 172.21.15.138:0/3636009244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 msgr2=0x56274d80d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d80d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 -- 172.21.15.138:0/3636009244 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56274d802a80 con 0x56274d7f9610 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a3400640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d80d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:12:06.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f95a2000640 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0144af1486242f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f958f400640 1 -- 172.21.15.138:0/3636009244 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590058070 con 0x56274d7f9610 2024-08-20T01:12:06.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f958f400640 1 -- 172.21.15.138:0/3636009244 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95900048c0 con 0x56274d7f9610 2024-08-20T01:12:06.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.494+0000 7f958f400640 1 -- 172.21.15.138:0/3636009244 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035030 con 0x56274d7f9610 2024-08-20T01:12:06.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3636009244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 msgr2=0x56274d7f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9590006010 tx=0x7f959002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3636009244 shutdown_connections 2024-08-20T01:12:06.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d8055a0 0x56274d80d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d7f9610 0x56274d7f5b30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/3636009244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d8111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3636009244 >> 172.21.15.138:0/3636009244 conn(0x56274d7f6b20 msgr2=0x56274d7f5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3636009244 shutdown_connections 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- 172.21.15.138:0/3636009244 wait complete. 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.495+0000 7f95ac5c5740 1 -- start start 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 Processor -- start 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d80bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d878880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b0340 con 0x56274d8055a0 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b17b0 con 0x56274d7f9610 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41b0a10 con 0x56274d7f8cf0 2024-08-20T01:12:06.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95ac5c5740 10 client.0 _ll_register_callbacks cb 0x7f9590009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95a3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d878880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95a2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95a2000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47510/0 (socket says 172.21.15.138:47510) 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95a2000640 1 -- 172.21.15.138:0/2206543597 learned_addr learned my addr 172.21.15.138:0/2206543597 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.496+0000 7f95a3400640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d80bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 -- 172.21.15.138:0/2206543597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 msgr2=0x56274d80bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d80bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 -- 172.21.15.138:0/2206543597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 msgr2=0x56274d878880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d878880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 -- 172.21.15.138:0/2206543597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56274d878fc0 con 0x56274d7f9610 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95a2000640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9590056040 tx=0x7f9590004fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e24c5f0e7ff99d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f958cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9590035070 con 0x56274d7f9610 2024-08-20T01:12:06.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9590031b10 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.497+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95900048c0 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9590065050 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 10 client.26036 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56274d8803f0 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95900048c0 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f959005b030 con 0x56274d7f9610 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 10 client.26036 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 10 client.26036 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 20 client.26036 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d880b00 0x56274d882f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 4 client.26036 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "90be3842", "release": true, "await": true}] 2024-08-20T01:12:06.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.498+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "90be3842", "release": true, "await": true}) -- 0x56274d76c760 con 0x56274d880b00 2024-08-20T01:12:06.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.499+0000 7f95a3400640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d880b00 0x56274d882f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:06.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.499+0000 7f95a3400640 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d880b00 0x56274d882f20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c992fa607aa068f server_cookie=87d3562e68330bcf in_seq=0 out_seq=0 2024-08-20T01:12:06.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.499+0000 7f958cc00640 10 client.26036 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.505+0000 7f958cc00640 1 -- 172.21.15.138:0/2206543597 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56274d76c760 con 0x56274d880b00 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.505+0000 7f958cc00640 10 client.26036 handle_command_reply: tid=0 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 client.26036 shutdown 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 msgr2=0x56274d80c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9590056040 tx=0x7f9590004fa0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 shutdown_connections 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56274d880b00 0x56274d882f20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56274d8055a0 0x56274d878880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56274d7f9610 0x56274d80c1c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f958cc00640 0 client.26036 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 --2- 172.21.15.138:0/2206543597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56274d7f8cf0 0x56274d80bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 >> 172.21.15.138:0/2206543597 conn(0x56274d7f6b20 msgr2=0x56274d7f5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 shutdown_connections 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.506+0000 7f95ac5c5740 1 -- 172.21.15.138:0/2206543597 wait complete. 2024-08-20T01:12:06.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95ac5c5740 20 client.26036 trim_cache size 0 max 16384 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 31, 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 20, 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "90be3842": { 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 20, 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:12:06.506 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 139.8, 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:06.507 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 msgr2=0x7f95880b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 0x7f95880b6280 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f95940e1b50 tx=0x7f95940ea550 comp rx=0 tx=0).stop 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 msgr2=0x7f95a419b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:06.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9f0 tx=0x7f959400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 shutdown_connections 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95880b3dc0 0x7f95880b6280 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95a410b030 0x7f95a41a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f95a410a6b0 0x7f95a419b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 --2- 172.21.15.138:0/1742082657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95a4104ca0 0x7f95a419b200 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.507+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 >> 172.21.15.138:0/1742082657 conn(0x7f95a4100730 msgr2=0x7f95a4102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.508+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 shutdown_connections 2024-08-20T01:12:06.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:06.508+0000 7f95a9a00640 1 -- 172.21.15.138:0/1742082657 wait complete. 2024-08-20T01:12:07.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 138577 0.70 MB/sec execute 19 sec latency 19238.779 ms 2024-08-20T01:12:07.548 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '90be3842', total seconds elapsed: 24.1 2024-08-20T01:12:07.549 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.85 2024-08-20T01:12:07.549 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:12:07.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:07 smithi138 ceph-mon[18076]: pgmap v482: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 143 KiB/s wr, 31 op/s 2024-08-20T01:12:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:07 smithi167 ceph-mon[21965]: pgmap v482: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 143 KiB/s wr, 31 op/s 2024-08-20T01:12:07.978 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:07.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:07 smithi142 ceph-mon[21799]: pgmap v482: 129 pgs: 129 active+clean; 84 MiB data, 797 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 143 KiB/s wr, 31 op/s 2024-08-20T01:12:08.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 139954 0.94 MB/sec execute 20 sec latency 13.337 ms 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 -- 172.21.15.138:0/2625389107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810aea0 msgr2=0x7f09181116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 --2- 172.21.15.138:0/2625389107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810aea0 0x7f09181116a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0910009ca0 tx=0x7f091002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 -- 172.21.15.138:0/2625389107 shutdown_connections 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 --2- 172.21.15.138:0/2625389107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810aea0 0x7f09181116a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 --2- 172.21.15.138:0/2625389107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810a520 0x7f091810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 --2- 172.21.15.138:0/2625389107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f0918104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 -- 172.21.15.138:0/2625389107 >> 172.21.15.138:0/2625389107 conn(0x7f0918100540 msgr2=0x7f0918102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:08.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.582+0000 7f091dc00640 1 -- 172.21.15.138:0/2625389107 shutdown_connections 2024-08-20T01:12:08.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.583+0000 7f091dc00640 1 -- 172.21.15.138:0/2625389107 wait complete. 2024-08-20T01:12:08.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 Processor -- start 2024-08-20T01:12:08.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 -- start start 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f091819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810aea0 0x7f09181a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0918114b70 con 0x7f0918104b10 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f09181149f0 con 0x7f091810a520 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.584+0000 7f091dc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0918114cf0 con 0x7f091810aea0 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f091819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:08.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f091819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41580/0 (socket says 172.21.15.138:41580) 2024-08-20T01:12:08.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52148/0 (socket says 172.21.15.138:52148) 2024-08-20T01:12:08.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917e00640 1 -- 172.21.15.138:0/2131120707 learned_addr learned my addr 172.21.15.138:0/2131120707 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:08.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f091d200640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810aea0 0x7f09181a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 -- 172.21.15.138:0/2131120707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810aea0 msgr2=0x7f09181a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810aea0 0x7f09181a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 -- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 msgr2=0x7f091819af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f091819af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 -- 172.21.15.138:0/2131120707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09181a0fb0 con 0x7f091810a520 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.585+0000 7f0917400640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0904005fd0 tx=0x7f090400dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.586+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904009c30 con 0x7f091810a520 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.586+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0904009dd0 con 0x7f091810a520 2024-08-20T01:12:08.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.586+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09181a12a0 con 0x7f091810a520 2024-08-20T01:12:08.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.586+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904003e20 con 0x7f091810a520 2024-08-20T01:12:08.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.586+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09181a7b30 con 0x7f091810a520 2024-08-20T01:12:08.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.587+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0918104ef0 con 0x7f091810a520 2024-08-20T01:12:08.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.590+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0904003fe0 con 0x7f091810a520 2024-08-20T01:12:08.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.591+0000 7f0914c00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 0x7f08fc0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:08.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.591+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f09040e3df0 con 0x7f091810a520 2024-08-20T01:12:08.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.591+0000 7f0917e00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 0x7f08fc0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:08.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.592+0000 7f0917e00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 0x7f08fc0b6320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0908009cd0 tx=0x7f0908005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:08.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.592+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0904017020 con 0x7f091810a520 2024-08-20T01:12:08.863 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.865+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f091819c030 con 0x7f091810a520 2024-08-20T01:12:08.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.866+0000 7f0914c00640 1 -- 172.21.15.138:0/2131120707 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11935 (secure 0 0 0) 0x7f09040aa780 con 0x7f091810a520 2024-08-20T01:12:08.864 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:12:08.865 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 31 2024-08-20T01:12:08.865 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":31,"btime":"2024-08-20T01:11:09:621725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:11:09.220799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.868+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 msgr2=0x7f08fc0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.868+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 0x7f08fc0b6320 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0908009cd0 tx=0x7f0908005f90 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.868+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 msgr2=0x7f091819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.868+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0904005fd0 tx=0x7f090400dde0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 shutdown_connections 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08fc0b3e60 0x7f08fc0b6320 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f091810aea0 0x7f09181a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f091810a520 0x7f091819b440 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 --2- 172.21.15.138:0/2131120707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0918104b10 0x7f091819af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:08.867 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 >> 172.21.15.138:0/2131120707 conn(0x7f0918100540 msgr2=0x7f0918100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:08.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 shutdown_connections 2024-08-20T01:12:08.868 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:08.869+0000 7f091dc00640 1 -- 172.21.15.138:0/2131120707 wait complete. 2024-08-20T01:12:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:08 smithi142 ceph-mon[21799]: pgmap v483: 129 pgs: 129 active+clean; 85 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 30 op/s 2024-08-20T01:12:09.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:08 smithi138 ceph-mon[18076]: pgmap v483: 129 pgs: 129 active+clean; 85 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 30 op/s 2024-08-20T01:12:09.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 142281 1.56 MB/sec execute 21 sec latency 20.199 ms 2024-08-20T01:12:09.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:08 smithi167 ceph-mon[21965]: pgmap v483: 129 pgs: 129 active+clean; 85 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 30 op/s 2024-08-20T01:12:09.655 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0' 2024-08-20T01:12:09.656 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 139.85 --await-for 45.0 2024-08-20T01:12:09.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:09 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2131120707' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:10.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:09 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2131120707' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:10.062 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:10.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 144328 1.81 MB/sec execute 22 sec latency 13.573 ms 2024-08-20T01:12:10.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2131120707' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:10 smithi142 ceph-mon[21799]: pgmap v484: 129 pgs: 129 active+clean; 86 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 343 KiB/s wr, 57 op/s 2024-08-20T01:12:11.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:10 smithi138 ceph-mon[18076]: pgmap v484: 129 pgs: 129 active+clean; 86 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 343 KiB/s wr, 57 op/s 2024-08-20T01:12:11.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146136 2.26 MB/sec execute 23 sec latency 21.487 ms 2024-08-20T01:12:11.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:10 smithi167 ceph-mon[21965]: pgmap v484: 129 pgs: 129 active+clean; 86 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 974 KiB/s rd, 343 KiB/s wr, 57 op/s 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 -- 172.21.15.138:0/154069521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70103810 msgr2=0x7fca70103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 --2- 172.21.15.138:0/154069521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70103810 0x7fca70103bf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fca60009900 tx=0x7fca6002a400 comp rx=0 tx=0).stop 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 -- 172.21.15.138:0/154069521 shutdown_connections 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 --2- 172.21.15.138:0/154069521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 --2- 172.21.15.138:0/154069521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70104130 0x7fca7010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 --2- 172.21.15.138:0/154069521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70103810 0x7fca70103bf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.372 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 -- 172.21.15.138:0/154069521 >> 172.21.15.138:0/154069521 conn(0x7fca700fd2d0 msgr2=0x7fca700ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.191+0000 7fca75000640 1 -- 172.21.15.138:0/154069521 shutdown_connections 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.192+0000 7fca75000640 1 -- 172.21.15.138:0/154069521 wait complete. 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.193+0000 7fca75000640 1 Processor -- start 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.193+0000 7fca75000640 1 -- start start 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70103810 0x7fca70198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca70112930 con 0x7fca70103810 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca701127b0 con 0x7fca7010cc70 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca75000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca70112ab0 con 0x7fca70104130 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56394/0 (socket says 172.21.15.138:56394) 2024-08-20T01:12:11.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 -- 172.21.15.138:0/499870306 learned_addr learned my addr 172.21.15.138:0/499870306 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6f400640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70103810 0x7fca70198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 -- 172.21.15.138:0/499870306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 msgr2=0x7fca7019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 -- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70103810 msgr2=0x7fca70198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70103810 0x7fca70198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6ea00640 1 -- 172.21.15.138:0/499870306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7019edf0 con 0x7fca70104130 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.194+0000 7fca6fe00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca6ea00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e9c0 tx=0x7fca5c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca67e00640 1 -- 172.21.15.138:0/499870306 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00ccc0 con 0x7fca70104130 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca67e00640 1 -- 172.21.15.138:0/499870306 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c0035f0 con 0x7fca70104130 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca7019f0e0 con 0x7fca70104130 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca67e00640 1 -- 172.21.15.138:0/499870306 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00ce60 con 0x7fca70104130 2024-08-20T01:12:11.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.195+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca701a57a0 con 0x7fca70104130 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 Processor -- start 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 -- start start 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a23410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a2f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a31cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b840998600 con 0x55b840a2f8e0 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b840987b00 con 0x55b840a23950 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca79227740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b84098bb40 con 0x55b840a23030 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a2f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.199+0000 7fca6fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a23410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a2f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52168/0 (socket says 172.21.15.138:52168) 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6f400640 1 -- 172.21.15.138:0/20797321 learned_addr learned my addr 172.21.15.138:0/20797321 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca67e00640 1 -- 172.21.15.138:0/499870306 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fca5c004000 con 0x7fca70104130 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 -- 172.21.15.138:0/20797321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 msgr2=0x55b840a2f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a2f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6ea00640 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a31cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 -- 172.21.15.138:0/20797321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 msgr2=0x55b840a31cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a31cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 -- 172.21.15.138:0/20797321 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b84092dee0 con 0x55b840a23030 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6ea00640 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a31cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.200+0000 7fca6fe00640 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a23410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca68005fd0 tx=0x7fca68011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab75d08859c6de06 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.201+0000 7fca64c00640 1 -- 172.21.15.138:0/20797321 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004d20 con 0x55b840a23030 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.201+0000 7fca64c00640 1 -- 172.21.15.138:0/20797321 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6800f9a0 con 0x55b840a23030 2024-08-20T01:12:11.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.201+0000 7fca64c00640 1 -- 172.21.15.138:0/20797321 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68004470 con 0x55b840a23030 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 -- 172.21.15.138:0/20797321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 msgr2=0x55b840a23410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a23410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca68005fd0 tx=0x7fca68011bf0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca67e00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 0x7fca540b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 -- 172.21.15.138:0/20797321 shutdown_connections 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a31cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a2f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 --2- 172.21.15.138:0/20797321 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a23410 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 -- 172.21.15.138:0/20797321 >> 172.21.15.138:0/20797321 conn(0x55b840a20e60 msgr2=0x55b840a22000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca67e00640 1 -- 172.21.15.138:0/499870306 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fca5c0e4720 con 0x7fca70104130 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca6fe00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 0x7fca540b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 -- 172.21.15.138:0/20797321 shutdown_connections 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.202+0000 7fca79227740 1 -- 172.21.15.138:0/20797321 wait complete. 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 -- start start 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca6fe00640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 0x7fca540b62a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca6800f790 tx=0x7fca68000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 Processor -- start 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a2cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b8407ea7b0 con 0x55b840a2f8e0 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b840922dd0 con 0x55b840a23030 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b840a32560 con 0x55b840a23950 2024-08-20T01:12:11.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.203+0000 7fca79227740 10 client.0 _ll_register_callbacks cb 0x55b8406940a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a2cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52180/0 (socket says 172.21.15.138:52180) 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 -- 172.21.15.138:0/4035004340 learned_addr learned my addr 172.21.15.138:0/4035004340 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a2cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56410/0 (socket says 172.21.15.138:56410) 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6fe00640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 -- 172.21.15.138:0/4035004340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 msgr2=0x55b840a2cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a2cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 -- 172.21.15.138:0/4035004340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 msgr2=0x55b840a2d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a2d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 -- 172.21.15.138:0/4035004340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8406f6ba0 con 0x55b840a23030 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca6f400640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca60000c00 tx=0x7fca60005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3652cad791bba3d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.204+0000 7fca59600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60035040 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fca600048c0 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca6800f9a0 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fca60005d50 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca79227740 10 client.26054 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b840a3e060 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.205+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca6800f9a0 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fca60003c60 con 0x55b840a23030 2024-08-20T01:12:11.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 10 client.26054 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 10 client.26054 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 20 client.26054 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840a3f520 0x55b840a41930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 4 client.26054 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.206+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b84099a550 con 0x55b840a3f520 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.207+0000 7fca6ea00640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840a3f520 0x55b840a41930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.207+0000 7fca6ea00640 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840a3f520 0x55b840a41930 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a61c1dc0676f07d server_cookie=f57e440813539043 in_seq=0 out_seq=0 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.207+0000 7fca59600640 10 client.26054 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca59600640 1 -- 172.21.15.138:0/4035004340 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b84099a550 con 0x55b840a3f520 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca59600640 10 client.26054 handle_command_reply: tid=0 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 client.26054 shutdown 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 msgr2=0x55b840a2c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fca60000c00 tx=0x7fca60005270 comp rx=0 tx=0).stop 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 shutdown_connections 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840a3f520 0x55b840a41930 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a2d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a2cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca59600640 0 client.26054 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 --2- 172.21.15.138:0/4035004340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23030 0x55b840a2c840 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.208+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 >> 172.21.15.138:0/4035004340 conn(0x55b840a20e60 msgr2=0x55b840a269d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.209+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 shutdown_connections 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.209+0000 7fca79227740 1 -- 172.21.15.138:0/4035004340 wait complete. 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.209+0000 7fca79227740 20 client.26054 trim_cache size 0 max 16384 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.234+0000 7fca79227740 1 Processor -- start 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.234+0000 7fca79227740 1 -- start start 2024-08-20T01:12:11.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a3adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca701b2200 con 0x55b840a2f8e0 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca701afa70 con 0x55b840a23950 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca79227740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca701ad8c0 con 0x55b840a23030 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a3adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56414/0 (socket says 172.21.15.138:56414) 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a3adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52182/0 (socket says 172.21.15.138:52182) 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 -- 172.21.15.138:0/632159588 learned_addr learned my addr 172.21.15.138:0/632159588 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6fe00640 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 -- 172.21.15.138:0/632159588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 msgr2=0x55b840a3adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a3adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 -- 172.21.15.138:0/632159588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 msgr2=0x55b840a3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a3b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.235+0000 7fca6f400640 1 -- 172.21.15.138:0/632159588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b840a2c610 con 0x55b840a23030 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.236+0000 7fca6f400640 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca60031440 tx=0x7fca60009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62d233f221598790 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.236+0000 7fca5be00640 1 -- 172.21.15.138:0/632159588 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60035070 con 0x55b840a23030 2024-08-20T01:12:11.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.236+0000 7fca5be00640 1 -- 172.21.15.138:0/632159588 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca60040070 con 0x55b840a23030 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.236+0000 7fca5be00640 1 -- 172.21.15.138:0/632159588 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60053c80 con 0x55b840a23030 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- 172.21.15.138:0/632159588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 msgr2=0x55b840a2c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fca60031440 tx=0x7fca60009800 comp rx=0 tx=0).stop 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- 172.21.15.138:0/632159588 shutdown_connections 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a2f8e0 0x55b840a3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a23950 0x55b840a3adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 --2- 172.21.15.138:0/632159588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23030 0x55b840a2c0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- 172.21.15.138:0/632159588 >> 172.21.15.138:0/632159588 conn(0x55b840a20e60 msgr2=0x55b840a3a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- 172.21.15.138:0/632159588 shutdown_connections 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- 172.21.15.138:0/632159588 wait complete. 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.237+0000 7fca79227740 1 -- start start 2024-08-20T01:12:11.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 Processor -- start 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a23030 0x55b840a37820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a2f8e0 0x55b840a347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fca701adf30 con 0x55b840a23030 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fca701af3a0 con 0x55b840a2f8e0 2024-08-20T01:12:11.388 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fca701ae600 con 0x55b840a23950 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.238+0000 7fca79227740 10 client.0 _ll_register_callbacks cb 0x7fca680093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a23030 0x55b840a37820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56416/0 (socket says 172.21.15.138:56416) 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 -- 172.21.15.138:0/3464677618 learned_addr learned my addr 172.21.15.138:0/3464677618 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:11.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 -- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a2f8e0 msgr2=0x55b840a347f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a2f8e0 0x55b840a347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 -- 172.21.15.138:0/3464677618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a23030 msgr2=0x55b840a37820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a23030 0x55b840a37820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 -- 172.21.15.138:0/3464677618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b840a2a7b0 con 0x55b840a23950 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca6ea00640 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0e1b20 tx=0x7fca5c005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f66e7e60c69ff9f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:11.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.239+0000 7fca59600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0b4030 con 0x55b840a23950 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fca5c003a50 con 0x55b840a23950 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca60040070 con 0x55b840a23950 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fca5c0af330 con 0x55b840a23950 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca79227740 10 client.26017 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:11.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b840aaa750 con 0x55b840a23950 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca60040070 con 0x55b840a23950 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.240+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fca5c0e1d40 con 0x55b840a23950 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 10 client.26017 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 10 client.26017 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 20 client.26017 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:11.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840aaaef0 0x55b840aad310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:11.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 4 client.26017 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}] 2024-08-20T01:12:11.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 139.85, "await_for": 45.0}) -- 0x55b840995ff0 con 0x55b840aaaef0 2024-08-20T01:12:11.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.241+0000 7fca6f400640 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840aaaef0 0x55b840aad310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:11.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.242+0000 7fca6f400640 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840aaaef0 0x55b840aad310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd51222fbdfebe4 server_cookie=5ce63a812a7e93d3 in_seq=0 out_seq=0 2024-08-20T01:12:11.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.242+0000 7fca59600640 10 client.26017 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:11.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.449+0000 7fca59600640 1 -- 172.21.15.138:0/3464677618 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b840995ff0 con 0x55b840aaaef0 2024-08-20T01:12:11.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.449+0000 7fca59600640 10 client.26017 handle_command_reply: tid=0 2024-08-20T01:12:11.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.449+0000 7fca79227740 1 client.26017 shutdown 2024-08-20T01:12:11.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.449+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 msgr2=0x55b840a37d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.449+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0e1b20 tx=0x7fca5c005f10 comp rx=0 tx=0).stop 2024-08-20T01:12:11.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 shutdown_connections 2024-08-20T01:12:11.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b840aaaef0 0x55b840aad310 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.450 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b840a2f8e0 0x55b840a347f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b840a23950 0x55b840a37d60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 --2- 172.21.15.138:0/3464677618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b840a23030 0x55b840a37820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 >> 172.21.15.138:0/3464677618 conn(0x55b840a20e60 msgr2=0x55b840a3aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.451 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca59600640 0 client.26017 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:11.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 shutdown_connections 2024-08-20T01:12:11.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 1 -- 172.21.15.138:0/3464677618 wait complete. 2024-08-20T01:12:11.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.450+0000 7fca79227740 20 client.26017 trim_cache size 0 max 16384 2024-08-20T01:12:11.453 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:12:11.453 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 31, 2024-08-20T01:12:11.453 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:12:11.453 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 22, 2024-08-20T01:12:11.454 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:12:11.454 INFO:teuthology.orchestra.run.smithi138.stdout: "de06de7f": { 2024-08-20T01:12:11.454 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 22, 2024-08-20T01:12:11.454 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:12:11.455 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:11.455 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:12:11.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:11.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:12:11.455 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:12:11.456 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 139.8, 2024-08-20T01:12:11.456 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:12:11.456 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:12:11.456 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:12:11.456 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:11.457 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:12:11.457 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:11.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:11.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:11.457 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:11.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:11.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:11.458 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:12:11.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 msgr2=0x7fca540b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 0x7fca540b62a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fca6800f790 tx=0x7fca68000f40 comp rx=0 tx=0).stop 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 msgr2=0x7fca70199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fca5c00e9c0 tx=0x7fca5c00ee80 comp rx=0 tx=0).stop 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 shutdown_connections 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fca540b3de0 0x7fca540b62a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca7010cc70 0x7fca7019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fca70104130 0x7fca70199280 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 --2- 172.21.15.138:0/499870306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca70103810 0x7fca70198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:11.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 >> 172.21.15.138:0/499870306 conn(0x7fca700fd2d0 msgr2=0x7fca700fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:11.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 shutdown_connections 2024-08-20T01:12:11.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:11.451+0000 7fca75000640 1 -- 172.21.15.138:0/499870306 wait complete. 2024-08-20T01:12:12.136 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 2.19 MB/sec execute 24 sec latency 685.605 ms 2024-08-20T01:12:12.472 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'de06de7f', quiesce took 4.9 seconds. Will release after: 14.950000000000001 2024-08-20T01:12:13.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 2.10 MB/sec execute 25 sec latency 1685.741 ms 2024-08-20T01:12:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:13 smithi167 ceph-mon[21965]: pgmap v485: 129 pgs: 129 active+clean; 89 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 637 KiB/s wr, 91 op/s 2024-08-20T01:12:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:13 smithi142 ceph-mon[21799]: pgmap v485: 129 pgs: 129 active+clean; 89 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 637 KiB/s wr, 91 op/s 2024-08-20T01:12:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:13 smithi138 ceph-mon[18076]: pgmap v485: 129 pgs: 129 active+clean; 89 MiB data, 789 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 637 KiB/s wr, 91 op/s 2024-08-20T01:12:14.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 2.02 MB/sec execute 26 sec latency 2685.932 ms 2024-08-20T01:12:15.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.95 MB/sec execute 27 sec latency 3686.091 ms 2024-08-20T01:12:15.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:15 smithi167 ceph-mon[21965]: pgmap v486: 129 pgs: 129 active+clean; 91 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 KiB/s wr, 119 op/s 2024-08-20T01:12:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:15 smithi142 ceph-mon[21799]: pgmap v486: 129 pgs: 129 active+clean; 91 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 KiB/s wr, 119 op/s 2024-08-20T01:12:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:15 smithi138 ceph-mon[18076]: pgmap v486: 129 pgs: 129 active+clean; 91 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 848 KiB/s wr, 119 op/s 2024-08-20T01:12:16.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.88 MB/sec execute 28 sec latency 4686.174 ms 2024-08-20T01:12:17.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.81 MB/sec execute 29 sec latency 5686.296 ms 2024-08-20T01:12:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:17 smithi167 ceph-mon[21965]: pgmap v487: 129 pgs: 129 active+clean; 91 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 KiB/s wr, 107 op/s 2024-08-20T01:12:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:17 smithi142 ceph-mon[21799]: pgmap v487: 129 pgs: 129 active+clean; 91 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 KiB/s wr, 107 op/s 2024-08-20T01:12:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:17 smithi138 ceph-mon[18076]: pgmap v487: 129 pgs: 129 active+clean; 91 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 818 KiB/s wr, 107 op/s 2024-08-20T01:12:18.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.75 MB/sec execute 30 sec latency 6686.431 ms 2024-08-20T01:12:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 -- 172.21.15.138:0/809741929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410a410 msgr2=0x7f5f7410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 --2- 172.21.15.138:0/809741929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410a410 0x7f5f7410a850 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009900 tx=0x7f5f5c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 -- 172.21.15.138:0/809741929 shutdown_connections 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 --2- 172.21.15.138:0/809741929 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410ad90 0x7f5f74111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 --2- 172.21.15.138:0/809741929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410a410 0x7f5f7410a850 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 --2- 172.21.15.138:0/809741929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f74104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 -- 172.21.15.138:0/809741929 >> 172.21.15.138:0/809741929 conn(0x7f5f74100450 msgr2=0x7f5f74102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 -- 172.21.15.138:0/809741929 shutdown_connections 2024-08-20T01:12:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.053+0000 7f5f73200640 1 -- 172.21.15.138:0/809741929 wait complete. 2024-08-20T01:12:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.054+0000 7f5f73200640 1 Processor -- start 2024-08-20T01:12:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.054+0000 7f5f73200640 1 -- start start 2024-08-20T01:12:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410a410 0x7f5f7419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410ad90 0x7f5f741a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74114a60 con 0x7f5f74104a10 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741148e0 con 0x7f5f7410ad90 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f73200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74114be0 con 0x7f5f7410a410 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58854/0 (socket says 172.21.15.138:58854) 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 -- 172.21.15.138:0/3389584358 learned_addr learned my addr 172.21.15.138:0/3389584358 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f72800640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410ad90 0x7f5f741a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f69400640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410a410 0x7f5f7419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 -- 172.21.15.138:0/3389584358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410a410 msgr2=0x7f5f7419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410a410 0x7f5f7419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 -- 172.21.15.138:0/3389584358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410ad90 msgr2=0x7f5f741a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410ad90 0x7f5f741a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.055+0000 7f5f71e00640 1 -- 172.21.15.138:0/3389584358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f741a0fc0 con 0x7f5f74104a10 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f71e00640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400ea60 tx=0x7f5f6400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f6b400640 1 -- 172.21.15.138:0/3389584358 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400cde0 con 0x7f5f74104a10 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f6b400640 1 -- 172.21.15.138:0/3389584358 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f640035f0 con 0x7f5f74104a10 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f6b400640 1 -- 172.21.15.138:0/3389584358 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64010690 con 0x7f5f74104a10 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f741a12b0 con 0x7f5f74104a10 2024-08-20T01:12:19.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.056+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741a7970 con 0x7f5f74104a10 2024-08-20T01:12:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 Processor -- start 2024-08-20T01:12:19.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 -- start start 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f6b400640 1 -- 172.21.15.138:0/3389584358 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5f640037a0 con 0x7f5f74104a10 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee2009b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dee1f73210 con 0x55dee1ffe0f0 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dee1f62710 con 0x55dee200a080 2024-08-20T01:12:19.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.059+0000 7f5f7c012740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dee1f66750 con 0x55dee1ffd7d0 2024-08-20T01:12:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee2009b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55594/0 (socket says 172.21.15.138:55594) 2024-08-20T01:12:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f71e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee2009b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58866/0 (socket says 172.21.15.138:58866) 2024-08-20T01:12:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 -- 172.21.15.138:0/406014868 learned_addr learned my addr 172.21.15.138:0/406014868 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f69400640 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 -- 172.21.15.138:0/406014868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 msgr2=0x55dee200c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 -- 172.21.15.138:0/406014868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 msgr2=0x55dee2009b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee2009b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 -- 172.21.15.138:0/406014868 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee1f2ede0 con 0x55dee1ffd7d0 2024-08-20T01:12:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.060+0000 7f5f72800640 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009fd0 tx=0x7f5f6c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca8db82d9239e5e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f63400640 1 -- 172.21.15.138:0/406014868 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004d20 con 0x55dee1ffd7d0 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f63400640 1 -- 172.21.15.138:0/406014868 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c004050 con 0x55dee1ffd7d0 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f63400640 1 -- 172.21.15.138:0/406014868 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c005280 con 0x55dee1ffd7d0 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f6b400640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 0x7f5f580b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f71e00640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 0x7f5f580b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f6b400640 1 -- 172.21.15.138:0/3389584358 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5f640e4680 con 0x7f5f74104a10 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f7c012740 1 -- 172.21.15.138:0/406014868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 msgr2=0x55dee1ffdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.061+0000 7f5f7c012740 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c009fd0 tx=0x7f5f6c0118d0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 -- 172.21.15.138:0/406014868 shutdown_connections 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee2009b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 --2- 172.21.15.138:0/406014868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee1ffdbb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 -- 172.21.15.138:0/406014868 >> 172.21.15.138:0/406014868 conn(0x55dee1ffb600 msgr2=0x55dee1ffc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 -- 172.21.15.138:0/406014868 shutdown_connections 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 -- 172.21.15.138:0/406014868 wait complete. 2024-08-20T01:12:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f71e00640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 0x7f5f580b6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55dee2000950 tx=0x7f5f640ea530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 -- start start 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.062+0000 7f5f7c012740 1 Processor -- start 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee2007060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 0x55dee20075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55dee1e9bab0 con 0x55dee200a080 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55dee1f09140 con 0x55dee1ffe0f0 2024-08-20T01:12:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55dee200cd60 con 0x55dee1ffd7d0 2024-08-20T01:12:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f7c012740 10 client.0 _ll_register_callbacks cb 0x55dee1f1c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f72800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58880/0 (socket says 172.21.15.138:58880) 2024-08-20T01:12:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f72800640 1 -- 172.21.15.138:0/3473516050 learned_addr learned my addr 172.21.15.138:0/3473516050 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f69400640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 0x55dee20075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.063+0000 7f5f72800640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee2007060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 -- 172.21.15.138:0/3473516050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 msgr2=0x55dee2007060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee2007060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 -- 172.21.15.138:0/3473516050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 msgr2=0x55dee2007ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 -- 172.21.15.138:0/3473516050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee1f0a170 con 0x55dee1ffe0f0 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f72800640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f69400640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 0x55dee20075a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c02a9e0 tx=0x7f5f5c004440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b299a925c1fc1b60 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f57e00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c035040 con 0x55dee1ffe0f0 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dee1f39a30 con 0x55dee1ffe0f0 2024-08-20T01:12:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5f5c005510 con 0x55dee1ffe0f0 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f5c04f050 con 0x55dee1ffe0f0 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f7c012740 10 client.26069 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.064+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dee1cc5790 con 0x55dee1ffe0f0 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dee1f39a30 con 0x55dee1ffe0f0 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5f5c003950 con 0x55dee1ffe0f0 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 10 client.26069 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 10 client.26069 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 20 client.26069 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee201a040 0x55dee201c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 4 client.26069 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dee1f75160 con 0x55dee201a040 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.065+0000 7f5f72800640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee201a040 0x55dee201c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.066+0000 7f5f72800640 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee201a040 0x55dee201c430 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6085c4ee5a9e3bf server_cookie=18aa6df457884121 in_seq=0 out_seq=0 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.066+0000 7f5f57e00640 10 client.26069 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f57e00640 1 -- 172.21.15.138:0/3473516050 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dee1f75160 con 0x55dee201a040 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f57e00640 10 client.26069 handle_command_reply: tid=0 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 client.26069 shutdown 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 msgr2=0x55dee20075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 0x55dee20075a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c02a9e0 tx=0x7f5f5c004440 comp rx=0 tx=0).stop 2024-08-20T01:12:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 shutdown_connections 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee201a040 0x55dee201c430 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee200a080 0x55dee2007ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f57e00640 0 client.26069 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffe0f0 0x55dee20075a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 --2- 172.21.15.138:0/3473516050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee2007060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 >> 172.21.15.138:0/3473516050 conn(0x55dee1ffb600 msgr2=0x55dee20011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 shutdown_connections 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 1 -- 172.21.15.138:0/3473516050 wait complete. 2024-08-20T01:12:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.067+0000 7f5f7c012740 20 client.26069 trim_cache size 0 max 16384 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.082+0000 7f5f7c012740 1 Processor -- start 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.082+0000 7f5f7c012740 1 -- start start 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee20068d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee200a080 0x55dee2021c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b4350 con 0x55dee1ffe0f0 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b1bc0 con 0x55dee1ffd7d0 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f7c012740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741afa10 con 0x55dee200a080 2024-08-20T01:12:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f69400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee20068d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:51756/0 (socket says 172.21.15.138:51756) 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 -- 172.21.15.138:0/2868951378 learned_addr learned my addr 172.21.15.138:0/2868951378 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee200a080 0x55dee2021c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 -- 172.21.15.138:0/2868951378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee200a080 msgr2=0x55dee2021c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee200a080 0x55dee2021c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 -- 172.21.15.138:0/2868951378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 msgr2=0x55dee20068d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee20068d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 -- 172.21.15.138:0/2868951378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dee1d3e620 con 0x55dee1ffd7d0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f69400640 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee20068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.083+0000 7f5f72800640 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00a010 tx=0x7f5f6c005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5cdc8cedb4457a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f62a00640 1 -- 172.21.15.138:0/2868951378 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004050 con 0x55dee1ffd7d0 2024-08-20T01:12:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f62a00640 1 -- 172.21.15.138:0/2868951378 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c00f4e0 con 0x55dee1ffd7d0 2024-08-20T01:12:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f62a00640 1 -- 172.21.15.138:0/2868951378 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c0286b0 con 0x55dee1ffd7d0 2024-08-20T01:12:19.086 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 -- 172.21.15.138:0/2868951378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 msgr2=0x55dee2001990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c00a010 tx=0x7f5f6c005bd0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 -- 172.21.15.138:0/2868951378 shutdown_connections 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee200a080 0x55dee2021c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee20068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 --2- 172.21.15.138:0/2868951378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee1ffd7d0 0x55dee2001990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 -- 172.21.15.138:0/2868951378 >> 172.21.15.138:0/2868951378 conn(0x55dee1ffb600 msgr2=0x55dee2007850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 -- 172.21.15.138:0/2868951378 shutdown_connections 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.084+0000 7f5f7c012740 1 -- 172.21.15.138:0/2868951378 wait complete. 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 -- start start 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 Processor -- start 2024-08-20T01:12:19.087 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee200cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b0080 con 0x55dee1ffe0f0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b14f0 con 0x55dee200a080 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.085+0000 7f5f7c012740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f741b0750 con 0x55dee1ffd7d0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f7c012740 10 client.0 _ll_register_callbacks cb 0x7f5f5c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58900/0 (socket says 172.21.15.138:58900) 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 -- 172.21.15.138:0/705650949 learned_addr learned my addr 172.21.15.138:0/705650949 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f72800640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee200cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f72800640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 -- 172.21.15.138:0/705650949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 msgr2=0x55dee200cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee200cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 -- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 msgr2=0x55dee200da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 -- 172.21.15.138:0/705650949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dee208b4e0 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f72800640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f69400640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c003df0 tx=0x7f5f5c02a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c768b4f638e5254 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.086+0000 7f5f57e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c035070 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f6c00f4e0 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f5f5c003800 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5f5c060050 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 10 client.16026 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dee208c800 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f6c00f4e0 con 0x55dee1ffe0f0 2024-08-20T01:12:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f5f5c05b030 con 0x55dee1ffe0f0 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 10 client.16026 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 10 client.16026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 20 client.16026 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee208d000 0x55dee208f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 4 client.16026 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.087+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dee1f70c00 con 0x55dee208d000 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.088+0000 7f5f72800640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee208d000 0x55dee208f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.088+0000 7f5f72800640 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee208d000 0x55dee208f420 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fab94e574ccea2a server_cookie=771e08194cd5264d in_seq=0 out_seq=0 2024-08-20T01:12:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.088+0000 7f5f57e00640 10 client.16026 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.120+0000 7f5f57e00640 1 -- 172.21.15.138:0/705650949 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dee1f70c00 con 0x55dee208d000 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f57e00640 10 client.16026 handle_command_reply: tid=0 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 client.16026 shutdown 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 msgr2=0x55dee200d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c003df0 tx=0x7f5f5c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 shutdown_connections 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55dee208d000 0x55dee208f420 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dee200a080 0x55dee200da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55dee1ffe0f0 0x55dee200d520 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 --2- 172.21.15.138:0/705650949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55dee1ffd7d0 0x55dee200cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f57e00640 0 client.16026 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 >> 172.21.15.138:0/705650949 conn(0x55dee1ffb600 msgr2=0x55dee1ffc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 shutdown_connections 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 1 -- 172.21.15.138:0/705650949 wait complete. 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.121+0000 7f5f7c012740 20 client.16026 trim_cache size 0 max 16384 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:12:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:12:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 msgr2=0x7f5f580b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 0x7f5f580b6200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55dee2000950 tx=0x7f5f640ea530 comp rx=0 tx=0).stop 2024-08-20T01:12:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 msgr2=0x7f5f7419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400ea60 tx=0x7f5f6400ef20 comp rx=0 tx=0).stop 2024-08-20T01:12:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 shutdown_connections 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f580b3d40 0x7f5f580b6200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f7410ad90 0x7f5f741a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f7410a410 0x7f5f7419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 --2- 172.21.15.138:0/3389584358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f74104a10 0x7f5f7419af20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 >> 172.21.15.138:0/3389584358 conn(0x7f5f74100450 msgr2=0x7f5f74102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 shutdown_connections 2024-08-20T01:12:19.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:19.122+0000 7f5f73200640 1 -- 172.21.15.138:0/3389584358 wait complete. 2024-08-20T01:12:19.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.70 MB/sec execute 31 sec latency 7686.558 ms 2024-08-20T01:12:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:19 smithi167 ceph-mon[21965]: pgmap v488: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 732 KiB/s wr, 104 op/s 2024-08-20T01:12:19.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:19 smithi142 ceph-mon[21799]: pgmap v488: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 732 KiB/s wr, 104 op/s 2024-08-20T01:12:20.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:19 smithi138 ceph-mon[18076]: pgmap v488: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 732 KiB/s wr, 104 op/s 2024-08-20T01:12:20.137 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.64 MB/sec execute 32 sec latency 8686.691 ms 2024-08-20T01:12:21.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.59 MB/sec execute 33 sec latency 9686.839 ms 2024-08-20T01:12:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:21 smithi167 ceph-mon[21965]: pgmap v489: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 KiB/s wr, 112 op/s 2024-08-20T01:12:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:21 smithi142 ceph-mon[21799]: pgmap v489: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 KiB/s wr, 112 op/s 2024-08-20T01:12:22.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:21 smithi138 ceph-mon[18076]: pgmap v489: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 715 KiB/s wr, 112 op/s 2024-08-20T01:12:22.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.55 MB/sec execute 34 sec latency 10686.956 ms 2024-08-20T01:12:23.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.50 MB/sec execute 35 sec latency 11687.061 ms 2024-08-20T01:12:23.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:23 smithi167 ceph-mon[21965]: pgmap v490: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 540 KiB/s wr, 87 op/s 2024-08-20T01:12:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:23 smithi142 ceph-mon[21799]: pgmap v490: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 540 KiB/s wr, 87 op/s 2024-08-20T01:12:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:23 smithi138 ceph-mon[18076]: pgmap v490: 129 pgs: 129 active+clean; 91 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 540 KiB/s wr, 87 op/s 2024-08-20T01:12:24.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.46 MB/sec execute 36 sec latency 12687.190 ms 2024-08-20T01:12:25.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.42 MB/sec execute 37 sec latency 13687.331 ms 2024-08-20T01:12:25.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:25 smithi167 ceph-mon[21965]: pgmap v491: 129 pgs: 129 active+clean; 92 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 54 op/s 2024-08-20T01:12:25.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:25 smithi142 ceph-mon[21799]: pgmap v491: 129 pgs: 129 active+clean; 92 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 54 op/s 2024-08-20T01:12:26.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:25 smithi138 ceph-mon[18076]: pgmap v491: 129 pgs: 129 active+clean; 92 MiB data, 786 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 277 KiB/s wr, 54 op/s 2024-08-20T01:12:26.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.38 MB/sec execute 38 sec latency 14687.477 ms 2024-08-20T01:12:27.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.35 MB/sec execute 39 sec latency 15687.615 ms 2024-08-20T01:12:27.423 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:12:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:27 smithi138 ceph-mon[18076]: pgmap v492: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 27 op/s 2024-08-20T01:12:27.810 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:27 smithi167 ceph-mon[21965]: pgmap v492: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 27 op/s 2024-08-20T01:12:27.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:27 smithi142 ceph-mon[21799]: pgmap v492: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 121 KiB/s wr, 27 op/s 2024-08-20T01:12:28.138 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.31 MB/sec execute 40 sec latency 16687.709 ms 2024-08-20T01:12:28.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 -- 172.21.15.138:0/3774647742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc108310 msgr2=0x7f8cdc108750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:28.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 --2- 172.21.15.138:0/3774647742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc108310 0x7f8cdc108750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009900 tx=0x7f8cc802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 -- 172.21.15.138:0/3774647742 shutdown_connections 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 --2- 172.21.15.138:0/3774647742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 0x7f8cdc10f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 --2- 172.21.15.138:0/3774647742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc108310 0x7f8cdc108750 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 --2- 172.21.15.138:0/3774647742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc102900 0x7f8cdc102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 -- 172.21.15.138:0/3774647742 >> 172.21.15.138:0/3774647742 conn(0x7f8cdc0fe390 msgr2=0x7f8cdc1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 -- 172.21.15.138:0/3774647742 shutdown_connections 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.433+0000 7f8cdb200640 1 -- 172.21.15.138:0/3774647742 wait complete. 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.434+0000 7f8cdb200640 1 Processor -- start 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.434+0000 7f8cdb200640 1 -- start start 2024-08-20T01:12:28.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc108310 0x7f8cdc19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 0x7f8cdc1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc112930 con 0x7f8cdc102900 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc1127b0 con 0x7f8cdc108c90 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cdb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8cdc112ab0 con 0x7f8cdc108310 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cd9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35002/0 (socket says 172.21.15.138:35002) 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2575693582 learned_addr learned my addr 172.21.15.138:0/2575693582 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cd9400640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc108310 0x7f8cdc19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.435+0000 7f8cda800640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 0x7f8cdc1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:28.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2575693582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc108310 msgr2=0x7f8cdc19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc108310 0x7f8cdc19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2575693582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 msgr2=0x7f8cdc1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 0x7f8cdc1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 -- 172.21.15.138:0/2575693582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cdc1a0fe0 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd9e00640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00ea60 tx=0x7f8ccc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00cde0 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ccc0035f0 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc010690 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cdc1a12d0 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.436+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cdc1a7990 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.437+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cdc102ce0 con 0x7f8cdc102900 2024-08-20T01:12:28.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.439+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8ccc0037a0 con 0x7f8cdc102900 2024-08-20T01:12:28.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.441+0000 7f8cd2a00640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 0x7f8cc00b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:28.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.441+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8ccc0e45f0 con 0x7f8cdc102900 2024-08-20T01:12:28.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.441+0000 7f8cd9400640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 0x7f8cc00b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:28.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.442+0000 7f8cd9400640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 0x7f8cc00b6320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8005ab0 tx=0x7f8cc8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:28.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.444+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f8ccc0ab550 con 0x7f8cdc102900 2024-08-20T01:12:28.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.666+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cdc02e340 con 0x7f8cdc102900 2024-08-20T01:12:28.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.669+0000 7f8cd2a00640 1 -- 172.21.15.138:0/2575693582 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11935 (secure 0 0 0) 0x7f8ccc0aae50 con 0x7f8cdc102900 2024-08-20T01:12:28.668 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:12:28.668 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":31,"btime":"2024-08-20T01:11:09:621725+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:11:09.220799+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:12:28.668 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 31 2024-08-20T01:12:28.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 msgr2=0x7f8cc00b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:28.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 0x7f8cc00b6320 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8005ab0 tx=0x7f8cc8005a20 comp rx=0 tx=0).stop 2024-08-20T01:12:28.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 msgr2=0x7f8cdc19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:28.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc00ea60 tx=0x7f8ccc00ef20 comp rx=0 tx=0).stop 2024-08-20T01:12:28.670 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 shutdown_connections 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8cc00b3e60 0x7f8cc00b6320 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cdc108c90 0x7f8cdc1a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8cdc108310 0x7f8cdc19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 --2- 172.21.15.138:0/2575693582 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cdc102900 0x7f8cdc19af30 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 >> 172.21.15.138:0/2575693582 conn(0x7f8cdc0fe390 msgr2=0x7f8cdc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 shutdown_connections 2024-08-20T01:12:28.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:28.672+0000 7f8cdb200640 1 -- 172.21.15.138:0/2575693582 wait complete. 2024-08-20T01:12:29.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.28 MB/sec execute 41 sec latency 17687.775 ms 2024-08-20T01:12:29.584 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id de06de7f --release --await' 2024-08-20T01:12:29.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id de06de7f --release --await 2024-08-20T01:12:29.957 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:12:29.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:29 smithi167 ceph-mon[21965]: pgmap v493: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-08-20T01:12:29.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:29 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2575693582' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:29 smithi142 ceph-mon[21799]: pgmap v493: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-08-20T01:12:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:29 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2575693582' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:30.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:29 smithi138 ceph-mon[18076]: pgmap v493: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 101 KiB/s wr, 22 op/s 2024-08-20T01:12:30.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:29 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2575693582' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:12:30.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 146360 1.25 MB/sec execute 42 sec latency 18687.847 ms 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 -- 172.21.15.138:0/2447571278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 msgr2=0x7f4940104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 --2- 172.21.15.138:0/2447571278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f4940104ef0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4930009900 tx=0x7f493002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 -- 172.21.15.138:0/2447571278 shutdown_connections 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 --2- 172.21.15.138:0/2447571278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010aea0 0x7f49401116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 --2- 172.21.15.138:0/2447571278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010a520 0x7f494010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 --2- 172.21.15.138:0/2447571278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f4940104ef0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 -- 172.21.15.138:0/2447571278 >> 172.21.15.138:0/2447571278 conn(0x7f4940100540 msgr2=0x7f4940102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 -- 172.21.15.138:0/2447571278 shutdown_connections 2024-08-20T01:12:30.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.577+0000 7f4945200640 1 -- 172.21.15.138:0/2447571278 wait complete. 2024-08-20T01:12:30.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.578+0000 7f4945200640 1 Processor -- start 2024-08-20T01:12:30.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.578+0000 7f4945200640 1 -- start start 2024-08-20T01:12:30.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f494019af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 0x7f494019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010aea0 0x7f49401a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4940114b20 con 0x7f4940104b10 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f49401149a0 con 0x7f494010a520 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4945200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4940114ca0 con 0x7f494010aea0 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f493f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f494019af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f493f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f494019af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35016/0 (socket says 172.21.15.138:35016) 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 0x7f494019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f493f400640 1 -- 172.21.15.138:0/3382001796 learned_addr learned my addr 172.21.15.138:0/3382001796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f493fe00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010aea0 0x7f49401a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 -- 172.21.15.138:0/3382001796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010aea0 msgr2=0x7f49401a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010aea0 0x7f49401a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 -- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 msgr2=0x7f494019af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f494019af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.579+0000 7f4937e00640 1 -- 172.21.15.138:0/3382001796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49401a0fe0 con 0x7f494010a520 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f4937e00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 0x7f494019b470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f492800e9f0 tx=0x7f492800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f493cc00640 1 -- 172.21.15.138:0/3382001796 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f492800ccc0 con 0x7f494010a520 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f493cc00640 1 -- 172.21.15.138:0/3382001796 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49280108e0 con 0x7f494010a520 2024-08-20T01:12:30.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49401a12d0 con 0x7f494010a520 2024-08-20T01:12:30.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49401a7990 con 0x7f494010a520 2024-08-20T01:12:30.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.580+0000 7f493cc00640 1 -- 172.21.15.138:0/3382001796 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4928018880 con 0x7f494010a520 2024-08-20T01:12:30.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 Processor -- start 2024-08-20T01:12:30.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f493cc00640 1 -- 172.21.15.138:0/3382001796 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4928010430 con 0x7f494010a520 2024-08-20T01:12:30.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 -- start start 2024-08-20T01:12:30.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a78b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b639a0b670 con 0x55b639a76730 2024-08-20T01:12:30.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b6399dfd70 con 0x55b639a6a7a0 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.584+0000 7f49491fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b6399cf270 con 0x55b639a69e80 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44506/0 (socket says 172.21.15.138:44506) 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493fe00640 1 -- 172.21.15.138:0/3159336109 learned_addr learned my addr 172.21.15.138:0/3159336109 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f4937e00640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a78b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 -- 172.21.15.138:0/3159336109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 msgr2=0x55b639a6a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 -- 172.21.15.138:0/3159336109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 msgr2=0x55b639a78b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a78b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493f400640 1 -- 172.21.15.138:0/3159336109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b63996b870 con 0x55b639a6a7a0 2024-08-20T01:12:30.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.585+0000 7f493fe00640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f493f400640 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a761f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4930004770 tx=0x7f4930031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2449cdb54199bdd7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f4934c00640 1 -- 172.21.15.138:0/3159336109 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930040070 con 0x55b639a6a7a0 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f4934c00640 1 -- 172.21.15.138:0/3159336109 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4930004030 con 0x55b639a6a7a0 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f4934c00640 1 -- 172.21.15.138:0/3159336109 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4930035040 con 0x55b639a6a7a0 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f493cc00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 0x7f49240b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f493cc00640 1 -- 172.21.15.138:0/3382001796 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f49280e4500 con 0x7f494010a520 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.586+0000 7f493fe00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 0x7f49240b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 -- 172.21.15.138:0/3159336109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 msgr2=0x55b639a761f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a761f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4930004770 tx=0x7f4930031b10 comp rx=0 tx=0).stop 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 -- 172.21.15.138:0/3159336109 shutdown_connections 2024-08-20T01:12:30.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a78b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a761f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 --2- 172.21.15.138:0/3159336109 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a6a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 -- 172.21.15.138:0/3159336109 >> 172.21.15.138:0/3159336109 conn(0x55b639a67cb0 msgr2=0x55b639a68e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 -- 172.21.15.138:0/3159336109 shutdown_connections 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f493fe00640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 0x7f49240b62a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4938011890 tx=0x7f4938009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.587+0000 7f49491fc740 1 -- 172.21.15.138:0/3159336109 wait complete. 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 -- start start 2024-08-20T01:12:30.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 Processor -- start 2024-08-20T01:12:30.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a732b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a737f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b639955ff0 con 0x55b639a76730 2024-08-20T01:12:30.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b6399d32b0 con 0x55b639a6a7a0 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b639831bb0 con 0x55b639a69e80 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.588+0000 7f49491fc740 10 client.0 _ll_register_callbacks cb 0x55b6396dad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44508/0 (socket says 172.21.15.138:44508) 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 -- 172.21.15.138:0/2259446953 learned_addr learned my addr 172.21.15.138:0/2259446953 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f4937e00640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a732b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493fe00640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a737f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 -- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 msgr2=0x55b639a732b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a732b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 -- 172.21.15.138:0/2259446953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 msgr2=0x55b639a737f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a737f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 -- 172.21.15.138:0/2259446953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b63972eb00 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f4937e00640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f493f400640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4930031440 tx=0x7f4930004490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59e605a6b1637a16 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.589+0000 7f492d600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493004c020 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4930004030 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f49300314a0 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4930057050 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f49491fc740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b6397a5960 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4930004030 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f49300035f0 con 0x55b639a69e80 2024-08-20T01:12:30.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.590+0000 7f49491fc740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f49491fc740 10 client.26050 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f49491fc740 20 client.26050 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639a85b30 0x55b639a87f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f49491fc740 4 client.26050 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6399e1cc0 con 0x55b639a85b30 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f4937e00640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639a85b30 0x55b639a87f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f4937e00640 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639a85b30 0x55b639a87f20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=832820eff27665ce server_cookie=bed239b4bc4e9b0 in_seq=0 out_seq=0 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.591+0000 7f492d600640 10 client.26050 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f492d600640 1 -- 172.21.15.138:0/2259446953 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6399e1cc0 con 0x55b639a85b30 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f492d600640 10 client.26050 handle_command_reply: tid=0 2024-08-20T01:12:30.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 client.26050 shutdown 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 msgr2=0x55b639a75e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4930031440 tx=0x7f4930004490 comp rx=0 tx=0).stop 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 shutdown_connections 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639a85b30 0x55b639a87f20 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a737f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f492d600640 0 client.26050 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a732b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 --2- 172.21.15.138:0/2259446953 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a75e60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.592+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 >> 172.21.15.138:0/2259446953 conn(0x55b639a67cb0 msgr2=0x55b639a6d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.593+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 shutdown_connections 2024-08-20T01:12:30.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.593+0000 7f49491fc740 1 -- 172.21.15.138:0/2259446953 wait complete. 2024-08-20T01:12:30.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.593+0000 7f49491fc740 20 client.26050 trim_cache size 0 max 16384 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.610+0000 7f49491fc740 1 Processor -- start 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.610+0000 7f49491fc740 1 -- start start 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a72240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a8d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f491c012770 con 0x55b639a76730 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f491c00ffe0 con 0x55b639a6a7a0 2024-08-20T01:12:30.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f49491fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f491c00de30 con 0x55b639a69e80 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f4937e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a8d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35042/0 (socket says 172.21.15.138:35042) 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 -- 172.21.15.138:0/699385149 learned_addr learned my addr 172.21.15.138:0/699385149 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493f400640 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a72240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 -- 172.21.15.138:0/699385149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 msgr2=0x55b639a72240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a72240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 -- 172.21.15.138:0/699385149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 msgr2=0x55b639a8d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a8d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f493fe00640 1 -- 172.21.15.138:0/699385149 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b639a761d0 con 0x55b639a76730 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.611+0000 7f4937e00640 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a8d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.612+0000 7f493fe00640 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49380039f0 tx=0x7f493801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cc59f8071e52fd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.612+0000 7f492fe00640 1 -- 172.21.15.138:0/699385149 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938004b20 con 0x55b639a76730 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.612+0000 7f492fe00640 1 -- 172.21.15.138:0/699385149 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4938004cc0 con 0x55b639a76730 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.612+0000 7f492fe00640 1 -- 172.21.15.138:0/699385149 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938005310 con 0x55b639a76730 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- 172.21.15.138:0/699385149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 msgr2=0x55b639a8da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f49380039f0 tx=0x7f493801a000 comp rx=0 tx=0).stop 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- 172.21.15.138:0/699385149 shutdown_connections 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a76730 0x55b639a8da60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a8d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 --2- 172.21.15.138:0/699385149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a69e80 0x55b639a72240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- 172.21.15.138:0/699385149 >> 172.21.15.138:0/699385149 conn(0x55b639a67cb0 msgr2=0x55b639a71b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- 172.21.15.138:0/699385149 shutdown_connections 2024-08-20T01:12:30.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- 172.21.15.138:0/699385149 wait complete. 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.613+0000 7f49491fc740 1 -- start start 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 Processor -- start 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a79e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a76730 0x55b639a7a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f491c00e4a0 con 0x55b639a69e80 2024-08-20T01:12:30.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f491c00f910 con 0x55b639a6a7a0 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f491c00eb70 con 0x55b639a76730 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f49491fc740 10 client.0 _ll_register_callbacks cb 0x7f49280e7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f493f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f4937e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a79e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f493f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35044/0 (socket says 172.21.15.138:35044) 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f493f400640 1 -- 172.21.15.138:0/676316453 learned_addr learned my addr 172.21.15.138:0/676316453 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.614+0000 7f493fe00640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a76730 0x55b639a7a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 -- 172.21.15.138:0/676316453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a76730 msgr2=0x55b639a7a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a76730 0x55b639a7a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 -- 172.21.15.138:0/676316453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 msgr2=0x55b639a7e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 -- 172.21.15.138:0/676316453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b639a7aa20 con 0x55b639a6a7a0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f493f400640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f4937e00640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a79e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49280e18a0 tx=0x7f49280e9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd925c20b176542f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f492d600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49280aa580 con 0x55b639a6a7a0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f49280a7400 con 0x55b639a6a7a0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4938004cc0 con 0x55b639a6a7a0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f492800ce60 con 0x55b639a6a7a0 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f49491fc740 10 client.26099 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:30.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.615+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55b639af03e0 con 0x55b639a6a7a0 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4938004cc0 con 0x55b639a6a7a0 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f49280a6780 con 0x55b639a6a7a0 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 10 client.26099 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 10 client.26099 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 20 client.26099 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639af0b60 0x55b639af2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 4 client.26099 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "de06de7f", "release": true, "await": true}] 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "de06de7f", "release": true, "await": true}) -- 0x55b6399dd760 con 0x55b639af0b60 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.616+0000 7f493f400640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639af0b60 0x55b639af2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.617+0000 7f493f400640 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639af0b60 0x55b639af2f80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abefa13252c42da7 server_cookie=fd1b20a08b8d65a7 in_seq=0 out_seq=0 2024-08-20T01:12:30.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.617+0000 7f492d600640 10 client.26099 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:30.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.624+0000 7f492d600640 1 -- 172.21.15.138:0/676316453 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b6399dd760 con 0x55b639af0b60 2024-08-20T01:12:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.624+0000 7f492d600640 10 client.26099 handle_command_reply: tid=0 2024-08-20T01:12:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.624+0000 7f49491fc740 1 client.26099 shutdown 2024-08-20T01:12:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 msgr2=0x55b639a79e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a79e70 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f49280e18a0 tx=0x7f49280e9040 comp rx=0 tx=0).stop 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 shutdown_connections 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b639af0b60 0x55b639af2f80 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b639a76730 0x55b639a7a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b639a6a7a0 0x55b639a79e70 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f492d600640 0 client.26099 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 --2- 172.21.15.138:0/676316453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b639a69e80 0x55b639a7e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 >> 172.21.15.138:0/676316453 conn(0x55b639a67cb0 msgr2=0x55b639a72350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 shutdown_connections 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.625+0000 7f49491fc740 1 -- 172.21.15.138:0/676316453 wait complete. 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.626+0000 7f49491fc740 20 client.26099 trim_cache size 0 max 16384 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:12:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 31, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 24, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "de06de7f": { 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 24, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:12:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 139.8, 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 msgr2=0x7f49240b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 0x7f49240b62a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4938011890 tx=0x7f4938009500 comp rx=0 tx=0).stop 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 msgr2=0x7f494019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 0x7f494019b470 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f492800e9f0 tx=0x7f492800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 shutdown_connections 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f49240b3de0 0x7f49240b62a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f494010aea0 0x7f49401a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494010a520 0x7f494019b470 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 --2- 172.21.15.138:0/3382001796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4940104b10 0x7f494019af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.627+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 >> 172.21.15.138:0/3382001796 conn(0x7f4940100540 msgr2=0x7f4940102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.628+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 shutdown_connections 2024-08-20T01:12:30.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:12:30.628+0000 7f4945200640 1 -- 172.21.15.138:0/3382001796 wait complete. 2024-08-20T01:12:31.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 147830 1.37 MB/sec execute 43 sec latency 19171.249 ms 2024-08-20T01:12:31.616 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'de06de7f', total seconds elapsed: 24.1 2024-08-20T01:12:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:31 smithi167 ceph-mon[21965]: pgmap v494: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 112 KiB/s wr, 24 op/s 2024-08-20T01:12:31.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:31 smithi142 ceph-mon[21799]: pgmap v494: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 112 KiB/s wr, 24 op/s 2024-08-20T01:12:32.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:31 smithi138 ceph-mon[18076]: pgmap v494: 129 pgs: 129 active+clean; 92 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 112 KiB/s wr, 24 op/s 2024-08-20T01:12:32.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 149650 1.61 MB/sec execute 44 sec latency 13.390 ms 2024-08-20T01:12:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 -- 172.21.15.138:0/511918307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 msgr2=0x7f48f006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 --2- 172.21.15.138:0/511918307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f006ab80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009900 tx=0x7f48e002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 -- 172.21.15.138:0/511918307 shutdown_connections 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 --2- 172.21.15.138:0/511918307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0105970 0x7f48f010f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 --2- 172.21.15.138:0/511918307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0104fd0 0x7f48f0105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 --2- 172.21.15.138:0/511918307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f006ab80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 -- 172.21.15.138:0/511918307 >> 172.21.15.138:0/511918307 conn(0x7f48f0077270 msgr2=0x7f48f0077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 -- 172.21.15.138:0/511918307 shutdown_connections 2024-08-20T01:12:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.252+0000 7f48f5200640 1 -- 172.21.15.138:0/511918307 wait complete. 2024-08-20T01:12:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.253+0000 7f48f5200640 1 Processor -- start 2024-08-20T01:12:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.253+0000 7f48f5200640 1 -- start start 2024-08-20T01:12:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 0x7f48f019b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0105970 0x7f48f01a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0112850 con 0x7f48f0105970 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01126d0 con 0x7f48f006a7a0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48f5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01129d0 con 0x7f48f0104fd0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48efe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0105970 0x7f48f01a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34806/0 (socket says 172.21.15.138:34806) 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 0x7f48f019b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 -- 172.21.15.138:0/3168401122 learned_addr learned my addr 172.21.15.138:0/3168401122 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 -- 172.21.15.138:0/3168401122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 msgr2=0x7f48f019b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 0x7f48f019b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 -- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0105970 msgr2=0x7f48f01a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0105970 0x7f48f01a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48ef400640 1 -- 172.21.15.138:0/3168401122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f01a1060 con 0x7f48f006a7a0 2024-08-20T01:12:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.254+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 0x7f48f019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48ef400640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005ab0 tx=0x7f48e0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48ecc00640 1 -- 172.21.15.138:0/3168401122 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e003f070 con 0x7f48f006a7a0 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48ecc00640 1 -- 172.21.15.138:0/3168401122 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e0031b60 con 0x7f48f006a7a0 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f01a12f0 con 0x7f48f006a7a0 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48ecc00640 1 -- 172.21.15.138:0/3168401122 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e0035030 con 0x7f48f006a7a0 2024-08-20T01:12:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.255+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a7a10 con 0x7f48f006a7a0 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 Processor -- start 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48ecc00640 1 -- 172.21.15.138:0/3168401122 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f48e0042070 con 0x7f48f006a7a0 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 -- start start 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587628530f0 0x55876285eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55876285f080 0x558762861470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587627c8210 con 0x5587628530f0 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5587627b7710 con 0x55876285f080 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48f96b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5587627bb750 con 0x5587628527d0 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48efe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48efe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44556/0 (socket says 172.21.15.138:44556) 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48efe00640 1 -- 172.21.15.138:0/4176904555 learned_addr learned my addr 172.21.15.138:0/4176904555 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.259+0000 7f48e7e00640 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55876285f080 0x558762861470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48ef400640 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587628530f0 0x55876285eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 -- 172.21.15.138:0/4176904555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55876285f080 msgr2=0x558762861470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55876285f080 0x558762861470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 -- 172.21.15.138:0/4176904555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587628530f0 msgr2=0x55876285eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587628530f0 0x55876285eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 -- 172.21.15.138:0/4176904555 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558762783de0 con 0x5587628527d0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48efe00640 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009f50 tx=0x7f48e80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccae8ceffc96efd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48e4c00640 1 -- 172.21.15.138:0/4176904555 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004d20 con 0x5587628527d0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48e4c00640 1 -- 172.21.15.138:0/4176904555 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e8004050 con 0x5587628527d0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.260+0000 7f48e4c00640 1 -- 172.21.15.138:0/4176904555 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8005280 con 0x5587628527d0 2024-08-20T01:12:32.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48ecc00640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 0x7f48d40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48ecc00640 1 -- 172.21.15.138:0/3168401122 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f48e0104a10 con 0x7f48f006a7a0 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48ef400640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 0x7f48d40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 -- 172.21.15.138:0/4176904555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 msgr2=0x558762852bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009f50 tx=0x7f48e80118d0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 -- 172.21.15.138:0/4176904555 shutdown_connections 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55876285f080 0x558762861470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587628530f0 0x55876285eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 --2- 172.21.15.138:0/4176904555 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762852bb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.261+0000 7f48f96b9740 1 -- 172.21.15.138:0/4176904555 >> 172.21.15.138:0/4176904555 conn(0x558762850600 msgr2=0x5587628517a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.262+0000 7f48ef400640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 0x7f48d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x558762855950 tx=0x7f48e00045e0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.262+0000 7f48f96b9740 1 -- 172.21.15.138:0/4176904555 shutdown_connections 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.262+0000 7f48f96b9740 1 -- 172.21.15.138:0/4176904555 wait complete. 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.262+0000 7f48f96b9740 1 -- start start 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 Processor -- start 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x55876285c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x55876285cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587626f0ab0 con 0x55876285f080 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55876275e140 con 0x5587628527d0 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558762861d60 con 0x5587628530f0 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48f96b9740 10 client.0 _ll_register_callbacks cb 0x5587627715d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:32.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44568/0 (socket says 172.21.15.138:44568) 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.263+0000 7f48e7e00640 1 -- 172.21.15.138:0/3703543978 learned_addr learned my addr 172.21.15.138:0/3703543978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48efe00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x55876285c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48efe00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x55876285cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 -- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 msgr2=0x55876285c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x55876285c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 -- 172.21.15.138:0/3703543978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 msgr2=0x55876285cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x55876285cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 -- 172.21.15.138:0/3703543978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55876275f170 con 0x5587628530f0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48efe00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x55876285cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48d8012d20 tx=0x7f48d800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c9acdd1bc12b62b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48dd600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8012110 con 0x5587628530f0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f48d8004050 con 0x5587628530f0 2024-08-20T01:12:32.267 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.264+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55876278ea30 con 0x5587628530f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f48d80122b0 con 0x5587628530f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 10 client.26068 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55876251a790 con 0x5587628530f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55876278ea30 con 0x5587628530f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f48d8010430 con 0x5587628530f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 10 client.26068 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 20 client.26068 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55876286f0f0 0x5587628714e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 4 client.26068 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.265+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587627ca160 con 0x55876286f0f0 2024-08-20T01:12:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.266+0000 7f48efe00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55876286f0f0 0x5587628714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.266+0000 7f48efe00640 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55876286f0f0 0x5587628714e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3357f28ed88c024d server_cookie=cabbf6df49acd9e7 in_seq=0 out_seq=0 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.266+0000 7f48dd600640 10 client.26068 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48dd600640 1 -- 172.21.15.138:0/3703543978 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587627ca160 con 0x55876286f0f0 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48dd600640 10 client.26068 handle_command_reply: tid=0 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 client.26068 shutdown 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 msgr2=0x55876285c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f48d8012d20 tx=0x7f48d800f790 comp rx=0 tx=0).stop 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 shutdown_connections 2024-08-20T01:12:32.270 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55876286f0f0 0x5587628714e0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x55876285cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x55876285c5a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48dd600640 0 client.26068 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3703543978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x55876285c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 >> 172.21.15.138:0/3703543978 conn(0x558762850600 msgr2=0x5587628561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.267+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 shutdown_connections 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.268+0000 7f48f96b9740 1 -- 172.21.15.138:0/3703543978 wait complete. 2024-08-20T01:12:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.268+0000 7f48f96b9740 20 client.26068 trim_cache size 0 max 16384 2024-08-20T01:12:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.285+0000 7f48f96b9740 1 Processor -- start 2024-08-20T01:12:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.285+0000 7f48f96b9740 1 -- start start 2024-08-20T01:12:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762856990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x558762877330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01b43d0 con 0x55876285f080 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01b1c40 con 0x5587628530f0 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48f96b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01afa90 con 0x5587628527d0 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762856990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34818/0 (socket says 172.21.15.138:34818) 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48e7e00640 1 -- 172.21.15.138:0/3110644048 learned_addr learned my addr 172.21.15.138:0/3110644048 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x558762877330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 -- 172.21.15.138:0/3110644048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 msgr2=0x558762876df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 -- 172.21.15.138:0/3110644048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 msgr2=0x558762877330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x558762877330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48efe00640 1 -- 172.21.15.138:0/3110644048 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55876285be40 con 0x5587628527d0 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.286+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.287+0000 7f48efe00640 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762856990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009f20 tx=0x7f48e8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=557bc3380aad7478 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.287+0000 7f48dfe00640 1 -- 172.21.15.138:0/3110644048 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8004050 con 0x5587628527d0 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.287+0000 7f48dfe00640 1 -- 172.21.15.138:0/3110644048 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e8004780 con 0x5587628527d0 2024-08-20T01:12:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.287+0000 7f48dfe00640 1 -- 172.21.15.138:0/3110644048 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8005030 con 0x5587628527d0 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- 172.21.15.138:0/3110644048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 msgr2=0x558762856990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762856990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009f20 tx=0x7f48e8009500 comp rx=0 tx=0).stop 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- 172.21.15.138:0/3110644048 shutdown_connections 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x558762877330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628530f0 0x558762876df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3110644048 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628527d0 0x558762856990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- 172.21.15.138:0/3110644048 >> 172.21.15.138:0/3110644048 conn(0x558762850600 msgr2=0x55876285deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- 172.21.15.138:0/3110644048 shutdown_connections 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- 172.21.15.138:0/3110644048 wait complete. 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 -- start start 2024-08-20T01:12:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.288+0000 7f48f96b9740 1 Processor -- start 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x558762863630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x5587628640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01b0100 con 0x55876285f080 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01b1570 con 0x5587628527d0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f48f01b07d0 con 0x5587628530f0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48f96b9740 10 client.0 _ll_register_callbacks cb 0x7f48d8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48e7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44586/0 (socket says 172.21.15.138:44586) 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48e7e00640 1 -- 172.21.15.138:0/3317063155 learned_addr learned my addr 172.21.15.138:0/3317063155 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48efe00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x558762863630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.289+0000 7f48efe00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x5587628640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 -- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 msgr2=0x558762863630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x558762863630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 -- 172.21.15.138:0/3317063155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 msgr2=0x5587628640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x5587628640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 -- 172.21.15.138:0/3317063155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558762593620 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48e7e00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48d8010a10 tx=0x7f48d8010cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507f72eb7ca713be server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48dd600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8010e60 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48e8004780 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f48d8003800 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f48d8003a00 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.290+0000 7f48f96b9740 10 client.26074 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5587628d9190 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e8004780 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f48d801c4b0 con 0x5587628530f0 2024-08-20T01:12:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 10 client.26074 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 20 client.26074 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587628d9990 0x5587628dbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 4 client.26074 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587627c5c00 con 0x5587628d9990 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.291+0000 7f48efe00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587628d9990 0x5587628dbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.292+0000 7f48efe00640 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587628d9990 0x5587628dbdb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac20f0f11cd3b66a server_cookie=f9615ae7e7acd100 in_seq=0 out_seq=0 2024-08-20T01:12:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.292+0000 7f48dd600640 10 client.26074 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.317+0000 7f48dd600640 1 -- 172.21.15.138:0/3317063155 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587627c5c00 con 0x5587628d9990 2024-08-20T01:12:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.317+0000 7f48dd600640 10 client.26074 handle_command_reply: tid=0 2024-08-20T01:12:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.317+0000 7f48f96b9740 1 client.26074 shutdown 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 msgr2=0x558762863b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48d8010a10 tx=0x7f48d8010cc0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 shutdown_connections 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587628d9990 0x5587628dbdb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55876285f080 0x5587628640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587628530f0 0x558762863b70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48dd600640 0 client.26074 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 --2- 172.21.15.138:0/3317063155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587628527d0 0x558762863630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 >> 172.21.15.138:0/3317063155 conn(0x558762850600 msgr2=0x5587628512e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 shutdown_connections 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 1 -- 172.21.15.138:0/3317063155 wait complete. 2024-08-20T01:12:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.318+0000 7f48f96b9740 20 client.26074 trim_cache size 0 max 16384 2024-08-20T01:12:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:12:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:12:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:12:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 msgr2=0x7f48d40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 0x7f48d40b6280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x558762855950 tx=0x7f48e00045e0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 msgr2=0x7f48f019afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005ab0 tx=0x7f48e0003c10 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 shutdown_connections 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f48d40b3dc0 0x7f48d40b6280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48f0105970 0x7f48f01a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48f0104fd0 0x7f48f019b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 --2- 172.21.15.138:0/3168401122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f48f006a7a0 0x7f48f019afc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 >> 172.21.15.138:0/3168401122 conn(0x7f48f0077270 msgr2=0x7f48f01081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 shutdown_connections 2024-08-20T01:12:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:32.319+0000 7f48f5200640 1 -- 172.21.15.138:0/3168401122 wait complete. 2024-08-20T01:12:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:32 smithi167 ceph-mon[21965]: pgmap v495: 129 pgs: 129 active+clean; 92 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:12:32.987 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:32 smithi138 ceph-mon[18076]: pgmap v495: 129 pgs: 129 active+clean; 92 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:12:32.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:32 smithi142 ceph-mon[21799]: pgmap v495: 129 pgs: 129 active+clean; 92 MiB data, 808 MiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:12:33.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 152655 1.97 MB/sec execute 45 sec latency 11.984 ms 2024-08-20T01:12:34.139 INFO:tasks.workunit.client.0.smithi138.stdout: 1 154038 2.07 MB/sec execute 46 sec latency 19.144 ms 2024-08-20T01:12:35.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 156478 2.32 MB/sec execute 47 sec latency 12.330 ms 2024-08-20T01:12:35.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:35 smithi167 ceph-mon[21965]: pgmap v496: 129 pgs: 129 active+clean; 97 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 496 KiB/s wr, 70 op/s 2024-08-20T01:12:35.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:35 smithi142 ceph-mon[21799]: pgmap v496: 129 pgs: 129 active+clean; 97 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 496 KiB/s wr, 70 op/s 2024-08-20T01:12:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:35 smithi138 ceph-mon[18076]: pgmap v496: 129 pgs: 129 active+clean; 97 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 560 KiB/s rd, 496 KiB/s wr, 70 op/s 2024-08-20T01:12:36.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 158875 2.50 MB/sec execute 48 sec latency 20.510 ms 2024-08-20T01:12:37.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 160583 2.63 MB/sec execute 49 sec latency 16.791 ms 2024-08-20T01:12:37.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:37 smithi167 ceph-mon[21965]: pgmap v497: 129 pgs: 129 active+clean; 100 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 KiB/s wr, 107 op/s 2024-08-20T01:12:37.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:37 smithi142 ceph-mon[21799]: pgmap v497: 129 pgs: 129 active+clean; 100 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 KiB/s wr, 107 op/s 2024-08-20T01:12:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:37 smithi138 ceph-mon[18076]: pgmap v497: 129 pgs: 129 active+clean; 100 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 797 KiB/s wr, 107 op/s 2024-08-20T01:12:38.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 162836 2.83 MB/sec execute 50 sec latency 295.085 ms 2024-08-20T01:12:39.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 164415 2.93 MB/sec execute 51 sec latency 11.900 ms 2024-08-20T01:12:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:39 smithi167 ceph-mon[21965]: pgmap v498: 129 pgs: 129 active+clean; 101 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-08-20T01:12:39.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:39 smithi142 ceph-mon[21799]: pgmap v498: 129 pgs: 129 active+clean; 101 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-08-20T01:12:40.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:39 smithi138 ceph-mon[18076]: pgmap v498: 129 pgs: 129 active+clean; 101 MiB data, 844 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 KiB/s wr, 108 op/s 2024-08-20T01:12:40.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 167055 3.20 MB/sec execute 52 sec latency 11.205 ms 2024-08-20T01:12:41.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 169067 3.27 MB/sec execute 53 sec latency 20.088 ms 2024-08-20T01:12:41.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:41 smithi167 ceph-mon[21965]: pgmap v499: 129 pgs: 129 active+clean; 104 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-08-20T01:12:41.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:41 smithi142 ceph-mon[21799]: pgmap v499: 129 pgs: 129 active+clean; 104 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-08-20T01:12:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:41 smithi138 ceph-mon[18076]: pgmap v499: 129 pgs: 129 active+clean; 104 MiB data, 863 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-08-20T01:12:42.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 170872 3.44 MB/sec execute 54 sec latency 17.245 ms 2024-08-20T01:12:43.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 171365 3.41 MB/sec execute 55 sec latency 431.319 ms 2024-08-20T01:12:43.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:43 smithi167 ceph-mon[21965]: pgmap v500: 129 pgs: 129 active+clean; 106 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-08-20T01:12:43.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:43 smithi142 ceph-mon[21799]: pgmap v500: 129 pgs: 129 active+clean; 106 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-08-20T01:12:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:43 smithi138 ceph-mon[18076]: pgmap v500: 129 pgs: 129 active+clean; 106 MiB data, 893 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-08-20T01:12:44.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 172609 3.44 MB/sec execute 56 sec latency 991.665 ms 2024-08-20T01:12:45.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 174380 3.59 MB/sec execute 57 sec latency 57.933 ms 2024-08-20T01:12:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 -- 172.21.15.138:0/1683259542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81141083f0 msgr2=0x7f8114108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 --2- 172.21.15.138:0/1683259542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81141083f0 0x7f8114108830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8100009900 tx=0x7f810002a410 comp rx=0 tx=0).stop 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 -- 172.21.15.138:0/1683259542 shutdown_connections 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 --2- 172.21.15.138:0/1683259542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8114108d70 0x7f811410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 --2- 172.21.15.138:0/1683259542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81141083f0 0x7f8114108830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 --2- 172.21.15.138:0/1683259542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 0x7f8114102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 -- 172.21.15.138:0/1683259542 >> 172.21.15.138:0/1683259542 conn(0x7f81140fe430 msgr2=0x7f8114100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 -- 172.21.15.138:0/1683259542 shutdown_connections 2024-08-20T01:12:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.443+0000 7f811ac00640 1 -- 172.21.15.138:0/1683259542 wait complete. 2024-08-20T01:12:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.444+0000 7f811ac00640 1 Processor -- start 2024-08-20T01:12:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.444+0000 7f811ac00640 1 -- start start 2024-08-20T01:12:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.444+0000 7f811ac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 0x7f811419b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f811ac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f811ac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8114108d70 0x7f81141a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f811ac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8114112a40 con 0x7f81141029f0 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f811ac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81141128c0 con 0x7f81141083f0 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f811ac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8114112bc0 con 0x7f8114108d70 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47818/0 (socket says 172.21.15.138:47818) 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 -- 172.21.15.138:0/2275395132 learned_addr learned my addr 172.21.15.138:0/2275395132 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113e00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 0x7f811419b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8118c00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8114108d70 0x7f81141a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 -- 172.21.15.138:0/2275395132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8114108d70 msgr2=0x7f81141a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8114108d70 0x7f81141a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 -- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 msgr2=0x7f811419b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 0x7f811419b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.445+0000 7f8113400640 1 -- 172.21.15.138:0/2275395132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81141a1230 con 0x7f81141083f0 2024-08-20T01:12:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f8113400640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8100005ab0 tx=0x7f8100003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f8110c00640 1 -- 172.21.15.138:0/2275395132 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810003f070 con 0x7f81141083f0 2024-08-20T01:12:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f8110c00640 1 -- 172.21.15.138:0/2275395132 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8100031b60 con 0x7f81141083f0 2024-08-20T01:12:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f8110c00640 1 -- 172.21.15.138:0/2275395132 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100035030 con 0x7f81141083f0 2024-08-20T01:12:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81141a14c0 con 0x7f81141083f0 2024-08-20T01:12:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.446+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a7be0 con 0x7f81141083f0 2024-08-20T01:12:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 Processor -- start 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 -- start start 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8110c00640 1 -- 172.21.15.138:0/2275395132 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8100042070 con 0x7f81141083f0 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd9a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cd01210 con 0x55ed7cd97e80 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed7ccf0710 con 0x55ed7cd8bef0 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f811daaa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ed7ccf4750 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8118c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8113e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8118c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59274/0 (socket says 172.21.15.138:59274) 2024-08-20T01:12:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8118c00640 1 -- 172.21.15.138:0/1056028413 learned_addr learned my addr 172.21.15.138:0/1056028413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.450+0000 7f8113e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47822/0 (socket says 172.21.15.138:47822) 2024-08-20T01:12:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8113400640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd9a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 -- 172.21.15.138:0/1056028413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 msgr2=0x55ed7cd97940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 -- 172.21.15.138:0/1056028413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 msgr2=0x55ed7cd9a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd9a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 -- 172.21.15.138:0/1056028413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7ccbcde0 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8113e00640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8113400640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd9a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.451+0000 7f8118c00640 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f810c009fd0 tx=0x7f810c0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86cf50da0e4f42d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8109200640 1 -- 172.21.15.138:0/1056028413 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004d20 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8109200640 1 -- 172.21.15.138:0/1056028413 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c004050 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8109200640 1 -- 172.21.15.138:0/1056028413 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c005280 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8110c00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 0x7f80f80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8110c00640 1 -- 172.21.15.138:0/2275395132 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f81001049c0 con 0x7f81141083f0 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.452+0000 7f8113e00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 0x7f80f80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f8113e00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 0x7f80f80b6280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55ed7cd8e750 tx=0x7f8104005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 -- 172.21.15.138:0/1056028413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 msgr2=0x55ed7cd8b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f810c009fd0 tx=0x7f810c0118d0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 -- 172.21.15.138:0/1056028413 shutdown_connections 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd9a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd97940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 --2- 172.21.15.138:0/1056028413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8b9b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 -- 172.21.15.138:0/1056028413 >> 172.21.15.138:0/1056028413 conn(0x55ed7cd893b0 msgr2=0x55ed7cd8a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 -- 172.21.15.138:0/1056028413 shutdown_connections 2024-08-20T01:12:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.453+0000 7f811daaa740 1 -- 172.21.15.138:0/1056028413 wait complete. 2024-08-20T01:12:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.454+0000 7f811daaa740 1 -- start start 2024-08-20T01:12:45.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.454+0000 7f811daaa740 1 Processor -- start 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd95540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cc29ab0 con 0x55ed7cd8bef0 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cc97140 con 0x55ed7cd97e80 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ed7cd9ab60 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f811daaa740 10 client.0 _ll_register_callbacks cb 0x55ed7ccaa5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59278/0 (socket says 172.21.15.138:59278) 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 -- 172.21.15.138:0/1590778516 learned_addr learned my addr 172.21.15.138:0/1590778516 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113400640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8118c00640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd95540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 -- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 msgr2=0x55ed7cd95540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd95540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 -- 172.21.15.138:0/1590778516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 msgr2=0x55ed7cd94db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd94db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113e00640 1 -- 172.21.15.138:0/1590778516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7cc98170 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.455+0000 7f8113400640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f8113e00640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8104009710 tx=0x7f8104004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e51e84695897be81 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f80fd600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810401ee40 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ed7ccc7a30 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8104018030 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8104034050 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f811daaa740 10 client.26086 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed7ca53790 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.456+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ed7ccc7a30 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f810400f4e0 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 10 client.26086 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 10 client.26086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 20 client.26086 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7cda7e00 0x55ed7cdaa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 4 client.26086 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed7cd03160 con 0x55ed7cda7e00 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f8113400640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7cda7e00 0x55ed7cdaa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f8113400640 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7cda7e00 0x55ed7cdaa1f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f783c31df90c773 server_cookie=58d30373a7c4bd3e in_seq=0 out_seq=0 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.457+0000 7f80fd600640 10 client.26086 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f80fd600640 1 -- 172.21.15.138:0/1590778516 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ed7cd03160 con 0x55ed7cda7e00 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f80fd600640 10 client.26086 handle_command_reply: tid=0 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 client.26086 shutdown 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 msgr2=0x55ed7cd97930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8104009710 tx=0x7f8104004730 comp rx=0 tx=0).stop 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 shutdown_connections 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7cda7e00 0x55ed7cdaa1f0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd95540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd94db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 --2- 172.21.15.138:0/1590778516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd97930 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f80fd600640 0 client.26086 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.458+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 >> 172.21.15.138:0/1590778516 conn(0x55ed7cd893b0 msgr2=0x55ed7cd8efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.459+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 shutdown_connections 2024-08-20T01:12:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.459+0000 7f811daaa740 1 -- 172.21.15.138:0/1590778516 wait complete. 2024-08-20T01:12:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.459+0000 7f811daaa740 20 client.26086 trim_cache size 0 max 16384 2024-08-20T01:12:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.474+0000 7f811daaa740 1 Processor -- start 2024-08-20T01:12:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.474+0000 7f811daaa740 1 -- start start 2024-08-20T01:12:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.474+0000 7f811daaa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.474+0000 7f811daaa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f811daaa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f811daaa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b4590 con 0x55ed7cd97e80 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f811daaa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b1e00 con 0x55ed7cd8bef0 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f811daaa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81141afc50 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47844/0 (socket says 172.21.15.138:47844) 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59292/0 (socket says 172.21.15.138:59292) 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 -- 172.21.15.138:0/3461542159 learned_addr learned my addr 172.21.15.138:0/3461542159 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8118c00640 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 -- 172.21.15.138:0/3461542159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 msgr2=0x55ed7cd8a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 -- 172.21.15.138:0/3461542159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 msgr2=0x55ed7cd935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 -- 172.21.15.138:0/3461542159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed7cacc620 con 0x55ed7cd8bef0 2024-08-20T01:12:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.475+0000 7f8113400640 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81000c7350 tx=0x7f810002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc57deaae85bb368 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.476+0000 7f80ffe00640 1 -- 172.21.15.138:0/3461542159 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100049040 con 0x55ed7cd8bef0 2024-08-20T01:12:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.476+0000 7f80ffe00640 1 -- 172.21.15.138:0/3461542159 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81000cf820 con 0x55ed7cd8bef0 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.476+0000 7f80ffe00640 1 -- 172.21.15.138:0/3461542159 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100101fe0 con 0x55ed7cd8bef0 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.476+0000 7f811daaa740 1 -- 172.21.15.138:0/3461542159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 msgr2=0x55ed7cd8a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.476+0000 7f811daaa740 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f81000c7350 tx=0x7f810002a410 comp rx=0 tx=0).stop 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 -- 172.21.15.138:0/3461542159 shutdown_connections 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd97e80 0x55ed7cd935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd8bef0 0x55ed7cd8a620 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 --2- 172.21.15.138:0/3461542159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd8a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 -- 172.21.15.138:0/3461542159 >> 172.21.15.138:0/3461542159 conn(0x55ed7cd893b0 msgr2=0x55ed7cd8ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 -- 172.21.15.138:0/3461542159 shutdown_connections 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 -- 172.21.15.138:0/3461542159 wait complete. 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 -- start start 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.477+0000 7f811daaa740 1 Processor -- start 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd91490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 0x55ed7cd9ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b02c0 con 0x55ed7cd8b5d0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b1730 con 0x55ed7cd97e80 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b0990 con 0x55ed7cd919d0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f811daaa740 10 client.0 _ll_register_callbacks cb 0x7f8104009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8113e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 0x55ed7cd9ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8113e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59738/0 (socket says 172.21.15.138:59738) 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8113e00640 1 -- 172.21.15.138:0/2262693314 learned_addr learned my addr 172.21.15.138:0/2262693314 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8113400640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd91490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 -- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 msgr2=0x55ed7cd91490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd91490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 -- 172.21.15.138:0/2262693314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 msgr2=0x55ed7cd90f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8118c00640 1 -- 172.21.15.138:0/2262693314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed7cd9f390 con 0x55ed7cd919d0 2024-08-20T01:12:45.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.478+0000 7f8113e00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f8118c00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 0x55ed7cd9ee50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f810c003fd0 tx=0x7f810c009d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e619819810b3fc19 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f80fd600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004d30 con 0x55ed7cd919d0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81000cf820 con 0x55ed7cd919d0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f810c004050 con 0x55ed7cd919d0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f810c035050 con 0x55ed7cd919d0 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f811daaa740 10 client.26092 fetch_fsmap learned FSMap version 31 2024-08-20T01:12:45.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ed7cda0680 con 0x55ed7cd919d0 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.479+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81000cf820 con 0x55ed7cd919d0 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f810c005400 con 0x55ed7cd919d0 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 10 client.26092 fetch_fsmap finished waiting for FSMap version 31 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 10 client.26092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 20 client.26092 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7ce12ad0 0x55ed7ce14ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 4 client.26092 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed7ccfec00 con 0x55ed7ce12ad0 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f8113e00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7ce12ad0 0x55ed7ce14ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:45.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f8113e00640 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7ce12ad0 0x55ed7ce14ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d43fed5a120a9a server_cookie=6389b5df3de15dce in_seq=0 out_seq=0 2024-08-20T01:12:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.480+0000 7f80fd600640 10 client.26092 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.556+0000 7f80fd600640 1 -- 172.21.15.138:0/2262693314 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed7ccfec00 con 0x55ed7ce12ad0 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.556+0000 7f80fd600640 10 client.26092 handle_command_reply: tid=0 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.556+0000 7f811daaa740 1 client.26092 shutdown 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 msgr2=0x55ed7cd9ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 0x55ed7cd9ee50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f810c003fd0 tx=0x7f810c009d60 comp rx=0 tx=0).stop 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 shutdown_connections 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ed7ce12ad0 0x55ed7ce14ec0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ed7cd919d0 0x55ed7cd9ee50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ed7cd97e80 0x55ed7cd91490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 --2- 172.21.15.138:0/2262693314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ed7cd8b5d0 0x55ed7cd90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 >> 172.21.15.138:0/2262693314 conn(0x55ed7cd893b0 msgr2=0x55ed7cd94240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f80fd600640 0 client.26092 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 shutdown_connections 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 1 -- 172.21.15.138:0/2262693314 wait complete. 2024-08-20T01:12:45.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.557+0000 7f811daaa740 20 client.26092 trim_cache size 0 max 16384 2024-08-20T01:12:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:12:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:12:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:12:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 msgr2=0x7f80f80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 0x7f80f80b6280 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55ed7cd8e750 tx=0x7f8104005f90 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 msgr2=0x7f811419b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8100005ab0 tx=0x7f8100003c10 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 shutdown_connections 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f80f80b3dc0 0x7f80f80b6280 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8114108d70 0x7f81141a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81141083f0 0x7f811419b6d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 --2- 172.21.15.138:0/2275395132 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81141029f0 0x7f811419b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 >> 172.21.15.138:0/2275395132 conn(0x7f81140fe430 msgr2=0x7f81141059d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.558+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 shutdown_connections 2024-08-20T01:12:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:45.559+0000 7f811ac00640 1 -- 172.21.15.138:0/2275395132 wait complete. 2024-08-20T01:12:45.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:45 smithi167 ceph-mon[21965]: pgmap v501: 129 pgs: 129 active+clean; 108 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-08-20T01:12:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:45 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:12:45.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:45 smithi142 ceph-mon[21799]: pgmap v501: 129 pgs: 129 active+clean; 108 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-08-20T01:12:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:45 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:12:46.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:45 smithi138 ceph-mon[18076]: pgmap v501: 129 pgs: 129 active+clean; 108 MiB data, 892 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-08-20T01:12:46.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:45 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:12:46.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 176834 3.75 MB/sec execute 58 sec latency 33.549 ms 2024-08-20T01:12:46.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:46 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:12:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:46 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:12:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:46 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:12:46.973 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:46 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:12:46.973 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:46 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:12:46.973 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:46 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:12:46.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:46 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:12:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:46 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:12:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:46 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:12:47.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 178434 3.83 MB/sec execute 59 sec latency 16.252 ms 2024-08-20T01:12:47.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:47 smithi167 ceph-mon[21965]: pgmap v502: 129 pgs: 129 active+clean; 111 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 227 op/s 2024-08-20T01:12:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:47 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:12:47.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:47 smithi142 ceph-mon[21799]: pgmap v502: 129 pgs: 129 active+clean; 111 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 227 op/s 2024-08-20T01:12:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:47 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:12:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:47 smithi138 ceph-mon[18076]: pgmap v502: 129 pgs: 129 active+clean; 111 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 227 op/s 2024-08-20T01:12:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:47 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:12:48.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 181174 4.04 MB/sec execute 60 sec latency 13.391 ms 2024-08-20T01:12:49.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 182806 4.07 MB/sec execute 61 sec latency 14.499 ms 2024-08-20T01:12:49.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:49 smithi167 ceph-mon[21965]: pgmap v503: 129 pgs: 129 active+clean; 110 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-08-20T01:12:49.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:49 smithi142 ceph-mon[21799]: pgmap v503: 129 pgs: 129 active+clean; 110 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-08-20T01:12:50.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:49 smithi138 ceph-mon[18076]: pgmap v503: 129 pgs: 129 active+clean; 110 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 206 op/s 2024-08-20T01:12:50.140 INFO:tasks.workunit.client.0.smithi138.stdout: 1 184927 4.23 MB/sec execute 62 sec latency 15.973 ms 2024-08-20T01:12:50.957 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:50 smithi138 ceph-mon[18076]: pgmap v504: 129 pgs: 129 active+clean; 112 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-08-20T01:12:50.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:50 smithi167 ceph-mon[21965]: pgmap v504: 129 pgs: 129 active+clean; 112 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-08-20T01:12:50.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:50 smithi142 ceph-mon[21799]: pgmap v504: 129 pgs: 129 active+clean; 112 MiB data, 916 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.5 MiB/s wr, 294 op/s 2024-08-20T01:12:51.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 187742 4.39 MB/sec execute 63 sec latency 16.016 ms 2024-08-20T01:12:52.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 189264 4.43 MB/sec execute 64 sec latency 12.000 ms 2024-08-20T01:12:53.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 191871 4.62 MB/sec execute 65 sec latency 23.367 ms 2024-08-20T01:12:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:53 smithi167 ceph-mon[21965]: pgmap v505: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-08-20T01:12:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:53 smithi142 ceph-mon[21799]: pgmap v505: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-08-20T01:12:54.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:53 smithi138 ceph-mon[18076]: pgmap v505: 129 pgs: 129 active+clean; 113 MiB data, 910 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-08-20T01:12:54.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 193989 4.66 MB/sec execute 66 sec latency 13.710 ms 2024-08-20T01:12:55.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 195778 4.77 MB/sec execute 67 sec latency 16.272 ms 2024-08-20T01:12:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:55 smithi167 ceph-mon[21965]: pgmap v506: 129 pgs: 129 active+clean; 120 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-08-20T01:12:55.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:55 smithi142 ceph-mon[21799]: pgmap v506: 129 pgs: 129 active+clean; 120 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-08-20T01:12:56.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:55 smithi138 ceph-mon[18076]: pgmap v506: 129 pgs: 129 active+clean; 120 MiB data, 949 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 2.3 MiB/s wr, 291 op/s 2024-08-20T01:12:56.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 198749 4.96 MB/sec execute 68 sec latency 13.451 ms 2024-08-20T01:12:57.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 200226 4.98 MB/sec execute 69 sec latency 15.502 ms 2024-08-20T01:12:57.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:57 smithi167 ceph-mon[21965]: pgmap v507: 129 pgs: 129 active+clean; 120 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-08-20T01:12:57.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:57 smithi142 ceph-mon[21799]: pgmap v507: 129 pgs: 129 active+clean; 120 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-08-20T01:12:58.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:57 smithi138 ceph-mon[18076]: pgmap v507: 129 pgs: 129 active+clean; 120 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.1 MiB/s wr, 277 op/s 2024-08-20T01:12:58.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 202617 5.11 MB/sec execute 70 sec latency 10.408 ms 2024-08-20T01:12:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 -- 172.21.15.138:0/3092064173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 msgr2=0x7f6e40111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 --2- 172.21.15.138:0/3092064173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e40111790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009ca0 tx=0x7f6e3802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 -- 172.21.15.138:0/3092064173 shutdown_connections 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 --2- 172.21.15.138:0/3092064173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e40111790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 --2- 172.21.15.138:0/3092064173 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 --2- 172.21.15.138:0/3092064173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 0x7f6e40104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 -- 172.21.15.138:0/3092064173 >> 172.21.15.138:0/3092064173 conn(0x7f6e401006b0 msgr2=0x7f6e40102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 -- 172.21.15.138:0/3092064173 shutdown_connections 2024-08-20T01:12:58.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.678+0000 7f6e46c00640 1 -- 172.21.15.138:0/3092064173 wait complete. 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 Processor -- start 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 -- start start 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 0x7f6e4019b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e401a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40114c70 con 0x7f6e40104c10 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40114af0 con 0x7f6e4010af90 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.679+0000 7f6e46c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40114df0 con 0x7f6e4010a610 2024-08-20T01:12:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33608/0 (socket says 172.21.15.138:33608) 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e3fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 0x7f6e4019b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 -- 172.21.15.138:0/903162892 learned_addr learned my addr 172.21.15.138:0/903162892 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e44c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e401a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 -- 172.21.15.138:0/903162892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 msgr2=0x7f6e401a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e401a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 -- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 msgr2=0x7f6e4019b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 0x7f6e4019b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 -- 172.21.15.138:0/903162892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401a1230 con 0x7f6e4010a610 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.680+0000 7f6e37e00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800e9f0 tx=0x7f6e2800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.681+0000 7f6e3d600640 1 -- 172.21.15.138:0/903162892 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800cd80 con 0x7f6e4010a610 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.681+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401a1520 con 0x7f6e4010a610 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.681+0000 7f6e3d600640 1 -- 172.21.15.138:0/903162892 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e280108e0 con 0x7f6e4010a610 2024-08-20T01:12:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.681+0000 7f6e3d600640 1 -- 172.21.15.138:0/903162892 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28018980 con 0x7f6e4010a610 2024-08-20T01:12:58.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.681+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401a7be0 con 0x7f6e4010a610 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 Processor -- start 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 -- start start 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e3d600640 1 -- 172.21.15.138:0/903162892 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6e28010430 con 0x7f6e4010a610 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641801f7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x5641801fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564180161210 con 0x5641801ebef0 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564180150710 con 0x5641801f7e80 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564180154750 con 0x5641801eb5d0 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x5641801fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33614/0 (socket says 172.21.15.138:33614) 2024-08-20T01:12:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e44c00640 1 -- 172.21.15.138:0/1558535370 learned_addr learned my addr 172.21.15.138:0/1558535370 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.684+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641801f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 -- 172.21.15.138:0/1558535370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 msgr2=0x5641801fa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x5641801fa270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 -- 172.21.15.138:0/1558535370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 msgr2=0x5641801f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641801f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 -- 172.21.15.138:0/1558535370 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56418011cde0 con 0x5641801eb5d0 2024-08-20T01:12:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009ca0 tx=0x7f6e380093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=909505680cda27e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e35200640 1 -- 172.21.15.138:0/1558535370 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38031cf0 con 0x5641801eb5d0 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e35200640 1 -- 172.21.15.138:0/1558535370 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e38036030 con 0x5641801eb5d0 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.685+0000 7f6e35200640 1 -- 172.21.15.138:0/1558535370 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e380042a0 con 0x5641801eb5d0 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e3d600640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 0x7f6e240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e3d600640 1 -- 172.21.15.138:0/903162892 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f6e280e45d0 con 0x7f6e4010a610 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 0x7f6e240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1558535370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 msgr2=0x5641801eb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009ca0 tx=0x7f6e380093e0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1558535370 shutdown_connections 2024-08-20T01:12:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x5641801fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641801f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1558535370 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801eb9b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.686+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1558535370 >> 172.21.15.138:0/1558535370 conn(0x5641801e93b0 msgr2=0x5641801ea5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.687+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1558535370 shutdown_connections 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.687+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1558535370 wait complete. 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.687+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 0x7f6e240b6200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5641801ee750 tx=0x7f6e30005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.687+0000 7f6e49bf1740 1 -- start start 2024-08-20T01:12:58.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 Processor -- start 2024-08-20T01:12:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801eb5d0 0x5641801f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564180089ab0 con 0x5641801eb5d0 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5641800f7140 con 0x5641801ebef0 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5641801fa980 con 0x5641801f7e80 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e49bf1740 10 client.0 _ll_register_callbacks cb 0x56418010a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.688+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52772/0 (socket says 172.21.15.138:52772) 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e37e00640 1 -- 172.21.15.138:0/1911249029 learned_addr learned my addr 172.21.15.138:0/1911249029 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33630/0 (socket says 172.21.15.138:33630) 2024-08-20T01:12:58.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801eb5d0 0x5641801f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 -- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 msgr2=0x5641801f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 -- 172.21.15.138:0/1911249029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801eb5d0 msgr2=0x5641801f78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801eb5d0 0x5641801f78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 -- 172.21.15.138:0/1911249029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5641800f8170 con 0x5641801f7e80 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e37e00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e44c00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e3802ade0 tx=0x7f6e3802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d371e44a66d7529 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38031660 con 0x5641801f7e80 2024-08-20T01:12:58.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564180127a30 con 0x5641801f7e80 2024-08-20T01:12:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.689+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f6e38031800 con 0x5641801f7e80 2024-08-20T01:12:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e3805d050 con 0x5641801f7e80 2024-08-20T01:12:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 10 client.26104 fetch_fsmap learned FSMap version 32 2024-08-20T01:12:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56417feb3790 con 0x5641801f7e80 2024-08-20T01:12:58.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564180127a30 con 0x5641801f7e80 2024-08-20T01:12:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f6e3804e070 con 0x5641801f7e80 2024-08-20T01:12:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 10 client.26104 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T01:12:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 10 client.26104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 20 client.26104 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:58.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641801fe950 0x564180200d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 4 client.26104 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:12:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.690+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564180163160 con 0x5641801fe950 2024-08-20T01:12:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.691+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641801fe950 0x564180200d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.691+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641801fe950 0x564180200d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571660dc2009f385 server_cookie=45574bd8b70d03ce in_seq=0 out_seq=0 2024-08-20T01:12:58.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.691+0000 7f6e2d600640 10 client.26104 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.692+0000 7f6e2d600640 1 -- 172.21.15.138:0/1911249029 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564180163160 con 0x5641801fe950 2024-08-20T01:12:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.692+0000 7f6e2d600640 10 client.26104 handle_command_reply: tid=0 2024-08-20T01:12:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.692+0000 7f6e49bf1740 1 client.26104 shutdown 2024-08-20T01:12:58.700 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.692+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 msgr2=0x5641801f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.692+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6e3802ade0 tx=0x7f6e3802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 shutdown_connections 2024-08-20T01:12:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641801fe950 0x564180200d60 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801f7e80 0x5641801f54b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.701 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x5641801f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/1911249029 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801eb5d0 0x5641801f78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e2d600640 0 client.26104 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 >> 172.21.15.138:0/1911249029 conn(0x5641801e93b0 msgr2=0x5641801eefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 shutdown_connections 2024-08-20T01:12:58.702 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 1 -- 172.21.15.138:0/1911249029 wait complete. 2024-08-20T01:12:58.703 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.693+0000 7f6e49bf1740 20 client.26104 trim_cache size 0 max 16384 2024-08-20T01:12:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.708+0000 7f6e49bf1740 1 Processor -- start 2024-08-20T01:12:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 -- start start 2024-08-20T01:12:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801f4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 0x564180222aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401b0090 con 0x5641801f7e80 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401ad930 con 0x5641801ebef0 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401ab780 con 0x5641801eb5d0 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e44c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 0x564180222aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:52780/0 (socket says 172.21.15.138:52780) 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 -- 172.21.15.138:0/87683609 learned_addr learned my addr 172.21.15.138:0/87683609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:58.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801f4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 -- 172.21.15.138:0/87683609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 msgr2=0x5641801f4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801f4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 -- 172.21.15.138:0/87683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 msgr2=0x564180222aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 0x564180222aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e37e00640 1 -- 172.21.15.138:0/87683609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56417ff2c620 con 0x5641801ebef0 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.709+0000 7f6e44c00640 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 0x564180222aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.710+0000 7f6e37e00640 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e280ea2f0 tx=0x7f6e28018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378dbc8d96766ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.710+0000 7f6e2fe00640 1 -- 172.21.15.138:0/87683609 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280aa6b0 con 0x5641801ebef0 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.710+0000 7f6e2fe00640 1 -- 172.21.15.138:0/87683609 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e28010720 con 0x5641801ebef0 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.710+0000 7f6e2fe00640 1 -- 172.21.15.138:0/87683609 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280e4260 con 0x5641801ebef0 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- 172.21.15.138:0/87683609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 msgr2=0x564180222560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6e280ea2f0 tx=0x7f6e28018420 comp rx=0 tx=0).stop 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- 172.21.15.138:0/87683609 shutdown_connections 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801f7e80 0x564180222aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801ebef0 0x564180222560 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/87683609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x5641801f4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- 172.21.15.138:0/87683609 >> 172.21.15.138:0/87683609 conn(0x5641801e93b0 msgr2=0x5641801ea450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- 172.21.15.138:0/87683609 shutdown_connections 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- 172.21.15.138:0/87683609 wait complete. 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 -- start start 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.711+0000 7f6e49bf1740 1 Processor -- start 2024-08-20T01:12:58.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641802021c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x564180281510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401abdf0 con 0x5641801ebef0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401ad260 con 0x5641801f7e80 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6e401ac4c0 con 0x5641801eb5d0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e49bf1740 10 client.0 _ll_register_callbacks cb 0x7f6e280ea420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33642/0 (socket says 172.21.15.138:33642) 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 -- 172.21.15.138:0/3217967700 learned_addr learned my addr 172.21.15.138:0/3217967700 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e44c00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x564180281510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e37e00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641802021c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 -- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 msgr2=0x564180281510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x564180281510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 -- 172.21.15.138:0/3217967700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 msgr2=0x5641802021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641802021c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e3fe00640 1 -- 172.21.15.138:0/3217967700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564180281c00 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.712+0000 7f6e37e00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641802021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e3fe00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x564180201460 tx=0x7f6e30004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88699cc179f999e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3002b420 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f6e3002b5c0 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56418015ec00 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6e3003a050 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e49bf1740 10 client.26110 fetch_fsmap learned FSMap version 32 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564180282f70 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56418015ec00 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.713+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7f6e30035930 con 0x5641801eb5d0 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 10 client.26110 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T01:12:58.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 10 client.26110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 20 client.26110 populate_metadata read hostname 'smithi138' 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641802836f0 0x564180285b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 4 client.26110 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56418015c340 con 0x5641802836f0 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e37e00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641802836f0 0x564180285b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e37e00640 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641802836f0 0x564180285b10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46bf9ab47fc68de server_cookie=cb8c9385c623565e in_seq=0 out_seq=0 2024-08-20T01:12:58.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.714+0000 7f6e2d600640 10 client.26110 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.795+0000 7f6e2d600640 1 -- 172.21.15.138:0/3217967700 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56418015c340 con 0x5641802836f0 2024-08-20T01:12:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.795+0000 7f6e2d600640 10 client.26110 handle_command_reply: tid=0 2024-08-20T01:12:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.795+0000 7f6e49bf1740 1 client.26110 shutdown 2024-08-20T01:12:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 msgr2=0x564180201c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.794 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x564180201460 tx=0x7f6e30004730 comp rx=0 tx=0).stop 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 shutdown_connections 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5641802836f0 0x564180285b10 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5641801f7e80 0x564180281510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5641801ebef0 0x5641802021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 --2- 172.21.15.138:0/3217967700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5641801eb5d0 0x564180201c80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 >> 172.21.15.138:0/3217967700 conn(0x5641801e93b0 msgr2=0x5641801ee780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e2d600640 0 client.26110 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 shutdown_connections 2024-08-20T01:12:58.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 1 -- 172.21.15.138:0/3217967700 wait complete. 2024-08-20T01:12:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.796+0000 7f6e49bf1740 20 client.26110 trim_cache size 0 max 16384 2024-08-20T01:12:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:12:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:12:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:12:58.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 msgr2=0x7f6e240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 0x7f6e240b6200 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x5641801ee750 tx=0x7f6e30005f90 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 msgr2=0x7f6e4019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800e9f0 tx=0x7f6e2800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 shutdown_connections 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6e240b3d40 0x7f6e240b6200 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6e4010af90 0x7f6e401a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e4010a610 0x7f6e4019b6d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 --2- 172.21.15.138:0/903162892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e40104c10 0x7f6e4019b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:12:58.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 >> 172.21.15.138:0/903162892 conn(0x7f6e401006b0 msgr2=0x7f6e40100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:12:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 shutdown_connections 2024-08-20T01:12:58.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:12:58.797+0000 7f6e46c00640 1 -- 172.21.15.138:0/903162892 wait complete. 2024-08-20T01:12:59.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 205065 5.20 MB/sec execute 71 sec latency 28.194 ms 2024-08-20T01:12:59.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:12:59 smithi167 ceph-mon[21965]: pgmap v508: 129 pgs: 129 active+clean; 121 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-08-20T01:12:59.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:12:59 smithi142 ceph-mon[21799]: pgmap v508: 129 pgs: 129 active+clean; 121 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-08-20T01:13:00.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:12:59 smithi138 ceph-mon[18076]: pgmap v508: 129 pgs: 129 active+clean; 121 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.6 MiB/s wr, 230 op/s 2024-08-20T01:13:00.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 206530 5.24 MB/sec execute 72 sec latency 16.887 ms 2024-08-20T01:13:01.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 209382 5.41 MB/sec execute 73 sec latency 11.526 ms 2024-08-20T01:13:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:01 smithi167 ceph-mon[21965]: pgmap v509: 129 pgs: 129 active+clean; 126 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-08-20T01:13:01.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:01 smithi142 ceph-mon[21799]: pgmap v509: 129 pgs: 129 active+clean; 126 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-08-20T01:13:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:01 smithi138 ceph-mon[18076]: pgmap v509: 129 pgs: 129 active+clean; 126 MiB data, 968 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 312 op/s 2024-08-20T01:13:02.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 209589 5.35 MB/sec execute 74 sec latency 73.622 ms 2024-08-20T01:13:03.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 210784 5.35 MB/sec execute 75 sec latency 278.607 ms 2024-08-20T01:13:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:03 smithi167 ceph-mon[21965]: pgmap v510: 129 pgs: 129 active+clean; 130 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 273 op/s 2024-08-20T01:13:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:03 smithi142 ceph-mon[21799]: pgmap v510: 129 pgs: 129 active+clean; 130 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 273 op/s 2024-08-20T01:13:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:03 smithi138 ceph-mon[18076]: pgmap v510: 129 pgs: 129 active+clean; 130 MiB data, 986 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.7 MiB/s wr, 273 op/s 2024-08-20T01:13:04.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 211420 5.29 MB/sec execute 76 sec latency 672.113 ms 2024-08-20T01:13:05.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 213299 5.39 MB/sec execute 77 sec latency 12.613 ms 2024-08-20T01:13:05.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:05 smithi167 ceph-mon[21965]: pgmap v511: 129 pgs: 129 active+clean; 130 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-08-20T01:13:05.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:05 smithi142 ceph-mon[21799]: pgmap v511: 129 pgs: 129 active+clean; 130 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-08-20T01:13:06.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:05 smithi138 ceph-mon[18076]: pgmap v511: 129 pgs: 129 active+clean; 130 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.0 MiB/s wr, 301 op/s 2024-08-20T01:13:06.141 INFO:tasks.workunit.client.0.smithi138.stdout: 1 216069 5.49 MB/sec execute 78 sec latency 12.704 ms 2024-08-20T01:13:07.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 217401 5.51 MB/sec execute 79 sec latency 13.109 ms 2024-08-20T01:13:07.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:07 smithi167 ceph-mon[21965]: pgmap v512: 129 pgs: 129 active+clean; 130 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-08-20T01:13:07.985 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:07 smithi138 ceph-mon[18076]: pgmap v512: 129 pgs: 129 active+clean; 130 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-08-20T01:13:07.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:07 smithi142 ceph-mon[21799]: pgmap v512: 129 pgs: 129 active+clean; 130 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 1.6 MiB/s wr, 255 op/s 2024-08-20T01:13:08.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 219793 5.61 MB/sec execute 80 sec latency 151.026 ms 2024-08-20T01:13:09.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 219797 5.54 MB/sec execute 81 sec latency 800.678 ms 2024-08-20T01:13:09.709 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:09 smithi138 ceph-mon[18076]: pgmap v513: 129 pgs: 129 active+clean; 134 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-08-20T01:13:09.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:09 smithi167 ceph-mon[21965]: pgmap v513: 129 pgs: 129 active+clean; 134 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-08-20T01:13:09.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:09 smithi142 ceph-mon[21799]: pgmap v513: 129 pgs: 129 active+clean; 134 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 274 op/s 2024-08-20T01:13:10.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220209 5.53 MB/sec execute 82 sec latency 1442.406 ms 2024-08-20T01:13:11.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.46 MB/sec execute 83 sec latency 559.074 ms 2024-08-20T01:13:11.918 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 -- 172.21.15.138:0/2349375748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410a410 msgr2=0x7fadd410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 --2- 172.21.15.138:0/2349375748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410a410 0x7fadd410a850 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009900 tx=0x7fadc002a410 comp rx=0 tx=0).stop 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 -- 172.21.15.138:0/2349375748 shutdown_connections 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 --2- 172.21.15.138:0/2349375748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd410ad90 0x7fadd4111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 --2- 172.21.15.138:0/2349375748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410a410 0x7fadd410a850 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 --2- 172.21.15.138:0/2349375748 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd4104a10 0x7fadd4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 -- 172.21.15.138:0/2349375748 >> 172.21.15.138:0/2349375748 conn(0x7fadd4100410 msgr2=0x7fadd4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 -- 172.21.15.138:0/2349375748 shutdown_connections 2024-08-20T01:13:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.920+0000 7faddae00640 1 -- 172.21.15.138:0/2349375748 wait complete. 2024-08-20T01:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.921+0000 7faddae00640 1 Processor -- start 2024-08-20T01:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.921+0000 7faddae00640 1 -- start start 2024-08-20T01:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 0x7fadd419af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd410a410 0x7fadd419b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 0x7fadd41a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4114a60 con 0x7fadd410ad90 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41148e0 con 0x7fadd4104a10 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7faddae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4114be0 con 0x7fadd410a410 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 0x7fadd419af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 0x7fadd41a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 0x7fadd419af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60238/0 (socket says 172.21.15.138:60238) 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadd9a00640 1 -- 172.21.15.138:0/3656331493 learned_addr learned my addr 172.21.15.138:0/3656331493 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadd9000640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd410a410 0x7fadd419b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 -- 172.21.15.138:0/3656331493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd410a410 msgr2=0x7fadd419b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd410a410 0x7fadd419b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 -- 172.21.15.138:0/3656331493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 msgr2=0x7fadd419af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 0x7fadd419af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.922+0000 7fadda400640 1 -- 172.21.15.138:0/3656331493 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd41a0fe0 con 0x7fadd410ad90 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7fadda400640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 0x7fadd41a0810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7fadd2000640 1 -- 172.21.15.138:0/3656331493 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00ce20 con 0x7fadd410ad90 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7fadd2000640 1 -- 172.21.15.138:0/3656331493 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadcc005020 con 0x7fadd410ad90 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7fadd2000640 1 -- 172.21.15.138:0/3656331493 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc010690 con 0x7fadd410ad90 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41a12d0 con 0x7fadd410ad90 2024-08-20T01:13:11.923 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.923+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd41a7990 con 0x7fadd410ad90 2024-08-20T01:13:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.926+0000 7faddf215740 1 Processor -- start 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.926+0000 7fadd2000640 1 -- 172.21.15.138:0/3656331493 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fadcc0054d0 con 0x7fadd410ad90 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 -- start start 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 0x55e71dbdcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbe93c0 0x55e71dbeb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e71db52210 con 0x55e71dbdcb10 2024-08-20T01:13:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e71db41710 con 0x55e71dbe93c0 2024-08-20T01:13:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7faddf215740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e71db45750 con 0x55e71dbdd430 2024-08-20T01:13:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45354/0 (socket says 172.21.15.138:45354) 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7fadd9a00640 1 -- 172.21.15.138:0/3030870724 learned_addr learned my addr 172.21.15.138:0/3030870724 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.927+0000 7fadd9000640 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 0x55e71dbdcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 -- 172.21.15.138:0/3030870724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbe93c0 msgr2=0x55e71dbeb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbe93c0 0x55e71dbeb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 -- 172.21.15.138:0/3030870724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 msgr2=0x55e71dbdcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 0x55e71dbdcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 -- 172.21.15.138:0/3030870724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71db0dde0 con 0x55e71dbdd430 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9000640 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 0x55e71dbdcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd9a00640 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009e10 tx=0x7fadc8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4472c11c8235d70 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadc6a00640 1 -- 172.21.15.138:0/3030870724 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc801e070 con 0x55e71dbdd430 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadc6a00640 1 -- 172.21.15.138:0/3030870724 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc8019ae0 con 0x55e71dbdd430 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadc6a00640 1 -- 172.21.15.138:0/3030870724 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc800fb40 con 0x55e71dbdd430 2024-08-20T01:13:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.928+0000 7fadd2000640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 0x7fadbc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7fadd2000640 1 -- 172.21.15.138:0/3656331493 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fadcc0e4a70 con 0x7fadd410ad90 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7fadd9000640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 0x7fadbc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 -- 172.21.15.138:0/3030870724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 msgr2=0x55e71dbe8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009e10 tx=0x7fadc8011810 comp rx=0 tx=0).stop 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 -- 172.21.15.138:0/3030870724 shutdown_connections 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbe93c0 0x55e71dbeb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe8e80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 --2- 172.21.15.138:0/3030870724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbdcb10 0x55e71dbdcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 -- 172.21.15.138:0/3030870724 >> 172.21.15.138:0/3030870724 conn(0x55e71dbda940 msgr2=0x55e71dbdbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 -- 172.21.15.138:0/3030870724 shutdown_connections 2024-08-20T01:13:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7faddf215740 1 -- 172.21.15.138:0/3030870724 wait complete. 2024-08-20T01:13:11.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.929+0000 7fadd9000640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 0x7fadbc0b6200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005ab0 tx=0x7fadc0005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 -- start start 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 Processor -- start 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdcb10 0x55e71dbe8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.933 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e71da7aab0 con 0x55e71dbe93c0 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e71dae8140 con 0x55e71dbdd430 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.930+0000 7faddf215740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e71dbec0a0 con 0x55e71dbdcb10 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7faddf215740 10 client.0 _ll_register_callbacks cb 0x55e71dafb5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadd9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadd9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60256/0 (socket says 172.21.15.138:60256) 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadd9000640 1 -- 172.21.15.138:0/1444607237 learned_addr learned my addr 172.21.15.138:0/1444607237 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadd9a00640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdcb10 0x55e71dbe8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 -- 172.21.15.138:0/1444607237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdcb10 msgr2=0x55e71dbe8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdcb10 0x55e71dbe8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 -- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 msgr2=0x55e71dbe5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 -- 172.21.15.138:0/1444607237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71dae9170 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadd9000640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadda400640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe6530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0e1e10 tx=0x7fadcc0aba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc29a635b424bdc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.931+0000 7fadbb400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadcc0b4380 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fadcc0a78c0 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc8019ae0 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fadcc0b3ef0 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7faddf215740 10 client.16110 fetch_fsmap learned FSMap version 32 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e71d8a4790 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc8019ae0 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fadcc0a9fa0 con 0x55e71dbe93c0 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.932+0000 7faddf215740 10 client.16110 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T01:13:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7faddf215740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7faddf215740 20 client.16110 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dbf9030 0x55e71dbfb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7faddf215740 4 client.16110 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e71db54160 con 0x55e71dbf9030 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7fadd9a00640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dbf9030 0x55e71dbfb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7fadd9a00640 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dbf9030 0x55e71dbfb420 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee2af7c0d537273a server_cookie=7f1922e49a210fd0 in_seq=0 out_seq=0 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.933+0000 7fadbb400640 10 client.16110 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7fadbb400640 1 -- 172.21.15.138:0/1444607237 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e71db54160 con 0x55e71dbf9030 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7fadbb400640 10 client.16110 handle_command_reply: tid=0 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7faddf215740 1 client.16110 shutdown 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 msgr2=0x55e71dbe6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe6530 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fadcc0e1e10 tx=0x7fadcc0aba90 comp rx=0 tx=0).stop 2024-08-20T01:13:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 shutdown_connections 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.934+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dbf9030 0x55e71dbfb420 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe6530 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdd430 0x55e71dbe5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 --2- 172.21.15.138:0/1444607237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdcb10 0x55e71dbe8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 >> 172.21.15.138:0/1444607237 conn(0x55e71dbda940 msgr2=0x55e71dbe04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7fadbb400640 0 client.16110 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 shutdown_connections 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 1 -- 172.21.15.138:0/1444607237 wait complete. 2024-08-20T01:13:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.935+0000 7faddf215740 20 client.16110 trim_cache size 0 max 16384 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 Processor -- start 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 -- start start 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dc00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41b4310 con 0x55e71dbe93c0 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41b1bb0 con 0x55e71dbdcb10 2024-08-20T01:13:11.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7faddf215740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41afa00 con 0x55e71dbdd430 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7fadda400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dc00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:60272/0 (socket says 172.21.15.138:60272) 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7fadd9a00640 1 -- 172.21.15.138:0/2009599184 learned_addr learned my addr 172.21.15.138:0/2009599184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.950+0000 7fadd9000640 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 -- 172.21.15.138:0/2009599184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 msgr2=0x55e71dbe14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 -- 172.21.15.138:0/2009599184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 msgr2=0x55e71dc00d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dc00d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 -- 172.21.15.138:0/2009599184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e71d91d620 con 0x55e71dbdcb10 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9000640 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:11.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadd9a00640 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadc800f810 tx=0x7fadc8011e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=615dc35da014fda5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadc6000640 1 -- 172.21.15.138:0/2009599184 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc801e040 con 0x55e71dbdcb10 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadc6000640 1 -- 172.21.15.138:0/2009599184 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc802f070 con 0x55e71dbdcb10 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.951+0000 7fadc6000640 1 -- 172.21.15.138:0/2009599184 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc8019430 con 0x55e71dbdcb10 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- 172.21.15.138:0/2009599184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 msgr2=0x55e71dbe5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fadc800f810 tx=0x7fadc8011e40 comp rx=0 tx=0).stop 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- 172.21.15.138:0/2009599184 shutdown_connections 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dc00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbe14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 --2- 172.21.15.138:0/2009599184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbe5860 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- 172.21.15.138:0/2009599184 >> 172.21.15.138:0/2009599184 conn(0x55e71dbda940 msgr2=0x55e71dbe0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:11.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- 172.21.15.138:0/2009599184 shutdown_connections 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- 172.21.15.138:0/2009599184 wait complete. 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 -- start start 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.952+0000 7faddf215740 1 Processor -- start 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbf1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbf18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41b0070 con 0x55e71dbe93c0 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41b14e0 con 0x55e71dbdcb10 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41b0740 con 0x55e71dbdd430 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7faddf215740 10 client.0 _ll_register_callbacks cb 0x7fadcc0a7190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7fadda400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7fadd9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbf1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7fadda400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51950/0 (socket says 172.21.15.138:51950) 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7fadda400640 1 -- 172.21.15.138:0/710218700 learned_addr learned my addr 172.21.15.138:0/710218700 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.953+0000 7fadd9000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbf18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 -- 172.21.15.138:0/710218700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 msgr2=0x55e71dbf18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbf18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 -- 172.21.15.138:0/710218700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 msgr2=0x55e71dbe2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 -- 172.21.15.138:0/710218700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e71dbe3eb0 con 0x55e71dbdcb10 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadda400640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadd9a00640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbf1360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadc801c040 tx=0x7fadc80055f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53321c8bb118fec2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadbb400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc803a070 con 0x55e71dbdcb10 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadc802f070 con 0x55e71dbdcb10 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fadc800fcc0 con 0x55e71dbdcb10 2024-08-20T01:13:11.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fadc801e040 con 0x55e71dbdcb10 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7faddf215740 10 client.26168 fetch_fsmap learned FSMap version 32 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.954+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e71dc63200 con 0x55e71dbdcb10 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc802f070 con 0x55e71dbdcb10 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) ==== 8358+0+0 (secure 0 0 0) 0x7fadc8025530 con 0x55e71dbdcb10 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 10 client.26168 fetch_fsmap finished waiting for FSMap version 32 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 10 client.26168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 20 client.26168 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dc63730 0x55e71dc65b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 4 client.26168 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e71db4fc00 con 0x55e71dc63730 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.955+0000 7fadd9000640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dc63730 0x55e71dc65b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.956+0000 7fadd9000640 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dc63730 0x55e71dc65b50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f00d8991d1534293 server_cookie=4347dbf8b8570af1 in_seq=0 out_seq=0 2024-08-20T01:13:11.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:11.956+0000 7fadbb400640 10 client.26168 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:11.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:11 smithi167 ceph-mon[21965]: pgmap v514: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-08-20T01:13:11.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:11 smithi142 ceph-mon[21799]: pgmap v514: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-08-20T01:13:12.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:11 smithi138 ceph-mon[18076]: pgmap v514: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.6 MiB/s wr, 331 op/s 2024-08-20T01:13:12.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.40 MB/sec execute 84 sec latency 1559.150 ms 2024-08-20T01:13:12.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:12 smithi142 ceph-mon[21799]: pgmap v515: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 245 op/s 2024-08-20T01:13:13.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:12 smithi138 ceph-mon[18076]: pgmap v515: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 245 op/s 2024-08-20T01:13:13.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.33 MB/sec execute 85 sec latency 2559.201 ms 2024-08-20T01:13:13.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:12 smithi167 ceph-mon[21965]: pgmap v515: 129 pgs: 129 active+clean; 140 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 245 op/s 2024-08-20T01:13:14.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.27 MB/sec execute 86 sec latency 3559.258 ms 2024-08-20T01:13:15.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.21 MB/sec execute 87 sec latency 4559.348 ms 2024-08-20T01:13:15.916 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:15 smithi167 ceph-mon[21965]: pgmap v516: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-08-20T01:13:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:15 smithi142 ceph-mon[21799]: pgmap v516: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-08-20T01:13:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:15 smithi138 ceph-mon[18076]: pgmap v516: 129 pgs: 129 active+clean; 141 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 222 op/s 2024-08-20T01:13:16.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 220561 5.15 MB/sec execute 88 sec latency 5559.481 ms 2024-08-20T01:13:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.203+0000 7fadbb400640 1 -- 172.21.15.138:0/710218700 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e71db4fc00 con 0x55e71dc63730 2024-08-20T01:13:16.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.203+0000 7fadbb400640 10 client.26168 handle_command_reply: tid=0 2024-08-20T01:13:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.203+0000 7faddf215740 1 client.26168 shutdown 2024-08-20T01:13:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.203+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 msgr2=0x55e71dbf1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.203+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbf1360 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fadc801c040 tx=0x7fadc80055f0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 shutdown_connections 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e71dc63730 0x55e71dc65b50 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e71dbe93c0 0x55e71dbe2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e71dbdd430 0x55e71dbf18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 --2- 172.21.15.138:0/710218700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e71dbdcb10 0x55e71dbf1360 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7fadbb400640 0 client.26168 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 >> 172.21.15.138:0/710218700 conn(0x55e71dbda940 msgr2=0x55e71dbe15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 shutdown_connections 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 1 -- 172.21.15.138:0/710218700 wait complete. 2024-08-20T01:13:16.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.204+0000 7faddf215740 20 client.26168 trim_cache size 0 max 16384 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 msgr2=0x7fadbc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 0x7fadbc0b6200 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fadc0005ab0 tx=0x7fadc0005fd0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 msgr2=0x7fadd41a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 0x7fadd41a0810 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fadcc009fd0 tx=0x7fadcc00ef10 comp rx=0 tx=0).stop 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 shutdown_connections 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fadbc0b3d40 0x7fadbc0b6200 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadd410ad90 0x7fadd41a0810 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fadd410a410 0x7fadd419b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 --2- 172.21.15.138:0/3656331493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadd4104a10 0x7fadd419af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 >> 172.21.15.138:0/3656331493 conn(0x7fadd4100410 msgr2=0x7fadd4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 shutdown_connections 2024-08-20T01:13:16.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:16.205+0000 7faddae00640 1 -- 172.21.15.138:0/3656331493 wait complete. 2024-08-20T01:13:17.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 222974 5.24 MB/sec execute 89 sec latency 5605.962 ms 2024-08-20T01:13:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:17 smithi167 ceph-mon[21965]: pgmap v517: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-08-20T01:13:17.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:17 smithi142 ceph-mon[21799]: pgmap v517: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-08-20T01:13:18.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:17 smithi138 ceph-mon[18076]: pgmap v517: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 200 op/s 2024-08-20T01:13:18.142 INFO:tasks.workunit.client.0.smithi138.stdout: 1 224437 5.26 MB/sec execute 90 sec latency 21.127 ms 2024-08-20T01:13:19.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 224581 5.21 MB/sec execute 91 sec latency 801.346 ms 2024-08-20T01:13:19.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:19 smithi167 ceph-mon[21965]: pgmap v518: 129 pgs: 129 active+clean; 144 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-08-20T01:13:19.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:19 smithi142 ceph-mon[21799]: pgmap v518: 129 pgs: 129 active+clean; 144 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-08-20T01:13:20.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:19 smithi138 ceph-mon[18076]: pgmap v518: 129 pgs: 129 active+clean; 144 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 180 op/s 2024-08-20T01:13:20.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 225072 5.19 MB/sec execute 92 sec latency 1555.470 ms 2024-08-20T01:13:20.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:20 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:20.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:20 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:21.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:20 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:21.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 227425 5.29 MB/sec execute 93 sec latency 14.838 ms 2024-08-20T01:13:21.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:21 smithi138 ceph-mon[18076]: pgmap v519: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-08-20T01:13:21.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:21 smithi167 ceph-mon[21965]: pgmap v519: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-08-20T01:13:21.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:21 smithi142 ceph-mon[21799]: pgmap v519: 129 pgs: 129 active+clean; 145 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 189 op/s 2024-08-20T01:13:22.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 228748 5.29 MB/sec execute 94 sec latency 16.686 ms 2024-08-20T01:13:23.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 230880 5.37 MB/sec execute 95 sec latency 13.293 ms 2024-08-20T01:13:23.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:23 smithi167 ceph-mon[21965]: pgmap v520: 129 pgs: 129 active+clean; 149 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-08-20T01:13:23.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:23 smithi142 ceph-mon[21799]: pgmap v520: 129 pgs: 129 active+clean; 149 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-08-20T01:13:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:23 smithi138 ceph-mon[18076]: pgmap v520: 129 pgs: 129 active+clean; 149 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 683 KiB/s rd, 2.5 MiB/s wr, 164 op/s 2024-08-20T01:13:24.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 230996 5.32 MB/sec execute 96 sec latency 807.601 ms 2024-08-20T01:13:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:24 smithi142 ceph-mon[21799]: pgmap v521: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-08-20T01:13:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:24 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:24.991 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:24 smithi138 ceph-mon[18076]: pgmap v521: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-08-20T01:13:24.991 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:24 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:25.143 INFO:tasks.workunit.client.0.smithi138.stdout: 1 230999 5.27 MB/sec execute 97 sec latency 1798.013 ms 2024-08-20T01:13:25.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:24 smithi167 ceph-mon[21965]: pgmap v521: 129 pgs: 129 active+clean; 148 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 2.6 MiB/s wr, 186 op/s 2024-08-20T01:13:25.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:24 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:13:26.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 232007 5.26 MB/sec execute 98 sec latency 279.145 ms 2024-08-20T01:13:27.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 234415 5.35 MB/sec execute 99 sec latency 22.006 ms 2024-08-20T01:13:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:27 smithi167 ceph-mon[21965]: pgmap v522: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 188 op/s 2024-08-20T01:13:27.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:27 smithi142 ceph-mon[21799]: pgmap v522: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 188 op/s 2024-08-20T01:13:28.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:27 smithi138 ceph-mon[18076]: pgmap v522: 129 pgs: 129 active+clean; 152 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 952 KiB/s rd, 2.9 MiB/s wr, 188 op/s 2024-08-20T01:13:28.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 235706 5.35 MB/sec execute 100 sec latency 20.885 ms 2024-08-20T01:13:29.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236017 5.30 MB/sec execute 101 sec latency 705.715 ms 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/3974615698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 msgr2=0x7f90c406aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- 172.21.15.138:0/3974615698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c406aeb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009860 tx=0x7f90b002a360 comp rx=0 tx=0).stop 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/3974615698 shutdown_connections 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- 172.21.15.138:0/3974615698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c410ef30 0x7f90c4111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- 172.21.15.138:0/3974615698 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c406aeb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- 172.21.15.138:0/3974615698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c41033c0 0x7f90c406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/3974615698 >> 172.21.15.138:0/3974615698 conn(0x7f90c4077270 msgr2=0x7f90c4077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/3974615698 shutdown_connections 2024-08-20T01:13:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/3974615698 wait complete. 2024-08-20T01:13:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 Processor -- start 2024-08-20T01:13:29.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- start start 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c41033c0 0x7f90c419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c410ef30 0x7f90c41a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4114a50 con 0x7f90c410ef30 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41148d0 con 0x7f90c41033c0 2024-08-20T01:13:29.324 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90c4114bd0 con 0x7f90c406aa70 2024-08-20T01:13:29.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:51670/0 (socket says 172.21.15.138:51670) 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 -- 172.21.15.138:0/46091482 learned_addr learned my addr 172.21.15.138:0/46091482 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c9800640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c410ef30 0x7f90c41a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c3e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c41033c0 0x7f90c419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 -- 172.21.15.138:0/46091482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c41033c0 msgr2=0x7f90c419b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c41033c0 0x7f90c419b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 -- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c410ef30 msgr2=0x7f90c41a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c410ef30 0x7f90c41a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 -- 172.21.15.138:0/46091482 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c41a1070 con 0x7f90c406aa70 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c8e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e9c0 tx=0x7f90b800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c1600640 1 -- 172.21.15.138:0/46091482 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b800cde0 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90c41a1360 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c1600640 1 -- 172.21.15.138:0/46091482 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b80035f0 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90c1600640 1 -- 172.21.15.138:0/46091482 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b8010690 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.322+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90c41a7a20 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c1600640 1 -- 172.21.15.138:0/46091482 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f90b8004000 con 0x7f90c406aa70 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 Processor -- start 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- start start 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedafe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedb27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608eed19210 con 0x5608eedb03c0 2024-08-20T01:13:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608eed08710 con 0x5608eeda4430 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5608eed0c750 con 0x5608eeda3b10 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:51674/0 (socket says 172.21.15.138:51674) 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/3726813909 learned_addr learned my addr 172.21.15.138:0/3726813909 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/3726813909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 msgr2=0x5608eedafe80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedafe80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/3726813909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 msgr2=0x5608eedb27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedb27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/3726813909 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eecd4de0 con 0x5608eeda3b10 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00a4b0 tx=0x7f90bc009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=809ea76f5d8c828f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90b6000640 1 -- 172.21.15.138:0/3726813909 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc013790 con 0x5608eeda3b10 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90b6000640 1 -- 172.21.15.138:0/3726813909 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90bc019070 con 0x5608eeda3b10 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90b6000640 1 -- 172.21.15.138:0/3726813909 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc014d30 con 0x5608eeda3b10 2024-08-20T01:13:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c1600640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 0x7f90ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c1600640 1 -- 172.21.15.138:0/46091482 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f90b80e46a0 con 0x7f90c406aa70 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c8e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 0x7f90ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- 172.21.15.138:0/3726813909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 msgr2=0x5608eeda3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90bc00a4b0 tx=0x7f90bc009650 comp rx=0 tx=0).stop 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- 172.21.15.138:0/3726813909 shutdown_connections 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedb27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedafe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- 172.21.15.138:0/3726813909 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda3b10 0x5608eeda3ef0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- 172.21.15.138:0/3726813909 >> 172.21.15.138:0/3726813909 conn(0x5608eeda1940 msgr2=0x5608eeda2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- 172.21.15.138:0/3726813909 shutdown_connections 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- 172.21.15.138:0/3726813909 wait complete. 2024-08-20T01:13:29.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c8e00640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 0x7f90ac0b6200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5608eeda6c90 tx=0x7f90b80aacd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- start start 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 Processor -- start 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedacfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5608eec41ab0 con 0x5608eedb03c0 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5608eecaf140 con 0x5608eeda3b10 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5608eedb30a0 con 0x5608eeda4430 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90ce858740 10 client.0 _ll_register_callbacks cb 0x5608eecc25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:29.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45238/0 (socket says 172.21.15.138:45238) 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/2060866184 learned_addr learned my addr 172.21.15.138:0/2060866184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c3e00640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedacfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/2060866184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 msgr2=0x5608eedacfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedacfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/2060866184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 msgr2=0x5608eedad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c9800640 1 -- 172.21.15.138:0/2060866184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eecb0170 con 0x5608eeda3b10 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.330+0000 7f90c3e00640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90c9800640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009de0 tx=0x7f90bc014890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9149d8ab493ac4a6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc0137a0 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f90bc013940 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90bc019070 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90bc013b40 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 10 client.26174 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5608eea6b790 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90bc019070 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f90bc020c30 con 0x5608eeda3b10 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 10 client.26174 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 20 client.26174 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eedbff50 0x5608eedc2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 4 client.26174 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608eed1b160 con 0x5608eedbff50 2024-08-20T01:13:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90c3e00640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eedbff50 0x5608eedc2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90c3e00640 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eedbff50 0x5608eedc2340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a9aa3b2d27a54df server_cookie=f29db5e255ad45f9 in_seq=0 out_seq=0 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 10 client.26174 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 1 -- 172.21.15.138:0/2060866184 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5608eed1b160 con 0x5608eedbff50 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 10 client.26174 handle_command_reply: tid=0 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 client.26174 shutdown 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 msgr2=0x5608eedafb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f90bc009de0 tx=0x7f90bc014890 comp rx=0 tx=0).stop 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 shutdown_connections 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eedbff50 0x5608eedc2340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eedad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedacfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 --2- 172.21.15.138:0/2060866184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedafb70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90aaa00640 0 client.26174 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 >> 172.21.15.138:0/2060866184 conn(0x5608eeda1940 msgr2=0x5608eeda74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 shutdown_connections 2024-08-20T01:13:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 1 -- 172.21.15.138:0/2060866184 wait complete. 2024-08-20T01:13:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.335+0000 7f90ce858740 20 client.26174 trim_cache size 0 max 16384 2024-08-20T01:13:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 Processor -- start 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 -- start start 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedc8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eedb03c0 0x5608eedb8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b4380 con 0x5608eeda3b10 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b1c20 con 0x5608eeda4430 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90ce858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41afa70 con 0x5608eedb03c0 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90c3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedc8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42180/0 (socket says 172.21.15.138:42180) 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.363+0000 7f90c9800640 1 -- 172.21.15.138:0/3592592787 learned_addr learned my addr 172.21.15.138:0/3592592787 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c9800640 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eedb03c0 0x5608eedb8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 -- 172.21.15.138:0/3592592787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eedb03c0 msgr2=0x5608eedb8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eedb03c0 0x5608eedb8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 -- 172.21.15.138:0/3592592787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 msgr2=0x5608eedc7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 -- 172.21.15.138:0/3592592787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eedacd90 con 0x5608eeda4430 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c9800640 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90c3e00640 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedc8060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f90b002a950 tx=0x7f90b002a360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e65a6adddad5faf2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90b5600640 1 -- 172.21.15.138:0/3592592787 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0037040 con 0x5608eeda4430 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90b5600640 1 -- 172.21.15.138:0/3592592787 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b0038040 con 0x5608eeda4430 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.364+0000 7f90b5600640 1 -- 172.21.15.138:0/3592592787 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b0003ac0 con 0x5608eeda4430 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 -- 172.21.15.138:0/3592592787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 msgr2=0x5608eedc8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.365 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedc8060 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f90b002a950 tx=0x7f90b002a360 comp rx=0 tx=0).stop 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 -- 172.21.15.138:0/3592592787 shutdown_connections 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eedb03c0 0x5608eedb8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda4430 0x5608eedc8060 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 --2- 172.21.15.138:0/3592592787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eeda3b10 0x5608eedc7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 -- 172.21.15.138:0/3592592787 >> 172.21.15.138:0/3592592787 conn(0x5608eeda1940 msgr2=0x5608eedaee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 -- 172.21.15.138:0/3592592787 shutdown_connections 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.365+0000 7f90ce858740 1 -- 172.21.15.138:0/3592592787 wait complete. 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.366+0000 7f90ce858740 1 -- start start 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.366+0000 7f90ce858740 1 Processor -- start 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedb55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedb5b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b00e0 con 0x5608eedb03c0 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b1550 con 0x5608eeda3b10 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f90c41b07b0 con 0x5608eeda4430 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90ce858740 10 client.0 _ll_register_callbacks cb 0x7f90bc009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:29.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90c9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedb55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90c3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedb5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90c9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42186/0 (socket says 172.21.15.138:42186) 2024-08-20T01:13:29.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.367+0000 7f90c9800640 1 -- 172.21.15.138:0/1616195970 learned_addr learned my addr 172.21.15.138:0/1616195970 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 -- 172.21.15.138:0/1616195970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 msgr2=0x5608eedb5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedb5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 -- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 msgr2=0x5608eedb55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedb55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 -- 172.21.15.138:0/1616195970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eeae4620 con 0x5608eedb03c0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c3e00640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedb5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90c9800640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90bc01e560 tx=0x7f90bc0011c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad0dff7a2ff1009 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90aaa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90bc01ea80 con 0x5608eedb03c0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90b0038040 con 0x5608eedb03c0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f90bc01ec20 con 0x5608eedb03c0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f90bc01ee20 con 0x5608eedb03c0 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90ce858740 10 client.16131 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:29.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.368+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5608eee2a0a0 con 0x5608eedb03c0 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90b0038040 con 0x5608eedb03c0 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f90bc020870 con 0x5608eedb03c0 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 10 client.16131 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 20 client.16131 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eee2a770 0x5608eee2cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 4 client.16131 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608eed16c00 con 0x5608eee2a770 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.369+0000 7f90c3e00640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eee2a770 0x5608eee2cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.370+0000 7f90c3e00640 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eee2a770 0x5608eee2cb90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0ace4a08afc00e7 server_cookie=5e117f2da524b463 in_seq=0 out_seq=0 2024-08-20T01:13:29.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.370+0000 7f90aaa00640 10 client.16131 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90aaa00640 1 -- 172.21.15.138:0/1616195970 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608eed16c00 con 0x5608eee2a770 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90aaa00640 10 client.16131 handle_command_reply: tid=0 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 client.16131 shutdown 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 msgr2=0x5608eee28af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90bc01e560 tx=0x7f90bc0011c0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 shutdown_connections 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5608eee2a770 0x5608eee2cb90 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5608eedb03c0 0x5608eee28af0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90aaa00640 0 client.16131 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:29.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5608eeda4430 0x5608eedb5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 --2- 172.21.15.138:0/1616195970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5608eeda3b10 0x5608eedb55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 >> 172.21.15.138:0/1616195970 conn(0x5608eeda1940 msgr2=0x5608eeda2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 shutdown_connections 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 1 -- 172.21.15.138:0/1616195970 wait complete. 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.451+0000 7f90ce858740 20 client.16131 trim_cache size 0 max 16384 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:13:29.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 msgr2=0x7f90ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 0x7f90ac0b6200 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5608eeda6c90 tx=0x7f90b80aacd0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 msgr2=0x7f90c419afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f90b800e9c0 tx=0x7f90b800ee80 comp rx=0 tx=0).stop 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 shutdown_connections 2024-08-20T01:13:29.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f90ac0b3d40 0x7f90ac0b6200 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90c410ef30 0x7f90c41a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90c41033c0 0x7f90c419b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 --2- 172.21.15.138:0/46091482 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f90c406aa70 0x7f90c419afd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 >> 172.21.15.138:0/46091482 conn(0x7f90c4077270 msgr2=0x7f90c41008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 shutdown_connections 2024-08-20T01:13:29.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:29.452+0000 7f90ca200640 1 -- 172.21.15.138:0/46091482 wait complete. 2024-08-20T01:13:29.944 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:29 smithi138 ceph-mon[18076]: pgmap v523: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.6 MiB/s wr, 174 op/s 2024-08-20T01:13:29.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:29 smithi167 ceph-mon[21965]: pgmap v523: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.6 MiB/s wr, 174 op/s 2024-08-20T01:13:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:29 smithi142 ceph-mon[21799]: pgmap v523: 129 pgs: 129 active+clean; 153 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 943 KiB/s rd, 2.6 MiB/s wr, 174 op/s 2024-08-20T01:13:30.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236017 5.25 MB/sec execute 102 sec latency 1705.758 ms 2024-08-20T01:13:31.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236018 5.20 MB/sec execute 103 sec latency 1868.770 ms 2024-08-20T01:13:31.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:31 smithi167 ceph-mon[21965]: pgmap v524: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-08-20T01:13:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:31 smithi142 ceph-mon[21799]: pgmap v524: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-08-20T01:13:32.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:31 smithi138 ceph-mon[18076]: pgmap v524: 129 pgs: 129 active+clean; 155 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 246 op/s 2024-08-20T01:13:32.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236018 5.15 MB/sec execute 104 sec latency 1837.206 ms 2024-08-20T01:13:33.144 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236018 5.10 MB/sec execute 105 sec latency 2837.326 ms 2024-08-20T01:13:33.831 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:33 smithi138 ceph-mon[18076]: pgmap v525: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-08-20T01:13:33.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:33 smithi167 ceph-mon[21965]: pgmap v525: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-08-20T01:13:33.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:33 smithi142 ceph-mon[21799]: pgmap v525: 129 pgs: 129 active+clean; 150 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 219 op/s 2024-08-20T01:13:34.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 236018 5.05 MB/sec execute 106 sec latency 3842.156 ms 2024-08-20T01:13:34.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:34 smithi167 ceph-mon[21965]: pgmap v526: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-08-20T01:13:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:34 smithi142 ceph-mon[21799]: pgmap v526: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-08-20T01:13:35.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:34 smithi138 ceph-mon[18076]: pgmap v526: 129 pgs: 129 active+clean; 147 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 2.3 MiB/s wr, 193 op/s 2024-08-20T01:13:35.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 237357 5.08 MB/sec execute 107 sec latency 4426.305 ms 2024-08-20T01:13:36.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 238675 5.09 MB/sec execute 108 sec latency 33.855 ms 2024-08-20T01:13:37.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 241371 5.20 MB/sec execute 109 sec latency 15.541 ms 2024-08-20T01:13:37.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:37 smithi167 ceph-mon[21965]: pgmap v527: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-08-20T01:13:37.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:37 smithi142 ceph-mon[21799]: pgmap v527: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-08-20T01:13:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:37 smithi138 ceph-mon[18076]: pgmap v527: 129 pgs: 129 active+clean; 145 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 172 op/s 2024-08-20T01:13:38.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 242710 5.21 MB/sec execute 110 sec latency 18.988 ms 2024-08-20T01:13:39.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 243533 5.18 MB/sec execute 111 sec latency 257.621 ms 2024-08-20T01:13:39.930 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:39 smithi138 ceph-mon[18076]: pgmap v528: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2024-08-20T01:13:39.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:39 smithi167 ceph-mon[21965]: pgmap v528: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2024-08-20T01:13:39.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:39 smithi142 ceph-mon[21799]: pgmap v528: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 1.6 MiB/s wr, 145 op/s 2024-08-20T01:13:40.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 243533 5.13 MB/sec execute 112 sec latency 1257.693 ms 2024-08-20T01:13:41.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 245262 5.20 MB/sec execute 113 sec latency 1335.494 ms 2024-08-20T01:13:41.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:41 smithi167 ceph-mon[21965]: pgmap v529: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-08-20T01:13:41.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:41 smithi142 ceph-mon[21799]: pgmap v529: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-08-20T01:13:42.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:41 smithi138 ceph-mon[18076]: pgmap v529: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 980 KiB/s rd, 2.6 MiB/s wr, 214 op/s 2024-08-20T01:13:42.149 INFO:tasks.workunit.client.0.smithi138.stdout: 1 248009 5.27 MB/sec execute 114 sec latency 22.461 ms 2024-08-20T01:13:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 -- 172.21.15.138:0/2219678393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 msgr2=0x7f82b4108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 --2- 172.21.15.138:0/2219678393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b4108640 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009860 tx=0x7f82a002a340 comp rx=0 tx=0).stop 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 -- 172.21.15.138:0/2219678393 shutdown_connections 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 --2- 172.21.15.138:0/2219678393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4108b80 0x7f82b410f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 --2- 172.21.15.138:0/2219678393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b4108640 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 --2- 172.21.15.138:0/2219678393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4102800 0x7f82b4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 -- 172.21.15.138:0/2219678393 >> 172.21.15.138:0/2219678393 conn(0x7f82b40fe280 msgr2=0x7f82b41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 -- 172.21.15.138:0/2219678393 shutdown_connections 2024-08-20T01:13:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.586+0000 7f82b3200640 1 -- 172.21.15.138:0/2219678393 wait complete. 2024-08-20T01:13:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.587+0000 7f82b3200640 1 Processor -- start 2024-08-20T01:13:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.587+0000 7f82b3200640 1 -- start start 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b419b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4108b80 0x7f82b41a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4112800 con 0x7f82b4108b80 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4112680 con 0x7f82b4102800 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b3200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82b4112980 con 0x7f82b4108200 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50598/0 (socket says 172.21.15.138:50598) 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 -- 172.21.15.138:0/3819271012 learned_addr learned my addr 172.21.15.138:0/3819271012 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1400640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b419b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b2800640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4108b80 0x7f82b41a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 -- 172.21.15.138:0/3819271012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 msgr2=0x7f82b419b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b419b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 -- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4108b80 msgr2=0x7f82b41a08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4108b80 0x7f82b41a08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1e00640 1 -- 172.21.15.138:0/3819271012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b41a1080 con 0x7f82b4102800 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.588+0000 7f82b1400640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82b1e00640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f82a400e9c0 tx=0x7f82a400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82aaa00640 1 -- 172.21.15.138:0/3819271012 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a400cde0 con 0x7f82b4102800 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82aaa00640 1 -- 172.21.15.138:0/3819271012 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a40035f0 con 0x7f82b4102800 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b41a1370 con 0x7f82b4102800 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82aaa00640 1 -- 172.21.15.138:0/3819271012 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a4010690 con 0x7f82b4102800 2024-08-20T01:13:42.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.589+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b41a7a30 con 0x7f82b4102800 2024-08-20T01:13:42.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.592+0000 7f82aaa00640 1 -- 172.21.15.138:0/3819271012 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f82a4004000 con 0x7f82b4102800 2024-08-20T01:13:42.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 Processor -- start 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 -- start start 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae39e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae3c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba0ada3210 con 0x55ba0ae2db10 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba0ad92710 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.593+0000 7f82bb922740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba0ad96750 con 0x55ba0ae2e430 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae39e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59496/0 (socket says 172.21.15.138:59496) 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b2800640 1 -- 172.21.15.138:0/1258067080 learned_addr learned my addr 172.21.15.138:0/1258067080 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:42.592 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae3c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 -- 172.21.15.138:0/1258067080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 msgr2=0x55ba0ae39e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae39e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 -- 172.21.15.138:0/1258067080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 msgr2=0x55ba0ae2def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 -- 172.21.15.138:0/1258067080 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba0ad5ede0 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1e00640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae39e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b2800640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.594+0000 7f82b1400640 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae3c7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82a0035040 tx=0x7f82a0009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e1b25a1981b275d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f829f400640 1 -- 172.21.15.138:0/1258067080 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0041020 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f82aaa00640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 0x7f82980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f829f400640 1 -- 172.21.15.138:0/1258067080 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0030dd0 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f829f400640 1 -- 172.21.15.138:0/1258067080 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0037040 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f82aaa00640 1 -- 172.21.15.138:0/3819271012 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82a40e46a0 con 0x7f82b4102800 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.595+0000 7f82b2800640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 0x7f82980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- 172.21.15.138:0/1258067080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 msgr2=0x55ba0ae3c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae3c7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f82a0035040 tx=0x7f82a0009490 comp rx=0 tx=0).stop 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- 172.21.15.138:0/1258067080 shutdown_connections 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae3c7b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae39e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 --2- 172.21.15.138:0/1258067080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae2def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- 172.21.15.138:0/1258067080 >> 172.21.15.138:0/1258067080 conn(0x55ba0ae2b940 msgr2=0x55ba0ae2cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82b2800640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 0x7f82980b6200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f82ac011890 tx=0x7f82ac009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- 172.21.15.138:0/1258067080 shutdown_connections 2024-08-20T01:13:42.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- 172.21.15.138:0/1258067080 wait complete. 2024-08-20T01:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.596+0000 7f82bb922740 1 -- start start 2024-08-20T01:13:42.598 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 Processor -- start 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae39b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae374d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ba0accbab0 con 0x55ba0ae2db10 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ba0ad39140 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ba0ae3d040 con 0x55ba0ae2e430 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.597+0000 7f82bb922740 10 client.0 _ll_register_callbacks cb 0x55ba0ad4c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae39b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39142/0 (socket says 172.21.15.138:39142) 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 -- 172.21.15.138:0/749213256 learned_addr learned my addr 172.21.15.138:0/749213256 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:42.599 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b2800640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae374d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 -- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 msgr2=0x55ba0ae374d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae374d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 -- 172.21.15.138:0/749213256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 msgr2=0x55ba0ae39b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae39b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 -- 172.21.15.138:0/749213256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba0ad3a170 con 0x55ba0ae2e430 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b2800640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f82b1400640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82a002a860 tx=0x7f82a0030f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91a854e5be1c9d70 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.598+0000 7f8293e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a004a030 con 0x55ba0ae2e430 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82a0004250 con 0x55ba0ae2e430 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a0030dd0 con 0x55ba0ae2e430 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82a0057050 con 0x55ba0ae2e430 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f82bb922740 10 client.26158 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:42.600 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.599+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ba0aaf5790 con 0x55ba0ae2e430 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0030dd0 con 0x55ba0ae2e430 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f82a0003820 con 0x55ba0ae2e430 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 10 client.26158 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 10 client.26158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 20 client.26158 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0ae49d20 0x55ba0ae4c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 4 client.26158 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba0ada5160 con 0x55ba0ae49d20 2024-08-20T01:13:42.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.600+0000 7f82b1e00640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0ae49d20 0x55ba0ae4c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.601+0000 7f82b1e00640 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0ae49d20 0x55ba0ae4c110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738870f8374420a server_cookie=b0ad2b1a2f1f85ee in_seq=0 out_seq=0 2024-08-20T01:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.601+0000 7f8293e00640 10 client.26158 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.601+0000 7f8293e00640 1 -- 172.21.15.138:0/749213256 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ba0ada5160 con 0x55ba0ae49d20 2024-08-20T01:13:42.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.601+0000 7f8293e00640 10 client.26158 handle_command_reply: tid=0 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 client.26158 shutdown 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 msgr2=0x55ba0ae36f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f82a002a860 tx=0x7f82a0030f80 comp rx=0 tx=0).stop 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 shutdown_connections 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0ae49d20 0x55ba0ae4c110 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae374d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae36f90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f8293e00640 0 client.26158 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 --2- 172.21.15.138:0/749213256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae39b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 >> 172.21.15.138:0/749213256 conn(0x55ba0ae2b940 msgr2=0x55ba0ae314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 shutdown_connections 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 1 -- 172.21.15.138:0/749213256 wait complete. 2024-08-20T01:13:42.603 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.602+0000 7f82bb922740 20 client.26158 trim_cache size 0 max 16384 2024-08-20T01:13:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.620+0000 7f82bb922740 1 Processor -- start 2024-08-20T01:13:42.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.620+0000 7f82bb922740 1 -- start start 2024-08-20T01:13:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2db10 0x55ba0ae32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae51c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41b4370 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41b1c10 con 0x55ba0ae2e430 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82bb922740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41afa60 con 0x55ba0ae2db10 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82b1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82b1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50640/0 (socket says 172.21.15.138:50640) 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82b1400640 1 -- 172.21.15.138:0/4111665184 learned_addr learned my addr 172.21.15.138:0/4111665184 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82b1e00640 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2db10 0x55ba0ae32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.621+0000 7f82b2800640 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae51c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.622+0000 7f82b1400640 1 -- 172.21.15.138:0/4111665184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2db10 msgr2=0x55ba0ae32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.622+0000 7f82b1400640 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2db10 0x55ba0ae32540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.622+0000 7f82b1400640 1 -- 172.21.15.138:0/4111665184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae3a3c0 msgr2=0x55ba0ae51c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.622+0000 7f82b1400640 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae51c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.622+0000 7f82b1400640 1 -- 172.21.15.138:0/4111665184 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba0ab6e620 con 0x55ba0ae2e430 2024-08-20T01:13:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.623+0000 7f82b1400640 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82a0001eb0 tx=0x7f82a00058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b2eccd52e5a8ee1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.623+0000 7f829ea00640 1 -- 172.21.15.138:0/4111665184 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0000e60 con 0x55ba0ae2e430 2024-08-20T01:13:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.623+0000 7f829ea00640 1 -- 172.21.15.138:0/4111665184 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0030b50 con 0x55ba0ae2e430 2024-08-20T01:13:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.623+0000 7f829ea00640 1 -- 172.21.15.138:0/4111665184 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a004a070 con 0x55ba0ae2e430 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 -- 172.21.15.138:0/4111665184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 msgr2=0x55ba0ae51710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f82a0001eb0 tx=0x7f82a00058d0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 -- 172.21.15.138:0/4111665184 shutdown_connections 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae51c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae2e430 0x55ba0ae51710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 --2- 172.21.15.138:0/4111665184 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2db10 0x55ba0ae32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 -- 172.21.15.138:0/4111665184 >> 172.21.15.138:0/4111665184 conn(0x55ba0ae2b940 msgr2=0x55ba0ae31e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 -- 172.21.15.138:0/4111665184 shutdown_connections 2024-08-20T01:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.624+0000 7f82bb922740 1 -- 172.21.15.138:0/4111665184 wait complete. 2024-08-20T01:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 -- start start 2024-08-20T01:13:42.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 Processor -- start 2024-08-20T01:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae42b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41b00d0 con 0x55ba0ae2db10 2024-08-20T01:13:42.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41b1540 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.625+0000 7f82bb922740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82b41b07a0 con 0x55ba0ae2e430 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82bb922740 10 client.0 _ll_register_callbacks cb 0x7f82ac009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50646/0 (socket says 172.21.15.138:50646) 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b1400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39164/0 (socket says 172.21.15.138:39164) 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 -- 172.21.15.138:0/1772067693 learned_addr learned my addr 172.21.15.138:0/1772067693 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b1e00640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae42b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 -- 172.21.15.138:0/1772067693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 msgr2=0x55ba0ae31a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 -- 172.21.15.138:0/1772067693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 msgr2=0x55ba0ae42b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:42.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae42b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b2800640 1 -- 172.21.15.138:0/1772067693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba0ae3e5f0 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b1400640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.626+0000 7f82b1e00640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae42b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.627+0000 7f82b2800640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00fc60 tx=0x7f82ac004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bb21b8945aeadd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.627+0000 7f8293e00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.627+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac004b20 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.627+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82ac004cc0 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.627+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82a0030b50 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82ac004040 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 10 client.26195 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ba0aeb4a00 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82a0030b50 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f82ac027070 con 0x55ba0ae3a3c0 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 10 client.26195 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 10 client.26195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 20 client.26195 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:42.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0aeb5290 0x55ba0aeb76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 4 client.26195 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.628+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba0ae36d30 con 0x55ba0aeb5290 2024-08-20T01:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.629+0000 7f82b1e00640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0aeb5290 0x55ba0aeb76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.629+0000 7f82b1e00640 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0aeb5290 0x55ba0aeb76b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9daefbe49f7d6c7c server_cookie=73b31aea188d4490 in_seq=0 out_seq=0 2024-08-20T01:13:42.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.629+0000 7f8293e00640 10 client.26195 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.680+0000 7f8293e00640 1 -- 172.21.15.138:0/1772067693 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ba0ae36d30 con 0x55ba0aeb5290 2024-08-20T01:13:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.680+0000 7f8293e00640 10 client.26195 handle_command_reply: tid=0 2024-08-20T01:13:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.680+0000 7f82bb922740 1 client.26195 shutdown 2024-08-20T01:13:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 msgr2=0x55ba0ae32200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00fc60 tx=0x7f82ac004ab0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 shutdown_connections 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ba0aeb5290 0x55ba0aeb76b0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba0ae3a3c0 0x55ba0ae32200 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ba0ae2e430 0x55ba0ae31a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 --2- 172.21.15.138:0/1772067693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ba0ae2db10 0x55ba0ae42b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 >> 172.21.15.138:0/1772067693 conn(0x55ba0ae2b940 msgr2=0x55ba0ae38de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f8293e00640 0 client.26195 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 shutdown_connections 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 1 -- 172.21.15.138:0/1772067693 wait complete. 2024-08-20T01:13:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.681+0000 7f82bb922740 20 client.26195 trim_cache size 0 max 16384 2024-08-20T01:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:13:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.682+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 msgr2=0x7f82980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.682+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 0x7f82980b6200 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f82ac011890 tx=0x7f82ac009500 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.682+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 msgr2=0x7f82b419afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.682+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f82a400e9c0 tx=0x7f82a400ee80 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 shutdown_connections 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82980b3d40 0x7f82980b6200 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82b4108b80 0x7f82b41a08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82b4108200 0x7f82b419b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 --2- 172.21.15.138:0/3819271012 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82b4102800 0x7f82b419afe0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 >> 172.21.15.138:0/3819271012 conn(0x7f82b40fe280 msgr2=0x7f82b41057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 shutdown_connections 2024-08-20T01:13:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:42.683+0000 7f82b3200640 1 -- 172.21.15.138:0/3819271012 wait complete. 2024-08-20T01:13:42.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:42 smithi142 ceph-mon[21799]: pgmap v530: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-08-20T01:13:43.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:42 smithi138 ceph-mon[18076]: pgmap v530: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-08-20T01:13:43.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 249216 5.28 MB/sec execute 115 sec latency 25.529 ms 2024-08-20T01:13:43.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:42 smithi167 ceph-mon[21965]: pgmap v530: 129 pgs: 129 active+clean; 154 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 1.7 MiB/s wr, 140 op/s 2024-08-20T01:13:44.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 251982 5.38 MB/sec execute 116 sec latency 18.093 ms 2024-08-20T01:13:45.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 253489 5.39 MB/sec execute 117 sec latency 17.268 ms 2024-08-20T01:13:45.346 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-20T01:13:45.346 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-20T01:13:45.347 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:13:45.718 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:13:45.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:45 smithi167 ceph-mon[21965]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-08-20T01:13:45.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:45 smithi142 ceph-mon[21799]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-08-20T01:13:46.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:45 smithi138 ceph-mon[18076]: pgmap v531: 129 pgs: 129 active+clean; 155 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-08-20T01:13:46.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 253572 5.34 MB/sec execute 118 sec latency 939.038 ms 2024-08-20T01:13:46.927 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:46 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:13:46.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:46 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:13:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:46 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:13:47.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 -- 172.21.15.138:0/1333594102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff068100ae0 msgr2=0x7ff068100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:47.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 --2- 172.21.15.138:0/1333594102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff068100ae0 0x7ff068100ec0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009900 tx=0x7ff05c02a410 comp rx=0 tx=0).stop 2024-08-20T01:13:47.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 -- 172.21.15.138:0/1333594102 shutdown_connections 2024-08-20T01:13:47.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 --2- 172.21.15.138:0/1333594102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff06810f1a0 0x7ff068111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 --2- 172.21.15.138:0/1333594102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06810ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 --2- 172.21.15.138:0/1333594102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff068100ae0 0x7ff068100ec0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 -- 172.21.15.138:0/1333594102 >> 172.21.15.138:0/1333594102 conn(0x7ff0680fc730 msgr2=0x7ff0680feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 -- 172.21.15.138:0/1333594102 shutdown_connections 2024-08-20T01:13:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.063+0000 7ff06f800640 1 -- 172.21.15.138:0/1333594102 wait complete. 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 Processor -- start 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 -- start start 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06819b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06810f1a0 0x7ff0681a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff068114cb0 con 0x7ff06810c870 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff068114b30 con 0x7ff06810f1a0 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06f800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff068114e30 con 0x7ff068100ae0 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.065+0000 7ff06ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39186/0 (socket says 172.21.15.138:39186) 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 -- 172.21.15.138:0/1160997296 learned_addr learned my addr 172.21.15.138:0/1160997296 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06d800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06810f1a0 0x7ff0681a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff067e00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06819b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 -- 172.21.15.138:0/1160997296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06810f1a0 msgr2=0x7ff0681a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06810f1a0 0x7ff0681a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 -- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 msgr2=0x7ff06819b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06819b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 -- 172.21.15.138:0/1160997296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0681a1310 con 0x7ff068100ae0 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff067e00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.066+0000 7ff06ce00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005ab0 tx=0x7ff05c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.067+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c03f070 con 0x7ff068100ae0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.067+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c031b60 con 0x7ff068100ae0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.067+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0681a15a0 con 0x7ff068100ae0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.067+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035030 con 0x7ff068100ae0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.067+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0681a7cc0 con 0x7ff068100ae0 2024-08-20T01:13:47.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.068+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff068100ec0 con 0x7ff068100ae0 2024-08-20T01:13:47.070 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.072+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff05c042070 con 0x7ff068100ae0 2024-08-20T01:13:47.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.074+0000 7ff065600640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 0x7ff0500b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:47.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.074+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff05c105380 con 0x7ff068100ae0 2024-08-20T01:13:47.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.074+0000 7ff067e00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 0x7ff0500b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:47.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.074+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7ff05c049030 con 0x7ff068100ae0 2024-08-20T01:13:47.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.075+0000 7ff067e00640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 0x7ff0500b6280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff06810ef30 tx=0x7ff054005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:47.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 255803 5.42 MB/sec execute 119 sec latency 943.804 ms 2024-08-20T01:13:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.388+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff06819c390 con 0x7ff068100ae0 2024-08-20T01:13:47.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.389+0000 7ff065600640 1 -- 172.21.15.138:0/1160997296 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11936 (secure 0 0 0) 0x7ff05c0cc2e0 con 0x7ff068100ae0 2024-08-20T01:13:47.387 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:13:47.388 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":34,"btime":"2024-08-20T01:13:24:677461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:13:24.223334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:13:47.388 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 34 2024-08-20T01:13:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 msgr2=0x7ff0500b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 0x7ff0500b6280 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff06810ef30 tx=0x7ff054005f90 comp rx=0 tx=0).stop 2024-08-20T01:13:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 msgr2=0x7ff06819b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005ab0 tx=0x7ff05c003c10 comp rx=0 tx=0).stop 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 shutdown_connections 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0500b3dc0 0x7ff0500b6280 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff06810f1a0 0x7ff0681a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff06810c870 0x7ff06819b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 --2- 172.21.15.138:0/1160997296 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff068100ae0 0x7ff06819b260 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 >> 172.21.15.138:0/1160997296 conn(0x7ff0680fc730 msgr2=0x7ff06810d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 shutdown_connections 2024-08-20T01:13:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:47.392+0000 7ff06f800640 1 -- 172.21.15.138:0/1160997296 wait complete. 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: pgmap v532: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1160997296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:13:47.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:47 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: pgmap v532: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1160997296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:13:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:13:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:13:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:13:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:47 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: pgmap v532: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1160997296' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:13:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:47 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:13:48.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257113 5.42 MB/sec execute 120 sec latency 62.877 ms 2024-08-20T01:13:48.297 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-20T01:13:48.298 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-20T01:13:48.664 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:13:48.961 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:48 smithi138 ceph-mon[18076]: pgmap v533: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:13:48.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:48 smithi167 ceph-mon[21965]: pgmap v533: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:13:48.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:48 smithi142 ceph-mon[21799]: pgmap v533: 129 pgs: 129 active+clean; 155 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 202 op/s 2024-08-20T01:13:49.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257113 5.37 MB/sec execute 121 sec latency 1062.922 ms 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 -- 172.21.15.138:0/3902280396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 msgr2=0x7fec4c10a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 --2- 172.21.15.138:0/3902280396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c10a960 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fec38009900 tx=0x7fec3802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 -- 172.21.15.138:0/3902280396 shutdown_connections 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 --2- 172.21.15.138:0/3902280396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c10aea0 0x7fec4c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 --2- 172.21.15.138:0/3902280396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c10a960 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 --2- 172.21.15.138:0/3902280396 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c104b10 0x7fec4c104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 -- 172.21.15.138:0/3902280396 >> 172.21.15.138:0/3902280396 conn(0x7fec4c100540 msgr2=0x7fec4c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.740+0000 7fec4b200640 1 -- 172.21.15.138:0/3902280396 shutdown_connections 2024-08-20T01:13:49.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.741+0000 7fec4b200640 1 -- 172.21.15.138:0/3902280396 wait complete. 2024-08-20T01:13:49.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.741+0000 7fec4b200640 1 Processor -- start 2024-08-20T01:13:49.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 -- start start 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c104b10 0x7fec4c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c10aea0 0x7fec4c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c114bb0 con 0x7fec4c104b10 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c114a30 con 0x7fec4c10a520 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c114d30 con 0x7fec4c10aea0 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec49400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec49400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34022/0 (socket says 172.21.15.138:34022) 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec49400640 1 -- 172.21.15.138:0/1868523962 learned_addr learned my addr 172.21.15.138:0/1868523962 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec49e00640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c104b10 0x7fec4c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.742+0000 7fec4a800640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c10aea0 0x7fec4c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 -- 172.21.15.138:0/1868523962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c10aea0 msgr2=0x7fec4c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c10aea0 0x7fec4c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 -- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c104b10 msgr2=0x7fec4c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c104b10 0x7fec4c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 -- 172.21.15.138:0/1868523962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1a0fb0 con 0x7fec4c10a520 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec49400640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec38005ab0 tx=0x7fec38003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.743+0000 7fec42a00640 1 -- 172.21.15.138:0/1868523962 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3803f070 con 0x7fec4c10a520 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.744+0000 7fec42a00640 1 -- 172.21.15.138:0/1868523962 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec38031b60 con 0x7fec4c10a520 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.744+0000 7fec42a00640 1 -- 172.21.15.138:0/1868523962 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38035030 con 0x7fec4c10a520 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.744+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c1a1240 con 0x7fec4c10a520 2024-08-20T01:13:49.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.744+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c1a7960 con 0x7fec4c10a520 2024-08-20T01:13:49.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 Processor -- start 2024-08-20T01:13:49.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 -- start start 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590007940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x56159000a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56158ff9b670 con 0x561590007e80 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56158ff6f7c0 con 0x56158fffb5d0 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec53c27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56158ff5ecc0 con 0x56158fffbef0 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec4a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.747+0000 7fec4a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34034/0 (socket says 172.21.15.138:34034) 2024-08-20T01:13:49.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 -- 172.21.15.138:0/670535947 learned_addr learned my addr 172.21.15.138:0/670535947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:49.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec42a00640 1 -- 172.21.15.138:0/1868523962 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fec38042070 con 0x7fec4c10a520 2024-08-20T01:13:49.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec49e00640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590007940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 -- 172.21.15.138:0/670535947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 msgr2=0x561590007940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec49400640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x56159000a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590007940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 -- 172.21.15.138:0/670535947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 msgr2=0x56159000a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x56159000a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 -- 172.21.15.138:0/670535947 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56158fef2110 con 0x56158fffb5d0 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec49400640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x56159000a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.748+0000 7fec4a800640 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec44011440 tx=0x7fec44011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9826505d96966b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec37400640 1 -- 172.21.15.138:0/670535947 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec44003c30 con 0x56158fffb5d0 2024-08-20T01:13:49.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec37400640 1 -- 172.21.15.138:0/670535947 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec44003dd0 con 0x56158fffb5d0 2024-08-20T01:13:49.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec37400640 1 -- 172.21.15.138:0/670535947 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec4400fae0 con 0x56158fffb5d0 2024-08-20T01:13:49.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec42a00640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 0x7fec300b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec53c27740 1 -- 172.21.15.138:0/670535947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 msgr2=0x56158fffb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.749+0000 7fec53c27740 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fec44011440 tx=0x7fec44011900 comp rx=0 tx=0).stop 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec42a00640 1 -- 172.21.15.138:0/1868523962 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fec38104a30 con 0x7fec4c10a520 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 -- 172.21.15.138:0/670535947 shutdown_connections 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x56159000a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590007940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 --2- 172.21.15.138:0/670535947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x56158fffb9b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 -- 172.21.15.138:0/670535947 >> 172.21.15.138:0/670535947 conn(0x56158fff9400 msgr2=0x56158fffa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec49e00640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 0x7fec300b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 -- 172.21.15.138:0/670535947 shutdown_connections 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec53c27740 1 -- 172.21.15.138:0/670535947 wait complete. 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.750+0000 7fec49e00640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 0x7fec300b6320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x56158fffa2e0 tx=0x7fec3c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.751+0000 7fec53c27740 1 -- start start 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.751+0000 7fec53c27740 1 Processor -- start 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.751+0000 7fec53c27740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x561590004e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.751+0000 7fec53c27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec53c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x5615900058a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec53c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56158ff04780 con 0x561590007e80 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec53c27740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56158ff62d00 con 0x56158fffb5d0 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec53c27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56158fdc17a0 con 0x56158fffbef0 2024-08-20T01:13:49.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec53c27740 10 client.0 _ll_register_callbacks cb 0x56158ff0c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x561590004e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:49594/0 (socket says 172.21.15.138:49594) 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 -- 172.21.15.138:0/588061742 learned_addr learned my addr 172.21.15.138:0/588061742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x561590004e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34042/0 (socket says 172.21.15.138:34042) 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 -- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 msgr2=0x561590004e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x561590004e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 -- 172.21.15.138:0/588061742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 msgr2=0x5615900058a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec4a800640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x5615900058a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x5615900058a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.752+0000 7fec49400640 1 -- 172.21.15.138:0/588061742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56158ff119c0 con 0x56158fffbef0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec49400640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec380cb220 tx=0x7fec380caca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86619bafcae4249d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec2be00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:13:49.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38049070 con 0x56158fffbef0 2024-08-20T01:13:49.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fec38102050 con 0x56158fffbef0 2024-08-20T01:13:49.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec44003dd0 con 0x56158fffbef0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fec38102250 con 0x56158fffbef0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec53c27740 10 client.26182 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56158fccdb90 con 0x56158fffbef0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.753+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec44003dd0 con 0x56158fffbef0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fec380ca4f0 con 0x56158fffbef0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 10 client.26182 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 10 client.26182 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 20 client.26182 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5615900177a0 0x561590019be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 4 client.26182 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56158ff71710 con 0x5615900177a0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.754+0000 7fec49e00640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5615900177a0 0x561590019be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.756 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.755+0000 7fec2be00640 10 client.26182 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.755+0000 7fec49e00640 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5615900177a0 0x561590019be0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969aae5aaff15d55 server_cookie=6f6492f4aa4752ae in_seq=0 out_seq=0 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec2be00640 1 -- 172.21.15.138:0/588061742 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56158ff71710 con 0x5615900177a0 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec2be00640 10 client.26182 handle_command_reply: tid=0 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 client.26182 shutdown 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 msgr2=0x561590005360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fec380cb220 tx=0x7fec380caca0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 shutdown_connections 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5615900177a0 0x561590019be0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x5615900058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffbef0 0x561590005360 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 --2- 172.21.15.138:0/588061742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffb5d0 0x561590004e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 >> 172.21.15.138:0/588061742 conn(0x56158fff9400 msgr2=0x56158fffef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec2be00640 0 client.26182 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:49.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.756+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 shutdown_connections 2024-08-20T01:13:49.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.757+0000 7fec53c27740 1 -- 172.21.15.138:0/588061742 wait complete. 2024-08-20T01:13:49.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.757+0000 7fec53c27740 20 client.26182 trim_cache size 0 max 16384 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 Processor -- start 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 -- start start 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159001f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159001f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590002030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b4270 con 0x561590007e80 2024-08-20T01:13:49.775 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b1b10 con 0x56158fffbef0 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec53c27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1af960 con 0x56158fffb5d0 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.776+0000 7fec49400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159001f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590002030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159001f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec4a800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590002030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42516/0 (socket says 172.21.15.138:42516) 2024-08-20T01:13:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec4a800640 1 -- 172.21.15.138:0/1504103294 learned_addr learned my addr 172.21.15.138:0/1504103294 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:49.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49400640 1 -- 172.21.15.138:0/1504103294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 msgr2=0x56159001f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49400640 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159001f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49400640 1 -- 172.21.15.138:0/1504103294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 msgr2=0x561590002030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49400640 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590002030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49400640 1 -- 172.21.15.138:0/1504103294 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561590003320 con 0x56158fffbef0 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.777+0000 7fec49e00640 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159001f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.778+0000 7fec49400640 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159001f9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec38000c00 tx=0x7fec380045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2769e254939ab97 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.778+0000 7fec36a00640 1 -- 172.21.15.138:0/1504103294 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380cb290 con 0x56158fffbef0 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.778+0000 7fec36a00640 1 -- 172.21.15.138:0/1504103294 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec380d4340 con 0x56158fffbef0 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.778+0000 7fec36a00640 1 -- 172.21.15.138:0/1504103294 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380ca6b0 con 0x56158fffbef0 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- 172.21.15.138:0/1504103294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 msgr2=0x56159001f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159001f9f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fec38000c00 tx=0x7fec380045e0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- 172.21.15.138:0/1504103294 shutdown_connections 2024-08-20T01:13:49.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590002030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159001f9f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 --2- 172.21.15.138:0/1504103294 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159001f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- 172.21.15.138:0/1504103294 >> 172.21.15.138:0/1504103294 conn(0x56158fff9400 msgr2=0x5615900133b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- 172.21.15.138:0/1504103294 shutdown_connections 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- 172.21.15.138:0/1504103294 wait complete. 2024-08-20T01:13:49.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.779+0000 7fec53c27740 1 -- start start 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 Processor -- start 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159000c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590081480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1affd0 con 0x561590007e80 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b1440 con 0x56158fffbef0 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c1b06a0 con 0x56158fffb5d0 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.780+0000 7fec53c27740 10 client.0 _ll_register_callbacks cb 0x7fec38005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:49612/0 (socket says 172.21.15.138:49612) 2024-08-20T01:13:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49e00640 1 -- 172.21.15.138:0/2694257904 learned_addr learned my addr 172.21.15.138:0/2694257904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159000c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec4a800640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590081480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 -- 172.21.15.138:0/2694257904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 msgr2=0x56159000bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 -- 172.21.15.138:0/2694257904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 msgr2=0x561590081480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590081480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49400640 1 -- 172.21.15.138:0/2694257904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615900819c0 con 0x56158fffbef0 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec49e00640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.781+0000 7fec4a800640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590081480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec49400640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159000c4e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec38005c20 tx=0x7fec380c7280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5567689b1a6cec01 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec2be00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:13:49.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec380cb290 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fec38102050 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec380d4340 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fec38102250 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec53c27740 10 client.26219 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5615900819c0 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.782+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec380d4340 con 0x56158fffbef0 2024-08-20T01:13:49.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7fec380ec1a0 con 0x56158fffbef0 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 10 client.26219 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 10 client.26219 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 20 client.26219 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561590082f00 0x561590085320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 4 client.26219 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x56158ff6d1b0 con 0x561590082f00 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.783+0000 7fec49e00640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561590082f00 0x561590085320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.784+0000 7fec49e00640 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561590082f00 0x561590085320 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc070f887723054 server_cookie=1bacdce97506c9e in_seq=0 out_seq=0 2024-08-20T01:13:49.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.784+0000 7fec2be00640 10 client.26219 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:49.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.847+0000 7fec2be00640 1 -- 172.21.15.138:0/2694257904 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56158ff6d1b0 con 0x561590082f00 2024-08-20T01:13:49.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.847+0000 7fec2be00640 10 client.26219 handle_command_reply: tid=0 2024-08-20T01:13:49.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.847+0000 7fec53c27740 1 client.26219 shutdown 2024-08-20T01:13:49.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 msgr2=0x56159000c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159000c4e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec38005c20 tx=0x7fec380c7280 comp rx=0 tx=0).stop 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 shutdown_connections 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561590082f00 0x561590085320 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561590007e80 0x561590081480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56158fffbef0 0x56159000c4e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 --2- 172.21.15.138:0/2694257904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56158fffb5d0 0x56159000bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec2be00640 0 client.26219 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 >> 172.21.15.138:0/2694257904 conn(0x56158fff9400 msgr2=0x56158fffe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 shutdown_connections 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 1 -- 172.21.15.138:0/2694257904 wait complete. 2024-08-20T01:13:49.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.848+0000 7fec53c27740 20 client.26219 trim_cache size 0 max 16384 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 34, 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 26, 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "948b0677": { 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 26, 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:13:49.848 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 150.0, 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:13:49.849 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 msgr2=0x7fec300b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 0x7fec300b6320 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x56158fffa2e0 tx=0x7fec3c005f90 comp rx=0 tx=0).stop 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 msgr2=0x7fec4c19b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fec38005ab0 tx=0x7fec38003c10 comp rx=0 tx=0).stop 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 shutdown_connections 2024-08-20T01:13:49.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fec300b3e60 0x7fec300b6320 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fec4c10aea0 0x7fec4c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fec4c10a520 0x7fec4c19b440 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 --2- 172.21.15.138:0/1868523962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec4c104b10 0x7fec4c19af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.849+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 >> 172.21.15.138:0/1868523962 conn(0x7fec4c100540 msgr2=0x7fec4c103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.850+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 shutdown_connections 2024-08-20T01:13:49.851 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:13:49.850+0000 7fec4b200640 1 -- 172.21.15.138:0/1868523962 wait complete. 2024-08-20T01:13:50.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.33 MB/sec execute 122 sec latency 1195.447 ms 2024-08-20T01:13:50.942 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '948b0677', quiesce took 5.6 seconds. Will release after: 24.4 2024-08-20T01:13:51.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.28 MB/sec execute 123 sec latency 1338.817 ms 2024-08-20T01:13:51.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:51 smithi167 ceph-mon[21965]: pgmap v534: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-08-20T01:13:51.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:51 smithi142 ceph-mon[21799]: pgmap v534: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-08-20T01:13:52.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:51 smithi138 ceph-mon[18076]: pgmap v534: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 249 op/s 2024-08-20T01:13:52.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.24 MB/sec execute 124 sec latency 2338.901 ms 2024-08-20T01:13:53.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.20 MB/sec execute 125 sec latency 3339.021 ms 2024-08-20T01:13:53.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:53 smithi167 ceph-mon[21965]: pgmap v535: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-08-20T01:13:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:53 smithi142 ceph-mon[21799]: pgmap v535: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-08-20T01:13:54.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:53 smithi138 ceph-mon[18076]: pgmap v535: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-08-20T01:13:54.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.16 MB/sec execute 126 sec latency 4339.119 ms 2024-08-20T01:13:55.150 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.12 MB/sec execute 127 sec latency 5339.180 ms 2024-08-20T01:13:55.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 -- 172.21.15.138:0/2707454519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac1029e0 msgr2=0x7f3fac102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/2707454519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac1029e0 0x7f3fac102dc0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009900 tx=0x7f3f9c02a410 comp rx=0 tx=0).stop 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 -- 172.21.15.138:0/2707454519 shutdown_connections 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/2707454519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac108d60 0x7f3fac10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/2707454519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 0x7f3fac108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/2707454519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac1029e0 0x7f3fac102dc0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 -- 172.21.15.138:0/2707454519 >> 172.21.15.138:0/2707454519 conn(0x7f3fac0fe420 msgr2=0x7f3fac100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 -- 172.21.15.138:0/2707454519 shutdown_connections 2024-08-20T01:13:55.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.799+0000 7f3fb1a00640 1 -- 172.21.15.138:0/2707454519 wait complete. 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 Processor -- start 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 -- start start 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac1029e0 0x7f3fac19b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 0x7f3fac19b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.800+0000 7f3fb1a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112a70 con 0x7f3fac108d60 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fb1a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1128f0 con 0x7f3fac1029e0 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fb1a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac112bf0 con 0x7f3fac1083e0 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42560/0 (socket says 172.21.15.138:42560) 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 -- 172.21.15.138:0/3963686102 learned_addr learned my addr 172.21.15.138:0/3963686102 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac1029e0 0x7f3fac19b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3faa000640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 0x7f3fac19b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 -- 172.21.15.138:0/3963686102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 msgr2=0x7f3fac19b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 0x7f3fac19b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 -- 172.21.15.138:0/3963686102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac1029e0 msgr2=0x7f3fac19b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac1029e0 0x7f3fac19b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 -- 172.21.15.138:0/3963686102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fac1a11d0 con 0x7f3fac108d60 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.801+0000 7f3fab400640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400eb10 tx=0x7f3fa400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.802+0000 7f3fa3400640 1 -- 172.21.15.138:0/3963686102 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4004b20 con 0x7f3fac108d60 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.802+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fac1a14c0 con 0x7f3fac108d60 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.802+0000 7f3fa3400640 1 -- 172.21.15.138:0/3963686102 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fa4004cc0 con 0x7f3fac108d60 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.802+0000 7f3fa3400640 1 -- 172.21.15.138:0/3963686102 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4010670 con 0x7f3fac108d60 2024-08-20T01:13:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.802+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fac1a7b80 con 0x7f3fac108d60 2024-08-20T01:13:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fb49cd740 1 Processor -- start 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fa3400640 1 -- 172.21.15.138:0/3963686102 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3fa4005020 con 0x7f3fac108d60 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fb49cd740 1 -- start start 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcf29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.805+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bd01270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56256bc68210 con 0x56256bcf2ef0 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56256bc57710 con 0x56256bcf25d0 2024-08-20T01:13:55.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56256bc5b750 con 0x56256bcfee80 2024-08-20T01:13:55.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcf29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcf29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34086/0 (socket says 172.21.15.138:34086) 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42572/0 (socket says 172.21.15.138:42572) 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faa000640 1 -- 172.21.15.138:0/2016553626 learned_addr learned my addr 172.21.15.138:0/2016553626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faa000640 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bd01270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 -- 172.21.15.138:0/2016553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 msgr2=0x56256bd01270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bd01270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 -- 172.21.15.138:0/2016553626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 msgr2=0x56256bcf29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcf29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.806+0000 7f3faaa00640 1 -- 172.21.15.138:0/2016553626 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256bc23de0 con 0x56256bcf2ef0 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3fa3400640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 0x7f3f900b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3faaa00640 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c02aa00 tx=0x7f3f9c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfee79ebc35c1cb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3fa3400640 1 -- 172.21.15.138:0/3963686102 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3fa40e4980 con 0x7f3fac108d60 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3faa000640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 0x7f3f900b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3f97e00640 1 -- 172.21.15.138:0/2016553626 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c035040 con 0x56256bcf2ef0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3f97e00640 1 -- 172.21.15.138:0/2016553626 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f9c0054f0 con 0x56256bcf2ef0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.807+0000 7f3f97e00640 1 -- 172.21.15.138:0/2016553626 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c004e40 con 0x56256bcf2ef0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- 172.21.15.138:0/2016553626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 msgr2=0x56256bcfe940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3faa000640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 0x7f3f900b6200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3f980118f0 tx=0x7f3f980097d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c02aa00 tx=0x7f3f9c005c00 comp rx=0 tx=0).stop 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- 172.21.15.138:0/2016553626 shutdown_connections 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bd01270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfe940 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/2016553626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcf29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- 172.21.15.138:0/2016553626 >> 172.21.15.138:0/2016553626 conn(0x56256bcf03b0 msgr2=0x56256bcf15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- 172.21.15.138:0/2016553626 shutdown_connections 2024-08-20T01:13:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- 172.21.15.138:0/2016553626 wait complete. 2024-08-20T01:13:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.808+0000 7f3fb49cd740 1 -- start start 2024-08-20T01:13:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 Processor -- start 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcfbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bcfc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56256bb90ab0 con 0x56256bcf2ef0 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56256bbfe140 con 0x56256bcf25d0 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56256bd01b00 con 0x56256bcfee80 2024-08-20T01:13:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3fb49cd740 10 client.0 _ll_register_callbacks cb 0x56256bc115d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3faa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3faa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42578/0 (socket says 172.21.15.138:42578) 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.809+0000 7f3faa000640 1 -- 172.21.15.138:0/3739282951 learned_addr learned my addr 172.21.15.138:0/3739282951 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3fab400640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bcfc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcfbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 -- 172.21.15.138:0/3739282951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 msgr2=0x56256bcfc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bcfc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 -- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 msgr2=0x56256bcfbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcfbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 -- 172.21.15.138:0/3739282951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256bbff170 con 0x56256bcf2ef0 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcfbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3faa000640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f98004560 tx=0x7f3f98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace78965763f6663 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.810+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98003b70 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3f9801a070 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56256bc2ea30 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f98003d10 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 10 client.16179 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56256b9ba790 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56256bc2ea30 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f3f98028070 con 0x56256bcf2ef0 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 10 client.16179 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 10 client.16179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 20 client.16179 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd0ed70 0x56256bd111e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 4 client.16179 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.811+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56256bc6a160 con 0x56256bd0ed70 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.812+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd0ed70 0x56256bd111e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.812+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd0ed70 0x56256bd111e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa3200d2c8e2b01d server_cookie=f9e71e330146a637 in_seq=0 out_seq=0 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.812+0000 7f3f94c00640 10 client.16179 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3f94c00640 1 -- 172.21.15.138:0/3739282951 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56256bc6a160 con 0x56256bd0ed70 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3f94c00640 10 client.16179 handle_command_reply: tid=0 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 client.16179 shutdown 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 msgr2=0x56256bcfc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3f98004560 tx=0x7f3f98004730 comp rx=0 tx=0).stop 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 shutdown_connections 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd0ed70 0x56256bd111e0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcfee80 0x56256bcfc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bcfc2e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3739282951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bcfbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 >> 172.21.15.138:0/3739282951 conn(0x56256bcf03b0 msgr2=0x56256bcf5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3f94c00640 0 client.16179 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 shutdown_connections 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.813+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3739282951 wait complete. 2024-08-20T01:13:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.814+0000 7f3fb49cd740 20 client.16179 trim_cache size 0 max 16384 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.828+0000 7f3fb49cd740 1 Processor -- start 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 -- start start 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 0x56256bcf17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 0x56256bd17080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b4450 con 0x56256bcf2ef0 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b1cf0 con 0x56256bcfee80 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1afb40 con 0x56256bcf25d0 2024-08-20T01:13:55.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42592/0 (socket says 172.21.15.138:42592) 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 -- 172.21.15.138:0/4179708316 learned_addr learned my addr 172.21.15.138:0/4179708316 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faaa00640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 0x56256bcf17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fab400640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 0x56256bd17080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 -- 172.21.15.138:0/4179708316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 msgr2=0x56256bcf17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 0x56256bcf17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 -- 172.21.15.138:0/4179708316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 msgr2=0x56256bd17080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 0x56256bd17080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faa000640 1 -- 172.21.15.138:0/4179708316 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56256bcf2d20 con 0x56256bcf2ef0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3faaa00640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 0x56256bcf17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.829+0000 7f3fab400640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 0x56256bd17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3faa000640 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f980295b0 tx=0x7f3f98003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283c19dce29c3d74 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3f97400640 1 -- 172.21.15.138:0/4179708316 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98004350 con 0x56256bcf2ef0 2024-08-20T01:13:55.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3f97400640 1 -- 172.21.15.138:0/4179708316 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f98011670 con 0x56256bcf2ef0 2024-08-20T01:13:55.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3f97400640 1 -- 172.21.15.138:0/4179708316 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98020680 con 0x56256bcf2ef0 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 -- 172.21.15.138:0/4179708316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 msgr2=0x56256bd16b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3f980295b0 tx=0x7f3f98003f30 comp rx=0 tx=0).stop 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 -- 172.21.15.138:0/4179708316 shutdown_connections 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcfee80 0x56256bd17080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcf2ef0 0x56256bd16b40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/4179708316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf25d0 0x56256bcf17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.830+0000 7f3fb49cd740 1 -- 172.21.15.138:0/4179708316 >> 172.21.15.138:0/4179708316 conn(0x56256bcf03b0 msgr2=0x56256bcf10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.831+0000 7f3fb49cd740 1 -- 172.21.15.138:0/4179708316 shutdown_connections 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.831+0000 7f3fb49cd740 1 -- 172.21.15.138:0/4179708316 wait complete. 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.831+0000 7f3fb49cd740 1 -- start start 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.831+0000 7f3fb49cd740 1 Processor -- start 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bd06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 0x56256bd072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b01b0 con 0x56256bcfee80 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b1620 con 0x56256bcf25d0 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3fac1b0880 con 0x56256bcf2ef0 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fb49cd740 10 client.0 _ll_register_callbacks cb 0x7f3f980099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bd06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 0x56256bd072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42598/0 (socket says 172.21.15.138:42598) 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fab400640 1 -- 172.21.15.138:0/3276252140 learned_addr learned my addr 172.21.15.138:0/3276252140 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 -- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 msgr2=0x56256bd06d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bd06d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 -- 172.21.15.138:0/3276252140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 msgr2=0x56256bcf83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faa000640 1 -- 172.21.15.138:0/3276252140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56256ba33620 con 0x56256bcf2ef0 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3fab400640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.832+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bd06d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3faa000640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 0x56256bd072d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f98029440 tx=0x7f3f98029870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88fe8eaa5c98d4da server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3f94c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f98024040 con 0x56256bcf2ef0 2024-08-20T01:13:55.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f3f98009040 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56256bc65c00 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3f98029c70 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3fb49cd740 10 client.26206 fetch_fsmap learned FSMap version 34 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56256bd79150 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.833+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56256bc65c00 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f3f98028070 con 0x56256bcf2ef0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 10 client.26206 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 10 client.26206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 20 client.26206 populate_metadata read hostname 'smithi138' 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd79680 0x56256bd7baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 4 client.26206 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56256bc63340 con 0x56256bd79680 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd79680 0x56256bd7baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:13:55.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3faaa00640 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd79680 0x56256bd7baa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e20f029b86b00d77 server_cookie=246e0472d4148943 in_seq=0 out_seq=0 2024-08-20T01:13:55.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.834+0000 7f3f94c00640 10 client.26206 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:55.882 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.884+0000 7f3f94c00640 1 -- 172.21.15.138:0/3276252140 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56256bc63340 con 0x56256bd79680 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.884+0000 7f3f94c00640 10 client.26206 handle_command_reply: tid=0 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.884+0000 7f3fb49cd740 1 client.26206 shutdown 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 msgr2=0x56256bd072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 0x56256bd072d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3f98029440 tx=0x7f3f98029870 comp rx=0 tx=0).stop 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 shutdown_connections 2024-08-20T01:13:55.883 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56256bd79680 0x56256bd7baa0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56256bcfee80 0x56256bcf83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56256bcf2ef0 0x56256bd072d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 --2- 172.21.15.138:0/3276252140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56256bcf25d0 0x56256bd06d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.885+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 >> 172.21.15.138:0/3276252140 conn(0x56256bcf03b0 msgr2=0x56256bcfbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.886+0000 7f3f94c00640 0 client.26206 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.886+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 shutdown_connections 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.886+0000 7f3fb49cd740 1 -- 172.21.15.138:0/3276252140 wait complete. 2024-08-20T01:13:55.884 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.886+0000 7f3fb49cd740 20 client.26206 trim_cache size 0 max 16384 2024-08-20T01:13:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:13:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:13:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:13:55.885 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 msgr2=0x7f3f900b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 0x7f3f900b6200 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3f980118f0 tx=0x7f3f980097d0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 msgr2=0x7f3fac1a0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3fa400eb10 tx=0x7f3fa400c2d0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 shutdown_connections 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3f900b3d40 0x7f3f900b6200 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fac108d60 0x7f3fac1a0a00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3fac1083e0 0x7f3fac19b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.886 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 --2- 172.21.15.138:0/3963686102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fac1029e0 0x7f3fac19b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:13:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 >> 172.21.15.138:0/3963686102 conn(0x7f3fac0fe420 msgr2=0x7f3fac104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:13:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 shutdown_connections 2024-08-20T01:13:55.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:13:55.888+0000 7f3fb1a00640 1 -- 172.21.15.138:0/3963686102 wait complete. 2024-08-20T01:13:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:55 smithi167 ceph-mon[21965]: pgmap v536: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-08-20T01:13:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:55 smithi142 ceph-mon[21799]: pgmap v536: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-08-20T01:13:56.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:55 smithi138 ceph-mon[18076]: pgmap v536: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 180 op/s 2024-08-20T01:13:56.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.08 MB/sec execute 128 sec latency 6339.251 ms 2024-08-20T01:13:57.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.04 MB/sec execute 129 sec latency 7339.438 ms 2024-08-20T01:13:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:57 smithi167 ceph-mon[21965]: pgmap v537: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:13:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:57 smithi142 ceph-mon[21799]: pgmap v537: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:13:58.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:57 smithi138 ceph-mon[18076]: pgmap v537: 129 pgs: 129 active+clean; 163 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:13:58.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 5.00 MB/sec execute 130 sec latency 8339.558 ms 2024-08-20T01:13:58.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:13:58 smithi142 ceph-mon[21799]: pgmap v538: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-20T01:13:59.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:13:58 smithi138 ceph-mon[18076]: pgmap v538: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-20T01:13:59.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.96 MB/sec execute 131 sec latency 9339.695 ms 2024-08-20T01:13:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:13:58 smithi167 ceph-mon[21965]: pgmap v538: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.1 MiB/s wr, 88 op/s 2024-08-20T01:14:00.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.92 MB/sec execute 132 sec latency 10339.760 ms 2024-08-20T01:14:01.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.89 MB/sec execute 133 sec latency 11339.830 ms 2024-08-20T01:14:01.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:01 smithi167 ceph-mon[21965]: pgmap v539: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:14:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:01 smithi142 ceph-mon[21799]: pgmap v539: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:14:02.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:01 smithi138 ceph-mon[18076]: pgmap v539: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.3 MiB/s wr, 114 op/s 2024-08-20T01:14:02.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.85 MB/sec execute 134 sec latency 12339.956 ms 2024-08-20T01:14:03.151 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.81 MB/sec execute 135 sec latency 13340.092 ms 2024-08-20T01:14:03.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:03 smithi167 ceph-mon[21965]: pgmap v540: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 809 KiB/s wr, 65 op/s 2024-08-20T01:14:03.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:03 smithi142 ceph-mon[21799]: pgmap v540: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 809 KiB/s wr, 65 op/s 2024-08-20T01:14:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:03 smithi138 ceph-mon[18076]: pgmap v540: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 809 KiB/s wr, 65 op/s 2024-08-20T01:14:04.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.78 MB/sec execute 136 sec latency 14340.245 ms 2024-08-20T01:14:05.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.74 MB/sec execute 137 sec latency 15340.382 ms 2024-08-20T01:14:05.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:05 smithi167 ceph-mon[21965]: pgmap v541: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 849 KiB/s wr, 83 op/s 2024-08-20T01:14:05.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:05 smithi142 ceph-mon[21799]: pgmap v541: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 849 KiB/s wr, 83 op/s 2024-08-20T01:14:06.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:05 smithi138 ceph-mon[18076]: pgmap v541: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 849 KiB/s wr, 83 op/s 2024-08-20T01:14:06.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.71 MB/sec execute 138 sec latency 16340.493 ms 2024-08-20T01:14:07.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.68 MB/sec execute 139 sec latency 17340.608 ms 2024-08-20T01:14:07.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:07 smithi167 ceph-mon[21965]: pgmap v542: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 665 KiB/s wr, 74 op/s 2024-08-20T01:14:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:07 smithi142 ceph-mon[21799]: pgmap v542: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 665 KiB/s wr, 74 op/s 2024-08-20T01:14:08.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:07 smithi138 ceph-mon[18076]: pgmap v542: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 665 KiB/s wr, 74 op/s 2024-08-20T01:14:08.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.64 MB/sec execute 140 sec latency 18340.731 ms 2024-08-20T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.011+0000 7f9f30200640 1 -- 172.21.15.138:0/4125577033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f28108d60 msgr2=0x7f9f2810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.011+0000 7f9f30200640 1 --2- 172.21.15.138:0/4125577033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f28108d60 0x7f9f2810f560 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009ca0 tx=0x7f9f2002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 -- 172.21.15.138:0/4125577033 shutdown_connections 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 --2- 172.21.15.138:0/4125577033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f28108d60 0x7f9f2810f560 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 --2- 172.21.15.138:0/4125577033 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f281083e0 0x7f9f28108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 --2- 172.21.15.138:0/4125577033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 0x7f9f28102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 -- 172.21.15.138:0/4125577033 >> 172.21.15.138:0/4125577033 conn(0x7f9f280fe420 msgr2=0x7f9f28100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 -- 172.21.15.138:0/4125577033 shutdown_connections 2024-08-20T01:14:09.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.012+0000 7f9f30200640 1 -- 172.21.15.138:0/4125577033 wait complete. 2024-08-20T01:14:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 Processor -- start 2024-08-20T01:14:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 -- start start 2024-08-20T01:14:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 0x7f9f28198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 0x7f9f2819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28112a30 con 0x7f9f281083e0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281128b0 con 0x7f9f281029e0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f30200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28112bb0 con 0x7f9f28108d60 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f2ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f2ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35890/0 (socket says 172.21.15.138:35890) 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.013+0000 7f9f2ce00640 1 -- 172.21.15.138:0/2514078602 learned_addr learned my addr 172.21.15.138:0/2514078602 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2d800640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 0x7f9f28198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2e200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 0x7f9f2819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 -- 172.21.15.138:0/2514078602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 msgr2=0x7f9f2819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 0x7f9f2819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 -- 172.21.15.138:0/2514078602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 msgr2=0x7f9f28198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 0x7f9f28198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 -- 172.21.15.138:0/2514078602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f2819ef70 con 0x7f9f281083e0 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2e200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 0x7f9f2819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:09.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9f0 tx=0x7f9f1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f26000640 1 -- 172.21.15.138:0/2514078602 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400cd80 con 0x7f9f281083e0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f26000640 1 -- 172.21.15.138:0/2514078602 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f140054d0 con 0x7f9f281083e0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f2819f260 con 0x7f9f281083e0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.014+0000 7f9f26000640 1 -- 172.21.15.138:0/2514078602 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14010690 con 0x7f9f281083e0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.015+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a5920 con 0x7f9f281083e0 2024-08-20T01:14:09.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 Processor -- start 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 -- start start 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f26000640 1 -- 172.21.15.138:0/2514078602 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9f14004e90 con 0x7f9f281083e0 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da0c02e210 con 0x55da0c0b9250 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da0c01d710 con 0x55da0c0b8930 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55da0c021750 con 0x55da0c0c51e0 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f2e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f2e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:41272/0 (socket says 172.21.15.138:41272) 2024-08-20T01:14:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 -- 172.21.15.138:0/1362750636 learned_addr learned my addr 172.21.15.138:0/1362750636 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.018+0000 7f9f2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 -- 172.21.15.138:0/1362750636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 msgr2=0x55da0c0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 -- 172.21.15.138:0/1362750636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 msgr2=0x55da0c0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 -- 172.21.15.138:0/1362750636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0bfe9de0 con 0x55da0c0b8930 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f2e200640 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f20035000 tx=0x7f9f20009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9193d01f383c66d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.019+0000 7f9f1aa00640 1 -- 172.21.15.138:0/1362750636 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20031cf0 con 0x55da0c0b8930 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f1aa00640 1 -- 172.21.15.138:0/1362750636 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20036040 con 0x55da0c0b8930 2024-08-20T01:14:09.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f1aa00640 1 -- 172.21.15.138:0/1362750636 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f200042a0 con 0x55da0c0b8930 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f26000640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 0x7f9f100b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f26000640 1 -- 172.21.15.138:0/2514078602 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9f140e47f0 con 0x7f9f281083e0 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f2d800640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 0x7f9f100b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 -- 172.21.15.138:0/1362750636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 msgr2=0x55da0c0b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9f20035000 tx=0x7f9f20009800 comp rx=0 tx=0).stop 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 -- 172.21.15.138:0/1362750636 shutdown_connections 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/1362750636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0b8d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.020+0000 7f9f32ffa740 1 -- 172.21.15.138:0/1362750636 >> 172.21.15.138:0/1362750636 conn(0x55da0c0b6760 msgr2=0x55da0c0b7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.021+0000 7f9f32ffa740 1 -- 172.21.15.138:0/1362750636 shutdown_connections 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.021+0000 7f9f32ffa740 1 -- 172.21.15.138:0/1362750636 wait complete. 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.021+0000 7f9f2d800640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 0x7f9f100b6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55da0c0bbab0 tx=0x7f9f1c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.021+0000 7f9f32ffa740 1 -- start start 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.021+0000 7f9f32ffa740 1 Processor -- start 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55da0bf56ab0 con 0x55da0c0b9250 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55da0bfc4140 con 0x55da0c0b8930 2024-08-20T01:14:09.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55da0c0c7e60 con 0x55da0c0c51e0 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f32ffa740 10 client.0 _ll_register_callbacks cb 0x55da0bfd75d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f2ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f2d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f2ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35916/0 (socket says 172.21.15.138:35916) 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f2ce00640 1 -- 172.21.15.138:0/160219259 learned_addr learned my addr 172.21.15.138:0/160219259 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.022+0000 7f9f2e200640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 -- 172.21.15.138:0/160219259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 msgr2=0x55da0c0c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 -- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 msgr2=0x55da0c0c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 -- 172.21.15.138:0/160219259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0bfc5170 con 0x55da0c0b9250 2024-08-20T01:14:09.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2d800640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f140048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8134aecc4eaf3b0b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f0f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14005180 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9f140e4df0 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f20036040 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9f140aab60 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f32ffa740 10 client.16197 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0bd80790 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.023+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20036040 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f9f140e43e0 con 0x55da0c0b9250 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 10 client.16197 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 20 client.16197 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:09.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c0d4db0 0x55da0c0d71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 4 client.16197 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da0c030160 con 0x55da0c0d4db0 2024-08-20T01:14:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.024+0000 7f9f2d800640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c0d4db0 0x55da0c0d71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.025+0000 7f9f2d800640 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c0d4db0 0x55da0c0d71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7434cf0e9dd466 server_cookie=e656a9b2ede76526 in_seq=0 out_seq=0 2024-08-20T01:14:09.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.025+0000 7f9f0f400640 10 client.16197 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f0f400640 1 -- 172.21.15.138:0/160219259 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da0c030160 con 0x55da0c0d4db0 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f0f400640 10 client.16197 handle_command_reply: tid=0 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 client.16197 shutdown 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 msgr2=0x55da0c0c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9c0 tx=0x7f9f140048f0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 shutdown_connections 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c0d4db0 0x55da0c0d71a0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b9250 0x55da0c0c1da0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f0f400640 0 client.16197 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/160219259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b8930 0x55da0c0c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 >> 172.21.15.138:0/160219259 conn(0x55da0c0b6760 msgr2=0x55da0c0bc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 shutdown_connections 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 1 -- 172.21.15.138:0/160219259 wait complete. 2024-08-20T01:14:09.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.026+0000 7f9f32ffa740 20 client.16197 trim_cache size 0 max 16384 2024-08-20T01:14:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.041+0000 7f9f32ffa740 1 Processor -- start 2024-08-20T01:14:09.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.041+0000 7f9f32ffa740 1 -- start start 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0b8930 0x55da0c0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281b2350 con 0x55da0c0c51e0 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281afbf0 con 0x55da0c0b9250 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281ada40 con 0x55da0c0b8930 2024-08-20T01:14:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35920/0 (socket says 172.21.15.138:35920) 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 -- 172.21.15.138:0/38862117 learned_addr learned my addr 172.21.15.138:0/38862117 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2d800640 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0b8930 0x55da0c0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 -- 172.21.15.138:0/38862117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0b8930 msgr2=0x55da0c0b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0b8930 0x55da0c0b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 -- 172.21.15.138:0/38862117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 msgr2=0x55da0c0b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.042+0000 7f9f2e200640 1 -- 172.21.15.138:0/38862117 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0bdf9620 con 0x55da0c0c51e0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.043+0000 7f9f2e200640 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009870 tx=0x7f9f20000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78932bc629f695f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.043+0000 7f9f1a000640 1 -- 172.21.15.138:0/38862117 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2004a040 con 0x55da0c0c51e0 2024-08-20T01:14:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.043+0000 7f9f1a000640 1 -- 172.21.15.138:0/38862117 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20036050 con 0x55da0c0c51e0 2024-08-20T01:14:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.043+0000 7f9f1a000640 1 -- 172.21.15.138:0/38862117 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f20004040 con 0x55da0c0c51e0 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- 172.21.15.138:0/38862117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 msgr2=0x55da0c0dca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9f20009870 tx=0x7f9f20000f40 comp rx=0 tx=0).stop 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- 172.21.15.138:0/38862117 shutdown_connections 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0c51e0 0x55da0c0dca20 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/38862117 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0b8930 0x55da0c0b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- 172.21.15.138:0/38862117 >> 172.21.15.138:0/38862117 conn(0x55da0c0b6760 msgr2=0x55da0c0c0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- 172.21.15.138:0/38862117 shutdown_connections 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- 172.21.15.138:0/38862117 wait complete. 2024-08-20T01:14:09.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 -- start start 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.044+0000 7f9f32ffa740 1 Processor -- start 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281ae0b0 con 0x55da0c0b8930 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281af520 con 0x55da0c0b9250 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281ae780 con 0x55da0c0c51e0 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f32ffa740 10 client.0 _ll_register_callbacks cb 0x7f9f140ea220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35922/0 (socket says 172.21.15.138:35922) 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 -- 172.21.15.138:0/3466246682 learned_addr learned my addr 172.21.15.138:0/3466246682 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2e200640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 -- 172.21.15.138:0/3466246682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 msgr2=0x55da0c0ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 -- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 msgr2=0x55da0c0c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0c8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2d800640 1 -- 172.21.15.138:0/3466246682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0c0cd660 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2e200640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.045+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f2d800640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55da0c0d7990 tx=0x7f9f1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24ae5299cede2b1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f0f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c003d00 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f9f1c005020 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f20036050 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f9f1c03b050 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f32ffa740 10 client.16203 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55da0c13f6d0 con 0x55da0c0b8930 2024-08-20T01:14:09.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.046+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f20036050 con 0x55da0c0b8930 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f9f1c02b380 con 0x55da0c0b8930 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 10 client.16203 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 10 client.16203 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 20 client.16203 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c13fee0 0x55da0c142300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 4 client.16203 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55da0c02bc00 con 0x55da0c13fee0 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c13fee0 0x55da0c142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f2ce00640 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c13fee0 0x55da0c142300 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38bed52a8e3f1b4a server_cookie=2a6cd9a076d4b8b6 in_seq=0 out_seq=0 2024-08-20T01:14:09.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.047+0000 7f9f0f400640 10 client.16203 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.062+0000 7f9f0f400640 1 -- 172.21.15.138:0/3466246682 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55da0c02bc00 con 0x55da0c13fee0 2024-08-20T01:14:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.062+0000 7f9f0f400640 10 client.16203 handle_command_reply: tid=0 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.062+0000 7f9f32ffa740 1 client.16203 shutdown 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 msgr2=0x55da0c0c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55da0c0d7990 tx=0x7f9f1c0046c0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 shutdown_connections 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55da0c13fee0 0x55da0c142300 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55da0c0c51e0 0x55da0c0ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55da0c0b9250 0x55da0c0c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 --2- 172.21.15.138:0/3466246682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55da0c0b8930 0x55da0c0c8a40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f0f400640 0 client.16203 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 >> 172.21.15.138:0/3466246682 conn(0x55da0c0b6760 msgr2=0x55da0c0c1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 shutdown_connections 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 1 -- 172.21.15.138:0/3466246682 wait complete. 2024-08-20T01:14:09.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.063+0000 7f9f32ffa740 20 client.16203 trim_cache size 0 max 16384 2024-08-20T01:14:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:14:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:14:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:14:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 msgr2=0x7f9f100b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 0x7f9f100b6280 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55da0c0bbab0 tx=0x7f9f1c005f90 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 msgr2=0x7f9f28199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400e9f0 tx=0x7f9f1400edf0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 shutdown_connections 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9f100b3dc0 0x7f9f100b6280 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9f28108d60 0x7f9f2819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f281083e0 0x7f9f28199410 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 --2- 172.21.15.138:0/2514078602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f281029e0 0x7f9f28198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 >> 172.21.15.138:0/2514078602 conn(0x7f9f280fe420 msgr2=0x7f9f28100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 shutdown_connections 2024-08-20T01:14:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:09.064+0000 7f9f30200640 1 -- 172.21.15.138:0/2514078602 wait complete. 2024-08-20T01:14:09.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.61 MB/sec execute 141 sec latency 19340.799 ms 2024-08-20T01:14:09.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:09 smithi167 ceph-mon[21965]: pgmap v543: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 385 KiB/s wr, 65 op/s 2024-08-20T01:14:09.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:09 smithi142 ceph-mon[21799]: pgmap v543: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 385 KiB/s wr, 65 op/s 2024-08-20T01:14:10.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:09 smithi138 ceph-mon[18076]: pgmap v543: 129 pgs: 129 active+clean; 158 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 385 KiB/s wr, 65 op/s 2024-08-20T01:14:10.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.58 MB/sec execute 142 sec latency 20340.841 ms 2024-08-20T01:14:10.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:10 smithi138 ceph-mon[18076]: pgmap v544: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 473 KiB/s wr, 70 op/s 2024-08-20T01:14:10.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:10 smithi167 ceph-mon[21965]: pgmap v544: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 473 KiB/s wr, 70 op/s 2024-08-20T01:14:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:10 smithi142 ceph-mon[21799]: pgmap v544: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 473 KiB/s wr, 70 op/s 2024-08-20T01:14:11.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.55 MB/sec execute 143 sec latency 21340.967 ms 2024-08-20T01:14:12.152 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.51 MB/sec execute 144 sec latency 22341.106 ms 2024-08-20T01:14:13.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.48 MB/sec execute 145 sec latency 23341.224 ms 2024-08-20T01:14:13.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:13 smithi167 ceph-mon[21965]: pgmap v545: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 406 KiB/s wr, 43 op/s 2024-08-20T01:14:13.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:13 smithi142 ceph-mon[21799]: pgmap v545: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 406 KiB/s wr, 43 op/s 2024-08-20T01:14:14.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:13 smithi138 ceph-mon[18076]: pgmap v545: 129 pgs: 129 active+clean; 160 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 406 KiB/s wr, 43 op/s 2024-08-20T01:14:14.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.45 MB/sec execute 146 sec latency 24341.367 ms 2024-08-20T01:14:15.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.42 MB/sec execute 147 sec latency 25341.472 ms 2024-08-20T01:14:15.343 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:14:15.721 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:15.960 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:15 smithi138 ceph-mon[18076]: pgmap v546: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 689 KiB/s wr, 49 op/s 2024-08-20T01:14:15.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:15 smithi167 ceph-mon[21965]: pgmap v546: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 689 KiB/s wr, 49 op/s 2024-08-20T01:14:15.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:15 smithi142 ceph-mon[21799]: pgmap v546: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 689 KiB/s wr, 49 op/s 2024-08-20T01:14:16.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.39 MB/sec execute 148 sec latency 26341.511 ms 2024-08-20T01:14:16.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 -- 172.21.15.138:0/3566287315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10a6b0 msgr2=0x7fdc1c10aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:16.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3566287315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c10aaf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc04009900 tx=0x7fdc0402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 -- 172.21.15.138:0/3566287315 shutdown_connections 2024-08-20T01:14:16.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3566287315 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c10b030 0x7fdc1c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3566287315 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c10aaf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3566287315 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 -- 172.21.15.138:0/3566287315 >> 172.21.15.138:0/3566287315 conn(0x7fdc1c1006f0 msgr2=0x7fdc1c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 -- 172.21.15.138:0/3566287315 shutdown_connections 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.309+0000 7fdc21c00640 1 -- 172.21.15.138:0/3566287315 wait complete. 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 Processor -- start 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 -- start start 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10b030 0x7fdc1c1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c114d40 con 0x7fdc1c104ca0 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c114bc0 con 0x7fdc1c10b030 2024-08-20T01:14:16.313 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.310+0000 7fdc21c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc1c114ec0 con 0x7fdc1c10a6b0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:35932/0 (socket says 172.21.15.138:35932) 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 -- 172.21.15.138:0/3986989647 learned_addr learned my addr 172.21.15.138:0/3986989647 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc13e00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1b400640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10b030 0x7fdc1c1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 -- 172.21.15.138:0/3986989647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c10a6b0 msgr2=0x7fdc1c19b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c19b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 -- 172.21.15.138:0/3986989647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10b030 msgr2=0x7fdc1c1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10b030 0x7fdc1c1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 -- 172.21.15.138:0/3986989647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc1c1a1250 con 0x7fdc1c104ca0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.311+0000 7fdc1aa00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9f0 tx=0x7fdc0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.312+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c00ccc0 con 0x7fdc1c104ca0 2024-08-20T01:14:16.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.312+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0c00ce60 con 0x7fdc1c104ca0 2024-08-20T01:14:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.312+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc1c1a1540 con 0x7fdc1c104ca0 2024-08-20T01:14:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.312+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c010430 con 0x7fdc1c104ca0 2024-08-20T01:14:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.312+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc1c1a7c00 con 0x7fdc1c104ca0 2024-08-20T01:14:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.313+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc1c105080 con 0x7fdc1c104ca0 2024-08-20T01:14:16.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.315+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fdc0c0046c0 con 0x7fdc1c104ca0 2024-08-20T01:14:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.317+0000 7fdc13400640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 0x7fdc000b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.317+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7fdc0c0e4940 con 0x7fdc1c104ca0 2024-08-20T01:14:16.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.317+0000 7fdc13e00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 0x7fdc000b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.318+0000 7fdc13e00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 0x7fdc000b6280 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005ab0 tx=0x7fdc04005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:16.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.318+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdc0c055e20 con 0x7fdc1c104ca0 2024-08-20T01:14:16.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.537+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc1c19c2d0 con 0x7fdc1c104ca0 2024-08-20T01:14:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.538+0000 7fdc13400640 1 -- 172.21.15.138:0/3986989647 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11936 (secure 0 0 0) 0x7fdc1c19c2d0 con 0x7fdc1c104ca0 2024-08-20T01:14:16.537 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:14:16.537 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":34,"btime":"2024-08-20T01:13:24:677461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:13:24.223334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:14:16.537 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 34 2024-08-20T01:14:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 msgr2=0x7fdc000b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 0x7fdc000b6280 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdc04005ab0 tx=0x7fdc04005a20 comp rx=0 tx=0).stop 2024-08-20T01:14:16.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 msgr2=0x7fdc1c19b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c00e9f0 tx=0x7fdc0c00edf0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 shutdown_connections 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc000b3dc0 0x7fdc000b6280 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc1c10b030 0x7fdc1c1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc1c10a6b0 0x7fdc1c19b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 --2- 172.21.15.138:0/3986989647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc1c104ca0 0x7fdc1c19b1a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 >> 172.21.15.138:0/3986989647 conn(0x7fdc1c1006f0 msgr2=0x7fdc1c102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 shutdown_connections 2024-08-20T01:14:16.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:16.540+0000 7fdc21c00640 1 -- 172.21.15.138:0/3986989647 wait complete. 2024-08-20T01:14:16.957 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:16 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3986989647' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:16.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:16 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3986989647' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:16.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:16 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3986989647' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:17.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.36 MB/sec execute 149 sec latency 27341.575 ms 2024-08-20T01:14:17.488 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 948b0677 --release --await' 2024-08-20T01:14:17.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 948b0677 --release --await 2024-08-20T01:14:17.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:17 smithi138 ceph-mon[18076]: pgmap v547: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 984 KiB/s wr, 38 op/s 2024-08-20T01:14:17.858 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:17.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:17 smithi167 ceph-mon[21965]: pgmap v547: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 984 KiB/s wr, 38 op/s 2024-08-20T01:14:17.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:17 smithi142 ceph-mon[21799]: pgmap v547: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 984 KiB/s wr, 38 op/s 2024-08-20T01:14:18.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 257321 4.33 MB/sec execute 150 sec latency 28341.638 ms 2024-08-20T01:14:18.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 -- 172.21.15.138:0/615323712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010a6b0 msgr2=0x7f131010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.495 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 --2- 172.21.15.138:0/615323712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010a6b0 0x7f131010aaf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f12f8009900 tx=0x7f12f802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 -- 172.21.15.138:0/615323712 shutdown_connections 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 --2- 172.21.15.138:0/615323712 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f131010b030 0x7f1310111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 --2- 172.21.15.138:0/615323712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010a6b0 0x7f131010aaf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 --2- 172.21.15.138:0/615323712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1310104ca0 0x7f1310105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.495+0000 7f1314c00640 1 -- 172.21.15.138:0/615323712 >> 172.21.15.138:0/615323712 conn(0x7f1310100730 msgr2=0x7f1310102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.496+0000 7f1314c00640 1 -- 172.21.15.138:0/615323712 shutdown_connections 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.496+0000 7f1314c00640 1 -- 172.21.15.138:0/615323712 wait complete. 2024-08-20T01:14:18.496 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.496+0000 7f1314c00640 1 Processor -- start 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 -- start start 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1310104ca0 0x7f131019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 0x7f131019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1310114d00 con 0x7f131010b030 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1310114b80 con 0x7f131010a6b0 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1314c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1310114e80 con 0x7f1310104ca0 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f130e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f1305400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 0x7f131019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f130e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56276/0 (socket says 172.21.15.138:56276) 2024-08-20T01:14:18.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f130e800640 1 -- 172.21.15.138:0/775718796 learned_addr learned my addr 172.21.15.138:0/775718796 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.497+0000 7f130de00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1310104ca0 0x7f131019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 -- 172.21.15.138:0/775718796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1310104ca0 msgr2=0x7f131019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1310104ca0 0x7f131019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 -- 172.21.15.138:0/775718796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 msgr2=0x7f131019b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 0x7f131019b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 -- 172.21.15.138:0/775718796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13101a1260 con 0x7f131010b030 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1305400640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 0x7f131019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f130e800640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f130800eb10 tx=0x7f130800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1307400640 1 -- 172.21.15.138:0/775718796 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308004b20 con 0x7f131010b030 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13101a1550 con 0x7f131010b030 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1307400640 1 -- 172.21.15.138:0/775718796 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1308004cc0 con 0x7f131010b030 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1307400640 1 -- 172.21.15.138:0/775718796 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1308010690 con 0x7f131010b030 2024-08-20T01:14:18.499 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.498+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13101a7de0 con 0x7f131010b030 2024-08-20T01:14:18.500 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.501+0000 7f1317642740 1 Processor -- start 2024-08-20T01:14:18.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1307400640 1 -- 172.21.15.138:0/775718796 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1308005020 con 0x7f131010b030 2024-08-20T01:14:18.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 -- start start 2024-08-20T01:14:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 0x55ce747390d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 0x55ce74745060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce74747990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce746da670 con 0x55ce74738cf0 2024-08-20T01:14:18.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce746aed70 con 0x55ce747455a0 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.502+0000 7f1317642740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ce7469e270 con 0x55ce74739610 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f1305400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 0x55ce747390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 0x55ce74745060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f1305400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 0x55ce747390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56284/0 (socket says 172.21.15.138:56284) 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f1305400640 1 -- 172.21.15.138:0/3600120360 learned_addr learned my addr 172.21.15.138:0/3600120360 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f1305400640 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce74747990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 -- 172.21.15.138:0/3600120360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 msgr2=0x55ce74747990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce74747990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 -- 172.21.15.138:0/3600120360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 msgr2=0x55ce747390d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 0x55ce747390d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 -- 172.21.15.138:0/3600120360 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce7463a870 con 0x55ce74739610 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.503+0000 7f130de00640 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 0x55ce74745060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1300009c20 tx=0x7f1300011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa9faba70700cf7b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f1307400640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 0x7f12f40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f12ff000640 1 -- 172.21.15.138:0/3600120360 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300009dd0 con 0x55ce74739610 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f12ff000640 1 -- 172.21.15.138:0/3600120360 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1300005020 con 0x55ce74739610 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f12ff000640 1 -- 172.21.15.138:0/3600120360 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300004a00 con 0x55ce74739610 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f1307400640 1 -- 172.21.15.138:0/775718796 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f13080e4890 con 0x7f131010b030 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.504+0000 7f1305400640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 0x7f12f40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- 172.21.15.138:0/3600120360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 msgr2=0x55ce74745060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 0x55ce74745060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1300009c20 tx=0x7f1300011810 comp rx=0 tx=0).stop 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- 172.21.15.138:0/3600120360 shutdown_connections 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce74747990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74739610 0x55ce74745060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 --2- 172.21.15.138:0/3600120360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74738cf0 0x55ce747390d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- 172.21.15.138:0/3600120360 >> 172.21.15.138:0/3600120360 conn(0x55ce74736b20 msgr2=0x55ce74737cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1305400640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 0x7f12f40b6280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006010 tx=0x7f12f8041040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- 172.21.15.138:0/3600120360 shutdown_connections 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- 172.21.15.138:0/3600120360 wait complete. 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.505+0000 7f1317642740 1 -- start start 2024-08-20T01:14:18.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 Processor -- start 2024-08-20T01:14:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747421f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.507 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74742730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ce74624ff0 con 0x55ce747455a0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ce746a22b0 con 0x55ce74739610 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ce74500bb0 con 0x55ce74738cf0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1317642740 10 client.0 _ll_register_callbacks cb 0x55ce743a9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f130de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f130e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74742730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f1305400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f130de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46328/0 (socket says 172.21.15.138:46328) 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.506+0000 7f130de00640 1 -- 172.21.15.138:0/1759075719 learned_addr learned my addr 172.21.15.138:0/1759075719 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 -- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 msgr2=0x55ce747421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747421f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 -- 172.21.15.138:0/1759075719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 msgr2=0x55ce74742730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74742730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 -- 172.21.15.138:0/1759075719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce743fdb00 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f130de00640 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f130001b040 tx=0x7f13000056b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e44bcc3d476ffcf1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.507+0000 7f12f3400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300029020 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1300004950 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1300005020 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1300034050 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f1317642740 10 client.26239 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ce74474960 con 0x55ce74738cf0 2024-08-20T01:14:18.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.508+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1300005020 con 0x55ce74738cf0 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1300023420 con 0x55ce74738cf0 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 10 client.26239 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 10 client.26239 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 20 client.26239 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce74754ba0 0x55ce74756f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 4 client.26239 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce746b0cc0 con 0x55ce74754ba0 2024-08-20T01:14:18.511 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.509+0000 7f1305400640 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce74754ba0 0x55ce74756f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.510+0000 7f12f3400640 10 client.26239 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.510+0000 7f1305400640 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce74754ba0 0x55ce74756f90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18217bda9c0ef710 server_cookie=9afe00034770360a in_seq=0 out_seq=0 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f12f3400640 1 -- 172.21.15.138:0/1759075719 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce746b0cc0 con 0x55ce74754ba0 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f12f3400640 10 client.26239 handle_command_reply: tid=0 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 client.26239 shutdown 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 msgr2=0x55ce74744da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f130001b040 tx=0x7f13000056b0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 shutdown_connections 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce74754ba0 0x55ce74756f90 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74742730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 --2- 172.21.15.138:0/1759075719 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce74744da0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f12f3400640 0 client.26239 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:18.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 >> 172.21.15.138:0/1759075719 conn(0x55ce74736b20 msgr2=0x55ce7473c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 shutdown_connections 2024-08-20T01:14:18.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 1 -- 172.21.15.138:0/1759075719 wait complete. 2024-08-20T01:14:18.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.511+0000 7f1317642740 20 client.26239 trim_cache size 0 max 16384 2024-08-20T01:14:18.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 Processor -- start 2024-08-20T01:14:18.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 -- start start 2024-08-20T01:14:18.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce747511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce7475c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b4680 con 0x55ce74739610 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b1f20 con 0x55ce747455a0 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1317642740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13101afd70 con 0x55ce74738cf0 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1305400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1305400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56298/0 (socket says 172.21.15.138:56298) 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.529+0000 7f1305400640 1 -- 172.21.15.138:0/281787392 learned_addr learned my addr 172.21.15.138:0/281787392 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f130de00640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce747511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f130e800640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce7475c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 -- 172.21.15.138:0/281787392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 msgr2=0x55ce747511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce747511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.530 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 -- 172.21.15.138:0/281787392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 msgr2=0x55ce7475c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce7475c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 -- 172.21.15.138:0/281787392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce74736950 con 0x55ce74739610 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f130e800640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce7475c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f1305400640 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f12f802a9e0 tx=0x7f12f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be0bdefb028bb5af server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f12fe600640 1 -- 172.21.15.138:0/281787392 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f8046070 con 0x55ce74739610 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f12fe600640 1 -- 172.21.15.138:0/281787392 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f8031e20 con 0x55ce74739610 2024-08-20T01:14:18.531 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.530+0000 7f12fe600640 1 -- 172.21.15.138:0/281787392 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f803d450 con 0x55ce74739610 2024-08-20T01:14:18.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 -- 172.21.15.138:0/281787392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 msgr2=0x55ce74741180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f12f802a9e0 tx=0x7f12f8009260 comp rx=0 tx=0).stop 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 -- 172.21.15.138:0/281787392 shutdown_connections 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce747455a0 0x55ce7475c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce74739610 0x55ce74741180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 --2- 172.21.15.138:0/281787392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce747511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 -- 172.21.15.138:0/281787392 >> 172.21.15.138:0/281787392 conn(0x55ce74736b20 msgr2=0x55ce74740a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 -- 172.21.15.138:0/281787392 shutdown_connections 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.531+0000 7f1317642740 1 -- 172.21.15.138:0/281787392 wait complete. 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 -- start start 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 Processor -- start 2024-08-20T01:14:18.534 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747490b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74749840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b03e0 con 0x55ce747455a0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b1850 con 0x55ce74739610 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f13101b0ab0 con 0x55ce74738cf0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.532+0000 7f1317642740 10 client.0 _ll_register_callbacks cb 0x7f13080093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74749840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46342/0 (socket says 172.21.15.138:46342) 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f1305400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74749840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56314/0 (socket says 172.21.15.138:56314) 2024-08-20T01:14:18.535 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 -- 172.21.15.138:0/3412407772 learned_addr learned my addr 172.21.15.138:0/3412407772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 -- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 msgr2=0x55ce747490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 -- 172.21.15.138:0/3412407772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 msgr2=0x55ce74749840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74749840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 -- 172.21.15.138:0/3412407772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce747375a0 con 0x55ce74738cf0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f1305400640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f130de00640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f13000239b0 tx=0x7f13000054e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8ad452e8c9a410 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.533+0000 7f12f3400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1300029040 con 0x55ce74738cf0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1300004040 con 0x55ce74738cf0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12f8031e20 con 0x55ce74738cf0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1300023c30 con 0x55ce74738cf0 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 10 client.26245 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:18.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ce747bf7f0 con 0x55ce74738cf0 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12f8031e20 con 0x55ce74738cf0 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1300004e10 con 0x55ce74738cf0 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 10 client.26245 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 10 client.26245 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 20 client.26245 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce747bfed0 0x55ce747c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 4 client.26245 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "948b0677", "release": true, "await": true}] 2024-08-20T01:14:18.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.534+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "948b0677", "release": true, "await": true}) -- 0x55ce746ac760 con 0x55ce747bfed0 2024-08-20T01:14:18.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.535+0000 7f1305400640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce747bfed0 0x55ce747c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:18.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.535+0000 7f1305400640 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce747bfed0 0x55ce747c22f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cc22360a83a42c9 server_cookie=1bc18acb0ff85dcd in_seq=0 out_seq=0 2024-08-20T01:14:18.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.535+0000 7f12f3400640 10 client.26245 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:18.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f12f3400640 1 -- 172.21.15.138:0/3412407772 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ce746ac760 con 0x55ce747bfed0 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f12f3400640 10 client.26245 handle_command_reply: tid=0 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 client.26245 shutdown 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 msgr2=0x55ce7474dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f13000239b0 tx=0x7f13000054e0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 shutdown_connections 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ce747bfed0 0x55ce747c22f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ce747455a0 0x55ce74749840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce74739610 0x55ce747490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 --2- 172.21.15.138:0/3412407772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ce74738cf0 0x55ce7474dbe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f12f3400640 0 client.26245 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 >> 172.21.15.138:0/3412407772 conn(0x55ce74736b20 msgr2=0x55ce7473cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 shutdown_connections 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 1 -- 172.21.15.138:0/3412407772 wait complete. 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.542+0000 7f1317642740 20 client.26245 trim_cache size 0 max 16384 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 34, 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 28, 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:14:18.543 INFO:teuthology.orchestra.run.smithi138.stdout: "948b0677": { 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 28, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 150.0, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:18.544 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:18.545 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.543+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 msgr2=0x7f12f40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.543+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 0x7f12f40b6280 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f12f8006010 tx=0x7f12f8041040 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.543+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 msgr2=0x7f13101a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.543+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f130800eb10 tx=0x7f130800c2d0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 shutdown_connections 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12f40b3dc0 0x7f12f40b6280 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f131010b030 0x7f13101a0a90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f131010a6b0 0x7f131019b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 --2- 172.21.15.138:0/775718796 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1310104ca0 0x7f131019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 >> 172.21.15.138:0/775718796 conn(0x7f1310100730 msgr2=0x7f1310102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 shutdown_connections 2024-08-20T01:14:18.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:18.544+0000 7f1314c00640 1 -- 172.21.15.138:0/775718796 wait complete. 2024-08-20T01:14:18.961 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:18 smithi138 ceph-mon[18076]: pgmap v548: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-08-20T01:14:18.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:18 smithi167 ceph-mon[21965]: pgmap v548: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-08-20T01:14:18.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:18 smithi142 ceph-mon[21799]: pgmap v548: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 1.1 MiB/s wr, 39 op/s 2024-08-20T01:14:19.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 259055 4.39 MB/sec execute 151 sec latency 28724.246 ms 2024-08-20T01:14:19.408 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '948b0677', total seconds elapsed: 34.1 2024-08-20T01:14:19.408 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-08-20T01:14:19.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:14:19.851 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:20.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 259433 4.37 MB/sec execute 152 sec latency 438.992 ms 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 -- 172.21.15.138:0/819784993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450104130 msgr2=0x7f945010c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 --2- 172.21.15.138:0/819784993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450104130 0x7f945010c620 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f943c009900 tx=0x7f943c02a410 comp rx=0 tx=0).stop 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 -- 172.21.15.138:0/819784993 shutdown_connections 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 --2- 172.21.15.138:0/819784993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f945010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 --2- 172.21.15.138:0/819784993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450104130 0x7f945010c620 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 --2- 172.21.15.138:0/819784993 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450103810 0x7f9450103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 -- 172.21.15.138:0/819784993 >> 172.21.15.138:0/819784993 conn(0x7f94500fd2d0 msgr2=0x7f94500ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 -- 172.21.15.138:0/819784993 shutdown_connections 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.760+0000 7f9456000640 1 -- 172.21.15.138:0/819784993 wait complete. 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.761+0000 7f9456000640 1 Processor -- start 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 -- start start 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 0x7f94501046e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:20.771 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f9450105160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9450112930 con 0x7f945010cc70 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f94501127b0 con 0x7f9450103810 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9456000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9450112ab0 con 0x7f9450104130 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f944fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9454c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 0x7f94501046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f944fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46358/0 (socket says 172.21.15.138:46358) 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f944fe00640 1 -- 172.21.15.138:0/1024737978 learned_addr learned my addr 172.21.15.138:0/1024737978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.762+0000 7f9454c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 0x7f94501046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39852/0 (socket says 172.21.15.138:39852) 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f9455600640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f9450105160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 -- 172.21.15.138:0/1024737978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 msgr2=0x7f94501046e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 0x7f94501046e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 -- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 msgr2=0x7f9450105160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f9450105160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:20.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 -- 172.21.15.138:0/1024737978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94501ab540 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f9455600640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f9450105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.763+0000 7f944fe00640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ab0 tx=0x7f943c003ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.764+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c03f070 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.764+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c031b50 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.764+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94501ab7d0 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.764+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c035030 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.764+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94501abd00 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.765+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9450103bf0 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.768+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f943c042070 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.769+0000 7f944d600640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 0x7f94380b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.769+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f943c104b80 con 0x7f9450104130 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.770+0000 7f9454c00640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 0x7f94380b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:20.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.770+0000 7f9454c00640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 0x7f94380b6320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9444009bd0 tx=0x7f9444005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:20.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:20.771+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f943c0cbae0 con 0x7f9450104130 2024-08-20T01:14:21.046 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.046+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f94501059e0 con 0x7f9450104130 2024-08-20T01:14:21.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.047+0000 7f944d600640 1 -- 172.21.15.138:0/1024737978 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11936 (secure 0 0 0) 0x7f94501059e0 con 0x7f9450104130 2024-08-20T01:14:21.047 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:14:21.047 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":34,"btime":"2024-08-20T01:13:24:677461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:13:24.223334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:14:21.047 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 34 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 msgr2=0x7f94380b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 0x7f94380b6320 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9444009bd0 tx=0x7f9444005f90 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 msgr2=0x7f9450104c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f943c005ab0 tx=0x7f943c003ae0 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 shutdown_connections 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94380b3e60 0x7f94380b6320 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945010cc70 0x7f9450105160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9450104130 0x7f9450104c20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 --2- 172.21.15.138:0/1024737978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9450103810 0x7f94501046e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 >> 172.21.15.138:0/1024737978 conn(0x7f94500fd2d0 msgr2=0x7f94500fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:21.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 shutdown_connections 2024-08-20T01:14:21.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:21.050+0000 7f9456000640 1 -- 172.21.15.138:0/1024737978 wait complete. 2024-08-20T01:14:21.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 260661 4.37 MB/sec execute 153 sec latency 708.179 ms 2024-08-20T01:14:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:21 smithi167 ceph-mon[21965]: pgmap v549: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-08-20T01:14:21.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:21 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1024737978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:21 smithi142 ceph-mon[21799]: pgmap v549: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-08-20T01:14:21.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:21 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1024737978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:22.027 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:21 smithi138 ceph-mon[18076]: pgmap v549: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 1.5 MiB/s wr, 79 op/s 2024-08-20T01:14:22.027 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:21 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1024737978' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:22.065 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-08-20T01:14:22.065 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-08-20T01:14:22.153 INFO:tasks.workunit.client.0.smithi138.stdout: 1 260661 4.34 MB/sec execute 154 sec latency 1011.567 ms 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 -- 172.21.15.138:0/3067192493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0102800 msgr2=0x7f24b0102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 --2- 172.21.15.138:0/3067192493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0102800 0x7f24b0102be0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009900 tx=0x7f24a402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 -- 172.21.15.138:0/3067192493 shutdown_connections 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 --2- 172.21.15.138:0/3067192493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108b80 0x7f24b010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 --2- 172.21.15.138:0/3067192493 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0108200 0x7f24b0108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 --2- 172.21.15.138:0/3067192493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0102800 0x7f24b0102be0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 -- 172.21.15.138:0/3067192493 >> 172.21.15.138:0/3067192493 conn(0x7f24b00fe280 msgr2=0x7f24b01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 -- 172.21.15.138:0/3067192493 shutdown_connections 2024-08-20T01:14:22.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.183+0000 7f24b6800640 1 -- 172.21.15.138:0/3067192493 wait complete. 2024-08-20T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.184+0000 7f24b6800640 1 Processor -- start 2024-08-20T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.184+0000 7f24b6800640 1 -- start start 2024-08-20T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 0x7f24b019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 0x7f24b01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24b0112890 con 0x7f24b0108b80 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24b0112710 con 0x7f24b0108200 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b6800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24b0112a10 con 0x7f24b0102800 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 0x7f24b01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 0x7f24b01a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56334/0 (socket says 172.21.15.138:56334) 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39870/0 (socket says 172.21.15.138:39870) 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b5e00640 1 -- 172.21.15.138:0/1231307776 learned_addr learned my addr 172.21.15.138:0/1231307776 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b5400640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 0x7f24b019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 -- 172.21.15.138:0/1231307776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 msgr2=0x7f24b019b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 0x7f24b019b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 -- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 msgr2=0x7f24b01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 0x7f24b01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b4a00640 1 -- 172.21.15.138:0/1231307776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b01a1110 con 0x7f24b0108200 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.185+0000 7f24b5400640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 0x7f24b019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24b4a00640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9f0 tx=0x7f249c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24ae000640 1 -- 172.21.15.138:0/1231307776 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c00ccc0 con 0x7f24b0108200 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24ae000640 1 -- 172.21.15.138:0/1231307776 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f249c0108e0 con 0x7f24b0108200 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b01a1400 con 0x7f24b0108200 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24ae000640 1 -- 172.21.15.138:0/1231307776 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c018880 con 0x7f24b0108200 2024-08-20T01:14:22.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.186+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b01a7ca0 con 0x7f24b0108200 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 Processor -- start 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 -- start start 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984bf3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984bf67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555984b5d210 con 0x555984bf43c0 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555984b4c710 con 0x555984be8430 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.189+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555984b50750 con 0x555984be7b10 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24ae000640 1 -- 172.21.15.138:0/1231307776 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f249c010430 con 0x7f24b0108200 2024-08-20T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46380/0 (socket says 172.21.15.138:46380) 2024-08-20T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 -- 172.21.15.138:0/4013811825 learned_addr learned my addr 172.21.15.138:0/4013811825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:22.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5400640 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984bf3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 -- 172.21.15.138:0/4013811825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 msgr2=0x555984bf3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984bf3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 -- 172.21.15.138:0/4013811825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 msgr2=0x555984bf67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984bf67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 -- 172.21.15.138:0/4013811825 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555984b18de0 con 0x555984be7b10 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5400640 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984bf3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.190+0000 7f24b5e00640 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24a8011470 tx=0x7f24a8011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e5a9a26432f2ad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24a2600640 1 -- 172.21.15.138:0/4013811825 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8004b20 con 0x555984be7b10 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24a2600640 1 -- 172.21.15.138:0/4013811825 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a8004cc0 con 0x555984be7b10 2024-08-20T01:14:22.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24a2600640 1 -- 172.21.15.138:0/4013811825 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a80053e0 con 0x555984be7b10 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ae000640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 0x7f24980b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4013811825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 msgr2=0x555984be7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24a8011470 tx=0x7f24a8011930 comp rx=0 tx=0).stop 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4013811825 shutdown_connections 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984bf67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984bf3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4013811825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.191+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4013811825 >> 172.21.15.138:0/4013811825 conn(0x555984be5940 msgr2=0x555984be6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.192+0000 7f24ae000640 1 -- 172.21.15.138:0/1231307776 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f249c0e4740 con 0x7f24b0108200 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.192+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4013811825 shutdown_connections 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.192+0000 7f24b5400640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 0x7f24980b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.192+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4013811825 wait complete. 2024-08-20T01:14:22.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.192+0000 7f24ba9fd740 1 -- start start 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 Processor -- start 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b5400640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 0x7f24980b6280 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a4004220 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bf3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bf0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bf14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555984a85ab0 con 0x555984be8430 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555984af3140 con 0x555984bf43c0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555984bf70a0 con 0x555984be7b10 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24ba9fd740 10 client.0 _ll_register_callbacks cb 0x555984b065d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bf0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bf3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bf14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bf3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46384/0 (socket says 172.21.15.138:46384) 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.193+0000 7f24b5400640 1 -- 172.21.15.138:0/3234193399 learned_addr learned my addr 172.21.15.138:0/3234193399 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 -- 172.21.15.138:0/3234193399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 msgr2=0x555984bf3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bf3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 -- 172.21.15.138:0/3234193399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 msgr2=0x555984bf0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bf0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 -- 172.21.15.138:0/3234193399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555984af4170 con 0x555984bf43c0 2024-08-20T01:14:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bf14c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24a801b040 tx=0x7f24a800fd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55c9a5fe7c3c95a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f2496a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8028070 con 0x555984bf43c0 2024-08-20T01:14:22.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24a8004cc0 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.194+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f24a800fb20 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24a8034050 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 10 client.26297 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5559848af790 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24a8004cc0 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f24a80245e0 con 0x555984bf43c0 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 10 client.26297 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 20 client.26297 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c03ea0 0x555984c06290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 4 client.26297 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:22.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.195+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555984b5f160 con 0x555984c03ea0 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.196+0000 7f24b5400640 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c03ea0 0x555984c06290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.196+0000 7f24b5400640 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c03ea0 0x555984c06290 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59c0e9baa95fa3b8 server_cookie=e03d0d6e0f88e3f2 in_seq=0 out_seq=0 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.196+0000 7f2496a00640 10 client.26297 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f2496a00640 1 -- 172.21.15.138:0/3234193399 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555984b5f160 con 0x555984c03ea0 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f2496a00640 10 client.26297 handle_command_reply: tid=0 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 client.26297 shutdown 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 msgr2=0x555984bf14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bf14c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24a801b040 tx=0x7f24a800fd50 comp rx=0 tx=0).stop 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 shutdown_connections 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c03ea0 0x555984c06290 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bf14c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bf0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f2496a00640 0 client.26297 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3234193399 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bf3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 >> 172.21.15.138:0/3234193399 conn(0x555984be5940 msgr2=0x555984beb500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 shutdown_connections 2024-08-20T01:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.197+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3234193399 wait complete. 2024-08-20T01:14:22.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.198+0000 7f24ba9fd740 20 client.26297 trim_cache size 0 max 16384 2024-08-20T01:14:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 Processor -- start 2024-08-20T01:14:22.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 -- start start 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984c0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.214+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01b44d0 con 0x555984bf43c0 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01b1d70 con 0x555984be8430 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01afbc0 con 0x555984be7b10 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984c0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46388/0 (socket says 172.21.15.138:46388) 2024-08-20T01:14:22.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39898/0 (socket says 172.21.15.138:39898) 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b5400640 1 -- 172.21.15.138:0/4205556439 learned_addr learned my addr 172.21.15.138:0/4205556439 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b4a00640 1 -- 172.21.15.138:0/4205556439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 msgr2=0x555984be7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.215+0000 7f24b4a00640 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24b4a00640 1 -- 172.21.15.138:0/4205556439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 msgr2=0x555984c0bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24b4a00640 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984c0bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24b4a00640 1 -- 172.21.15.138:0/4205556439 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555984928620 con 0x555984be8430 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24b5400640 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24b4a00640 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f249c0e1ae0 tx=0x7f249c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104fa3eb77e736f5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24a1c00640 1 -- 172.21.15.138:0/4205556439 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0af5a0 con 0x555984be8430 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24a1c00640 1 -- 172.21.15.138:0/4205556439 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f249c0b4050 con 0x555984be8430 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.216+0000 7f24a1c00640 1 -- 172.21.15.138:0/4205556439 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c0e4310 con 0x555984be8430 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4205556439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 msgr2=0x555984c0ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f249c0e1ae0 tx=0x7f249c00ce60 comp rx=0 tx=0).stop 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4205556439 shutdown_connections 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984bf43c0 0x555984c0bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984be8430 0x555984c0ba00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/4205556439 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984be7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4205556439 >> 172.21.15.138:0/4205556439 conn(0x555984be5940 msgr2=0x555984be6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4205556439 shutdown_connections 2024-08-20T01:14:22.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.217+0000 7f24ba9fd740 1 -- 172.21.15.138:0/4205556439 wait complete. 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.218+0000 7f24ba9fd740 1 -- start start 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.218+0000 7f24ba9fd740 1 Processor -- start 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bfca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bed7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01b0230 con 0x555984be8430 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01b16a0 con 0x555984bf43c0 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f24b01b0900 con 0x555984be7b10 2024-08-20T01:14:22.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24ba9fd740 10 client.0 _ll_register_callbacks cb 0x7f24a800f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24b4a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bfca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24b5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bed7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24b5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46404/0 (socket says 172.21.15.138:46404) 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.219+0000 7f24b5400640 1 -- 172.21.15.138:0/3840416009 learned_addr learned my addr 172.21.15.138:0/3840416009 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 -- 172.21.15.138:0/3840416009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 msgr2=0x555984bfc520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 -- 172.21.15.138:0/3840416009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 msgr2=0x555984bfca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bfca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 -- 172.21.15.138:0/3840416009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555984bedef0 con 0x555984bf43c0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5400640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24b5e00640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bed7b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000f80 tx=0x7f24a8005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=710081a4ec7ea4da server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f2496a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a8028050 con 0x555984bf43c0 2024-08-20T01:14:22.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f24a800fa90 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.220+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f249c0b4050 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f24a8011030 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 10 client.26303 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x555984c76c30 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f249c0b4050 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f24a80052f0 con 0x555984bf43c0 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 10 client.26303 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 10 client.26303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 20 client.26303 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c77270 0x555984c79690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 4 client.26303 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.221+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555984b5ac00 con 0x555984c77270 2024-08-20T01:14:22.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.222+0000 7f24b5400640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c77270 0x555984c79690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.223+0000 7f2496a00640 10 client.26303 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:22.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.223+0000 7f24b5400640 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c77270 0x555984c79690 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2d22133d7e3fe1 server_cookie=2fca1115b727a508 in_seq=0 out_seq=0 2024-08-20T01:14:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.250+0000 7f2496a00640 1 -- 172.21.15.138:0/3840416009 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555984b5ac00 con 0x555984c77270 2024-08-20T01:14:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f2496a00640 10 client.26303 handle_command_reply: tid=0 2024-08-20T01:14:22.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 client.26303 shutdown 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 msgr2=0x555984bed7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bed7b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f24a8000f80 tx=0x7f24a8005ea0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 shutdown_connections 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555984c77270 0x555984c79690 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555984bf43c0 0x555984bed7b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555984be8430 0x555984bfca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 --2- 172.21.15.138:0/3840416009 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555984be7b10 0x555984bfc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f2496a00640 0 client.26303 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 >> 172.21.15.138:0/3840416009 conn(0x555984be5940 msgr2=0x555984bebdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 shutdown_connections 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 1 -- 172.21.15.138:0/3840416009 wait complete. 2024-08-20T01:14:22.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.251+0000 7f24ba9fd740 20 client.26303 trim_cache size 0 max 16384 2024-08-20T01:14:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:14:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:14:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:14:22.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 msgr2=0x7f24980b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 0x7f24980b6280 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f24a4000c00 tx=0x7f24a4004220 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 msgr2=0x7f24b019b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f249c00e9f0 tx=0x7f249c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 shutdown_connections 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f24980b3dc0 0x7f24980b6280 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0108b80 0x7f24b01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24b0108200 0x7f24b019b5b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 --2- 172.21.15.138:0/1231307776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f24b0102800 0x7f24b019b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 >> 172.21.15.138:0/1231307776 conn(0x7f24b00fe280 msgr2=0x7f24b0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 shutdown_connections 2024-08-20T01:14:22.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:22.252+0000 7f24b6800640 1 -- 172.21.15.138:0/1231307776 wait complete. 2024-08-20T01:14:22.488 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:23.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262798 4.40 MB/sec execute 155 sec latency 1093.334 ms 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 -- 172.21.15.138:0/1056457540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e010ef10 msgr2=0x7ff7e0111300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 --2- 172.21.15.138:0/1056457540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e010ef10 0x7ff7e0111300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009ca0 tx=0x7ff7d802a8b0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 -- 172.21.15.138:0/1056457540 shutdown_connections 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 --2- 172.21.15.138:0/1056457540 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e010ef10 0x7ff7e0111300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 --2- 172.21.15.138:0/1056457540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 0x7ff7e010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 --2- 172.21.15.138:0/1056457540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e0075210 0x7ff7e00755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 -- 172.21.15.138:0/1056457540 >> 172.21.15.138:0/1056457540 conn(0x7ff7e00fe350 msgr2=0x7ff7e0100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 -- 172.21.15.138:0/1056457540 shutdown_connections 2024-08-20T01:14:23.159 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.160+0000 7ff7df200640 1 -- 172.21.15.138:0/1056457540 wait complete. 2024-08-20T01:14:23.160 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.161+0000 7ff7df200640 1 Processor -- start 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.161+0000 7ff7df200640 1 -- start start 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 0x7ff7e0102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 0x7ff7e0103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0114af0 con 0x7ff7e0075b30 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0114970 con 0x7ff7e010ef10 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7df200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e0114c70 con 0x7ff7e0075210 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7dde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7dde00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46416/0 (socket says 172.21.15.138:46416) 2024-08-20T01:14:23.161 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.162+0000 7ff7dde00640 1 -- 172.21.15.138:0/732332701 learned_addr learned my addr 172.21.15.138:0/732332701 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:23.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7de800640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 0x7ff7e0103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dd400640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 0x7ff7e0102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 -- 172.21.15.138:0/732332701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 msgr2=0x7ff7e0103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 0x7ff7e0103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 -- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 msgr2=0x7ff7e0102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 0x7ff7e0102bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 -- 172.21.15.138:0/732332701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7e01ab4d0 con 0x7ff7e0075210 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7de800640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 0x7ff7e0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7dde00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e9f0 tx=0x7ff7d000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7d6a00640 1 -- 172.21.15.138:0/732332701 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d000cd80 con 0x7ff7e0075210 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7d6a00640 1 -- 172.21.15.138:0/732332701 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d00108e0 con 0x7ff7e0075210 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.163+0000 7ff7d6a00640 1 -- 172.21.15.138:0/732332701 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0018980 con 0x7ff7e0075210 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.164+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7e01ab7c0 con 0x7ff7e0075210 2024-08-20T01:14:23.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.164+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7e01abcb0 con 0x7ff7e0075210 2024-08-20T01:14:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.167+0000 7ff7e783e740 1 Processor -- start 2024-08-20T01:14:23.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.167+0000 7ff7e783e740 1 -- start start 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 0x55d34f23a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f23ae80 0x55d34f23d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d34f1ce670 con 0x55d34f22e5d0 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34f1a27c0 con 0x55d34f23ae80 2024-08-20T01:14:23.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7e783e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d34f191cc0 con 0x55d34f22eef0 2024-08-20T01:14:23.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56400/0 (socket says 172.21.15.138:56400) 2024-08-20T01:14:23.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7de800640 1 -- 172.21.15.138:0/2159546774 learned_addr learned my addr 172.21.15.138:0/2159546774 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7d6a00640 1 -- 172.21.15.138:0/732332701 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff7d0010430 con 0x7ff7e0075210 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.168+0000 7ff7dd400640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f23ae80 0x55d34f23d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7dde00640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 0x55d34f23a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 -- 172.21.15.138:0/2159546774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 msgr2=0x55d34f23a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 0x55d34f23a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 -- 172.21.15.138:0/2159546774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f23ae80 msgr2=0x55d34f23d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f23ae80 0x55d34f23d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 -- 172.21.15.138:0/2159546774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34f125110 con 0x55d34f22e5d0 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7dde00640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 0x55d34f23a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:14:23.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7de800640 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009a20 tx=0x7ff7d8009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8e08e68f9cc43e4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.169+0000 7ff7cb400640 1 -- 172.21.15.138:0/2159546774 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8030800 con 0x55d34f22e5d0 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7cb400640 1 -- 172.21.15.138:0/2159546774 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d8036040 con 0x55d34f22e5d0 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7cb400640 1 -- 172.21.15.138:0/2159546774 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004480 con 0x55d34f22e5d0 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7d6a00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 0x7ff7c40b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7e783e740 1 -- 172.21.15.138:0/2159546774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 msgr2=0x55d34f22e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8009a20 tx=0x7ff7d8009360 comp rx=0 tx=0).stop 2024-08-20T01:14:23.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7d6a00640 1 -- 172.21.15.138:0/732332701 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff7d00e47a0 con 0x7ff7e0075210 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.170+0000 7ff7dde00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 0x7ff7c40b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 -- 172.21.15.138:0/2159546774 shutdown_connections 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f23ae80 0x55d34f23d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22eef0 0x55d34f23a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2159546774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f22e9b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 -- 172.21.15.138:0/2159546774 >> 172.21.15.138:0/2159546774 conn(0x55d34f22c400 msgr2=0x55d34f22d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 -- 172.21.15.138:0/2159546774 shutdown_connections 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7e783e740 1 -- 172.21.15.138:0/2159546774 wait complete. 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.171+0000 7ff7dde00640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 0x7ff7c40b63a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d34f22d2e0 tx=0x7ff7d000cf20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 -- start start 2024-08-20T01:14:23.171 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 Processor -- start 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f237e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f2388c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d34f137780 con 0x55d34f22e5d0 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d34f195d00 con 0x55d34f22eef0 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d34eff47a0 con 0x55d34f23ae80 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.172+0000 7ff7e783e740 10 client.0 _ll_register_callbacks cb 0x55d34f13f940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f237e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39942/0 (socket says 172.21.15.138:39942) 2024-08-20T01:14:23.173 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 -- 172.21.15.138:0/1351700893 learned_addr learned my addr 172.21.15.138:0/1351700893 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7de800640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f2388c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 -- 172.21.15.138:0/1351700893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 msgr2=0x55d34f2388c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f2388c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 -- 172.21.15.138:0/1351700893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 msgr2=0x55d34f237e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f237e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 -- 172.21.15.138:0/1351700893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34f1449c0 con 0x55d34f22eef0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc013710 tx=0x7ff7cc013bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277101173a39c167 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.173+0000 7ff7bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc011dd0 con 0x55d34f22eef0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d8036040 con 0x55d34f22eef0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff7cc004020 con 0x55d34f22eef0 2024-08-20T01:14:23.174 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff7cc026050 con 0x55d34f22eef0 2024-08-20T01:14:23.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7e783e740 10 client.26315 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:23.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d34ef00b90 con 0x55d34f22eef0 2024-08-20T01:14:23.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d8036040 con 0x55d34f22eef0 2024-08-20T01:14:23.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.174+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7ff7cc01a030 con 0x55d34f22eef0 2024-08-20T01:14:23.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 10 client.26315 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:23.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 10 client.26315 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 20 client.26315 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f24a7f0 0x55d34f24cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 4 client.26315 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d34f1a4710 con 0x55d34f24a7f0 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.175+0000 7ff7de800640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f24a7f0 0x55d34f24cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.176+0000 7ff7de800640 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f24a7f0 0x55d34f24cc90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e754e5f350394f1 server_cookie=4f982bdc606fdb7b in_seq=0 out_seq=0 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.176+0000 7ff7bfe00640 10 client.26315 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.176+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1351700893 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d34f1a4710 con 0x55d34f24a7f0 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.176+0000 7ff7bfe00640 10 client.26315 handle_command_reply: tid=0 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 client.26315 shutdown 2024-08-20T01:14:23.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 msgr2=0x55d34f238380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc013710 tx=0x7ff7cc013bd0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 shutdown_connections 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f24a7f0 0x55d34f24cc90 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f2388c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f238380 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1351700893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22e5d0 0x55d34f237e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7bfe00640 0 client.26315 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 >> 172.21.15.138:0/1351700893 conn(0x55d34f22c400 msgr2=0x55d34f231f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 shutdown_connections 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 1 -- 172.21.15.138:0/1351700893 wait complete. 2024-08-20T01:14:23.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.177+0000 7ff7e783e740 20 client.26315 trim_cache size 0 max 16384 2024-08-20T01:14:23.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 Processor -- start 2024-08-20T01:14:23.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 -- start start 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 0x55d34f252560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f252aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b87b0 con 0x55d34f22eef0 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b6050 con 0x55d34f22e5d0 2024-08-20T01:14:23.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.197+0000 7ff7e783e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b3ea0 con 0x55d34f23ae80 2024-08-20T01:14:23.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39952/0 (socket says 172.21.15.138:39952) 2024-08-20T01:14:23.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 -- 172.21.15.138:0/2551435255 learned_addr learned my addr 172.21.15.138:0/2551435255 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:23.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f252aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7dd400640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 0x55d34f252560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 -- 172.21.15.138:0/2551435255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 msgr2=0x55d34f252aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f252aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 -- 172.21.15.138:0/2551435255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 msgr2=0x55d34f252560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 0x55d34f252560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7de800640 1 -- 172.21.15.138:0/2551435255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d34f22ce80 con 0x55d34f22e5d0 2024-08-20T01:14:23.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.198+0000 7ff7dd400640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 0x55d34f252560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:23.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.199+0000 7ff7de800640 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8004af0 tx=0x7ff7d80040e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d58fd4295527fc0c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.199+0000 7ff7caa00640 1 -- 172.21.15.138:0/2551435255 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d804a040 con 0x55d34f22e5d0 2024-08-20T01:14:23.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.199+0000 7ff7caa00640 1 -- 172.21.15.138:0/2551435255 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d8036030 con 0x55d34f22e5d0 2024-08-20T01:14:23.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.199+0000 7ff7caa00640 1 -- 172.21.15.138:0/2551435255 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004580 con 0x55d34f22e5d0 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 -- 172.21.15.138:0/2551435255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 msgr2=0x55d34f238520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff7d8004af0 tx=0x7ff7d80040e0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 -- 172.21.15.138:0/2551435255 shutdown_connections 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f23ae80 0x55d34f252aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f22eef0 0x55d34f252560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 --2- 172.21.15.138:0/2551435255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22e5d0 0x55d34f238520 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 -- 172.21.15.138:0/2551435255 >> 172.21.15.138:0/2551435255 conn(0x55d34f22c400 msgr2=0x55d34f237e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 -- 172.21.15.138:0/2551435255 shutdown_connections 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.200+0000 7ff7e783e740 1 -- 172.21.15.138:0/2551435255 wait complete. 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.201+0000 7ff7e783e740 1 -- start start 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.201+0000 7ff7e783e740 1 Processor -- start 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f23f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f23ae80 0x55d34f2b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b4510 con 0x55d34f23ae80 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b5980 con 0x55d34f22eef0 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff7e01b4be0 con 0x55d34f22e5d0 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7e783e740 10 client.0 _ll_register_callbacks cb 0x7ff7cc0096c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7de800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f23ae80 0x55d34f2b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f23f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7de800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7de800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46444/0 (socket says 172.21.15.138:46444) 2024-08-20T01:14:23.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7de800640 1 -- 172.21.15.138:0/1341757470 learned_addr learned my addr 172.21.15.138:0/1341757470 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:23.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 -- 172.21.15.138:0/1341757470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 msgr2=0x55d34f23f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 -- 172.21.15.138:0/1341757470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f23ae80 msgr2=0x55d34f2b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f23ae80 0x55d34f2b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7dd400640 1 -- 172.21.15.138:0/1341757470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d34f2b4ba0 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.202+0000 7ff7de800640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.203+0000 7ff7dd400640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f23f850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0098a0 tx=0x7ff7cc0118f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82ef1f18360a96c9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.203+0000 7ff7bfe00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.203+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00fa40 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.203+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7ff7cc00fbe0 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.203+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7d8036030 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff7cc03d050 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 10 client.26321 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55d34f2b5e60 con 0x55d34f22eef0 2024-08-20T01:14:23.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d8036030 con 0x55d34f22eef0 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7ff7cc01a040 con 0x55d34f22eef0 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 10 client.26321 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 10 client.26321 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 20 client.26321 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f2b6680 0x55d34f2b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 4 client.26321 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55d34f1a01b0 con 0x55d34f2b6680 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.204+0000 7ff7de800640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f2b6680 0x55d34f2b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.205+0000 7ff7de800640 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f2b6680 0x55d34f2b8aa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4cef21c09ba0cee server_cookie=3da8858a0f0d3c15 in_seq=0 out_seq=0 2024-08-20T01:14:23.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.205+0000 7ff7bfe00640 10 client.26321 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:23.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7bfe00640 1 -- 172.21.15.138:0/1341757470 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55d34f1a01b0 con 0x55d34f2b6680 2024-08-20T01:14:23.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7bfe00640 10 client.26321 handle_command_reply: tid=0 2024-08-20T01:14:23.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 client.26321 shutdown 2024-08-20T01:14:23.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 msgr2=0x55d34f23f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f23f850 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0098a0 tx=0x7ff7cc0118f0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 shutdown_connections 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d34f2b6680 0x55d34f2b8aa0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d34f23ae80 0x55d34f2b4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d34f22eef0 0x55d34f23f850 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 --2- 172.21.15.138:0/1341757470 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d34f22e5d0 0x55d34f23f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 >> 172.21.15.138:0/1341757470 conn(0x55d34f22c400 msgr2=0x55d34f238630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7bfe00640 0 client.26321 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.401+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 shutdown_connections 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7e783e740 1 -- 172.21.15.138:0/1341757470 wait complete. 2024-08-20T01:14:23.401 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7e783e740 20 client.26321 trim_cache size 0 max 16384 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 34, 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 30, 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "12f7d6": { 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 30, 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 150.0, 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:23.403 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:14:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 msgr2=0x7ff7c40b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 0x7ff7c40b63a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x55d34f22d2e0 tx=0x7ff7d000cf20 comp rx=0 tx=0).stop 2024-08-20T01:14:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 msgr2=0x7ff7e0102680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:23.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.402+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000e9f0 tx=0x7ff7d000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 shutdown_connections 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff7c40b3ee0 0x7ff7c40b63a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff7e010ef10 0x7ff7e0103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7e0075b30 0x7ff7e0102bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 --2- 172.21.15.138:0/732332701 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff7e0075210 0x7ff7e0102680 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 >> 172.21.15.138:0/732332701 conn(0x7ff7e00fe350 msgr2=0x7ff7e0111060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 shutdown_connections 2024-08-20T01:14:23.405 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:23.403+0000 7ff7df200640 1 -- 172.21.15.138:0/732332701 wait complete. 2024-08-20T01:14:23.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:23 smithi167 ceph-mon[21965]: pgmap v550: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-08-20T01:14:23.970 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:23 smithi138 ceph-mon[18076]: pgmap v550: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-08-20T01:14:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:23 smithi142 ceph-mon[21799]: pgmap v550: 129 pgs: 129 active+clean; 159 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 669 KiB/s rd, 1.5 MiB/s wr, 76 op/s 2024-08-20T01:14:24.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.37 MB/sec execute 156 sec latency 918.011 ms 2024-08-20T01:14:24.707 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '12f7d6', quiesce took 5.3 seconds. Will release after: 24.7 2024-08-20T01:14:25.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:24 smithi138 ceph-mon[18076]: pgmap v551: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-08-20T01:14:25.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.35 MB/sec execute 157 sec latency 1918.132 ms 2024-08-20T01:14:25.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:24 smithi167 ceph-mon[21965]: pgmap v551: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-08-20T01:14:25.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:24 smithi142 ceph-mon[21799]: pgmap v551: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-08-20T01:14:26.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.32 MB/sec execute 158 sec latency 2918.222 ms 2024-08-20T01:14:27.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.29 MB/sec execute 159 sec latency 3918.328 ms 2024-08-20T01:14:27.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:27 smithi167 ceph-mon[21965]: pgmap v552: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:14:27.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:27 smithi142 ceph-mon[21799]: pgmap v552: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:14:28.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:27 smithi138 ceph-mon[18076]: pgmap v552: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:14:28.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.27 MB/sec execute 160 sec latency 4918.453 ms 2024-08-20T01:14:29.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.24 MB/sec execute 161 sec latency 5918.597 ms 2024-08-20T01:14:29.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:29 smithi167 ceph-mon[21965]: pgmap v553: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-08-20T01:14:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:29 smithi142 ceph-mon[21799]: pgmap v553: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-08-20T01:14:30.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:29 smithi138 ceph-mon[18076]: pgmap v553: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 112 op/s 2024-08-20T01:14:30.154 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.21 MB/sec execute 162 sec latency 6918.738 ms 2024-08-20T01:14:31.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.19 MB/sec execute 163 sec latency 7918.812 ms 2024-08-20T01:14:31.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:31 smithi167 ceph-mon[21965]: pgmap v554: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 969 KiB/s wr, 112 op/s 2024-08-20T01:14:31.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:31 smithi142 ceph-mon[21799]: pgmap v554: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 969 KiB/s wr, 112 op/s 2024-08-20T01:14:32.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:31 smithi138 ceph-mon[18076]: pgmap v554: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 969 KiB/s wr, 112 op/s 2024-08-20T01:14:32.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.16 MB/sec execute 164 sec latency 8918.928 ms 2024-08-20T01:14:33.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:32 smithi138 ceph-mon[18076]: pgmap v555: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 512 KiB/s wr, 72 op/s 2024-08-20T01:14:33.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.14 MB/sec execute 165 sec latency 9919.127 ms 2024-08-20T01:14:33.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:32 smithi167 ceph-mon[21965]: pgmap v555: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 512 KiB/s wr, 72 op/s 2024-08-20T01:14:33.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:32 smithi142 ceph-mon[21799]: pgmap v555: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 512 KiB/s wr, 72 op/s 2024-08-20T01:14:34.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.11 MB/sec execute 166 sec latency 10919.266 ms 2024-08-20T01:14:35.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.09 MB/sec execute 167 sec latency 11919.365 ms 2024-08-20T01:14:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.375+0000 7f2794200640 1 -- 172.21.15.138:0/3181198098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c1083f0 msgr2=0x7f278c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.375+0000 7f2794200640 1 --2- 172.21.15.138:0/3181198098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c1083f0 0x7f278c108830 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2778009900 tx=0x7f277802a410 comp rx=0 tx=0).stop 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 -- 172.21.15.138:0/3181198098 shutdown_connections 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 --2- 172.21.15.138:0/3181198098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c108d70 0x7f278c10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 --2- 172.21.15.138:0/3181198098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c1083f0 0x7f278c108830 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 --2- 172.21.15.138:0/3181198098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 -- 172.21.15.138:0/3181198098 >> 172.21.15.138:0/3181198098 conn(0x7f278c0fe430 msgr2=0x7f278c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 -- 172.21.15.138:0/3181198098 shutdown_connections 2024-08-20T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.376+0000 7f2794200640 1 -- 172.21.15.138:0/3181198098 wait complete. 2024-08-20T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 Processor -- start 2024-08-20T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 -- start start 2024-08-20T01:14:35.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c1083f0 0x7f278c19b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 0x7f278c1a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f278c112a80 con 0x7f278c108d70 2024-08-20T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f278c112900 con 0x7f278c1083f0 2024-08-20T01:14:35.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2794200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f278c112c00 con 0x7f278c1029f0 2024-08-20T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2792200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 0x7f278c1a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2791800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2791800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:50230/0 (socket says 172.21.15.138:50230) 2024-08-20T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2791800640 1 -- 172.21.15.138:0/3159887026 learned_addr learned my addr 172.21.15.138:0/3159887026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:35.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.377+0000 7f2792200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 0x7f278c1a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54986/0 (socket says 172.21.15.138:54986) 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2790e00640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c1083f0 0x7f278c19b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 -- 172.21.15.138:0/3159887026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c1083f0 msgr2=0x7f278c19b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c1083f0 0x7f278c19b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 -- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 msgr2=0x7f278c1a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 0x7f278c1a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 -- 172.21.15.138:0/3159887026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a1280 con 0x7f278c1029f0 2024-08-20T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f2791800640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.378+0000 7f278a000640 1 -- 172.21.15.138:0/3159887026 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000cd80 con 0x7f278c1029f0 2024-08-20T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.379+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c1a1570 con 0x7f278c1029f0 2024-08-20T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.379+0000 7f278a000640 1 -- 172.21.15.138:0/3159887026 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27800054d0 con 0x7f278c1029f0 2024-08-20T01:14:35.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.379+0000 7f278a000640 1 -- 172.21.15.138:0/3159887026 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780010690 con 0x7f278c1029f0 2024-08-20T01:14:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.379+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c1a7c30 con 0x7f278c1029f0 2024-08-20T01:14:35.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 Processor -- start 2024-08-20T01:14:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 -- start start 2024-08-20T01:14:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701af250 0x55b5701baca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.398 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b570124210 con 0x55b5701bb1e0 2024-08-20T01:14:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b570113710 con 0x55b5701af250 2024-08-20T01:14:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f27971ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b570117750 con 0x55b5701ae930 2024-08-20T01:14:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f278a000640 1 -- 172.21.15.138:0/3159887026 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2780004e90 con 0x7f278c1029f0 2024-08-20T01:14:35.399 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f2790e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f2790e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54990/0 (socket says 172.21.15.138:54990) 2024-08-20T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.382+0000 7f2790e00640 1 -- 172.21.15.138:0/3767823892 learned_addr learned my addr 172.21.15.138:0/3767823892 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2792200640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2791800640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701af250 0x55b5701baca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.400 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 -- 172.21.15.138:0/3767823892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 msgr2=0x55b5701aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 -- 172.21.15.138:0/3767823892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701af250 msgr2=0x55b5701baca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701af250 0x55b5701baca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 -- 172.21.15.138:0/3767823892 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5700dfde0 con 0x55b5701bb1e0 2024-08-20T01:14:35.401 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2792200640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.383+0000 7f2790e00640 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2778000c00 tx=0x7f2778004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe8a2779c65d0c0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f277ea00640 1 -- 172.21.15.138:0/3767823892 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2778035040 con 0x55b5701bb1e0 2024-08-20T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f277ea00640 1 -- 172.21.15.138:0/3767823892 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27780054f0 con 0x55b5701bb1e0 2024-08-20T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f277ea00640 1 -- 172.21.15.138:0/3767823892 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27780035f0 con 0x55b5701bb1e0 2024-08-20T01:14:35.402 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f278a000640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 0x7f27740b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f27971ea740 1 -- 172.21.15.138:0/3767823892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 msgr2=0x55b5701bd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.384+0000 7f27971ea740 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f2778000c00 tx=0x7f2778004170 comp rx=0 tx=0).stop 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 -- 172.21.15.138:0/3767823892 shutdown_connections 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f278a000640 1 -- 172.21.15.138:0/3159887026 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f27800e4790 con 0x7f278c1029f0 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701bd5d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.403 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701af250 0x55b5701baca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 --2- 172.21.15.138:0/3767823892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 -- 172.21.15.138:0/3767823892 >> 172.21.15.138:0/3767823892 conn(0x55b5701ac760 msgr2=0x55b5701ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 -- 172.21.15.138:0/3767823892 shutdown_connections 2024-08-20T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f2792200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 0x7f27740b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.404 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 -- 172.21.15.138:0/3767823892 wait complete. 2024-08-20T01:14:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 -- start start 2024-08-20T01:14:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f2792200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 0x7f27740b6200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2784005fd0 tx=0x7f27840096a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.385+0000 7f27971ea740 1 Processor -- start 2024-08-20T01:14:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.405 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 0x55b5701b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b57004cab0 con 0x55b5701af250 2024-08-20T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b5700ba140 con 0x55b5701bb1e0 2024-08-20T01:14:35.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b5701bdeb0 con 0x55b5701ae930 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f27971ea740 10 client.0 _ll_register_callbacks cb 0x55b5700cd5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2790e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 0x55b5701b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2790e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 0x55b5701b7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54992/0 (socket says 172.21.15.138:54992) 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50582/0 (socket says 172.21.15.138:50582) 2024-08-20T01:14:35.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2790e00640 1 -- 172.21.15.138:0/1335709303 learned_addr learned my addr 172.21.15.138:0/1335709303 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2791800640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 -- 172.21.15.138:0/1335709303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 msgr2=0x55b5701ba9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701ba9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 -- 172.21.15.138:0/1335709303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 msgr2=0x55b5701b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.408 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.386+0000 7f2792200640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 0x55b5701b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2792200640 1 -- 172.21.15.138:0/1335709303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5700bb170 con 0x55b5701bb1e0 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2792200640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2784009fd0 tx=0x7f2784004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33c937286f6463a0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2773400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278402c060 con 0x55b5701bb1e0 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27780054f0 con 0x55b5701bb1e0 2024-08-20T01:14:35.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f2784017030 con 0x55b5701bb1e0 2024-08-20T01:14:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2784003970 con 0x55b5701bb1e0 2024-08-20T01:14:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f27971ea740 10 client.26333 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.387+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b56fe76790 con 0x55b5701bb1e0 2024-08-20T01:14:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27780054f0 con 0x55b5701bb1e0 2024-08-20T01:14:35.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f2784004140 con 0x55b5701bb1e0 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 10 client.26333 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 10 client.26333 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 20 client.26333 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b5701cae60 0x55b5701cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 4 client.26333 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:35.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b570126160 con 0x55b5701cae60 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.388+0000 7f2791800640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b5701cae60 0x55b5701cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.389+0000 7f2791800640 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b5701cae60 0x55b5701cd290 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f9316d20532c2cd server_cookie=b2e24b950219dcc0 in_seq=0 out_seq=0 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.389+0000 7f2773400640 10 client.26333 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f2773400640 1 -- 172.21.15.138:0/1335709303 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b570126160 con 0x55b5701cae60 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f2773400640 10 client.26333 handle_command_reply: tid=0 2024-08-20T01:14:35.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 client.26333 shutdown 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 msgr2=0x55b5701b8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2784009fd0 tx=0x7f2784004ab0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 shutdown_connections 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b5701cae60 0x55b5701cd290 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701b8340 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701af250 0x55b5701b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f2773400640 0 client.26333 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 --2- 172.21.15.138:0/1335709303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701ae930 0x55b5701ba9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 >> 172.21.15.138:0/1335709303 conn(0x55b5701ac760 msgr2=0x55b5701b2300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 shutdown_connections 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 1 -- 172.21.15.138:0/1335709303 wait complete. 2024-08-20T01:14:35.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.390+0000 7f27971ea740 20 client.26333 trim_cache size 0 max 16384 2024-08-20T01:14:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 Processor -- start 2024-08-20T01:14:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 -- start start 2024-08-20T01:14:35.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701ae930 0x55b5701ab650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.405+0000 7f27971ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1b4530 con 0x55b5701bb1e0 2024-08-20T01:14:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f27971ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1b1dd0 con 0x55b5701ae930 2024-08-20T01:14:35.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f27971ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1afc20 con 0x55b5701af250 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:54996/0 (socket says 172.21.15.138:54996) 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 -- 172.21.15.138:0/2067985979 learned_addr learned my addr 172.21.15.138:0/2067985979 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2791800640 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701ae930 0x55b5701ab650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2790e00640 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 -- 172.21.15.138:0/2067985979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 msgr2=0x55b5701d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 -- 172.21.15.138:0/2067985979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701ae930 msgr2=0x55b5701ab650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701ae930 0x55b5701ab650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 -- 172.21.15.138:0/2067985979 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56feef620 con 0x55b5701bb1e0 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.406+0000 7f2792200640 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f27840045e0 tx=0x7f2784010570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7c396c014c96cb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f277e000640 1 -- 172.21.15.138:0/2067985979 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278401f240 con 0x55b5701bb1e0 2024-08-20T01:14:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f277e000640 1 -- 172.21.15.138:0/2067985979 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278401fcb0 con 0x55b5701bb1e0 2024-08-20T01:14:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f277e000640 1 -- 172.21.15.138:0/2067985979 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2784004200 con 0x55b5701bb1e0 2024-08-20T01:14:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 -- 172.21.15.138:0/2067985979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 msgr2=0x55b5701d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f27840045e0 tx=0x7f2784010570 comp rx=0 tx=0).stop 2024-08-20T01:14:35.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 -- 172.21.15.138:0/2067985979 shutdown_connections 2024-08-20T01:14:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701bb1e0 0x55b5701d3130 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 --2- 172.21.15.138:0/2067985979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701ae930 0x55b5701ab650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 -- 172.21.15.138:0/2067985979 >> 172.21.15.138:0/2067985979 conn(0x55b5701ac760 msgr2=0x55b5701aaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 -- 172.21.15.138:0/2067985979 shutdown_connections 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.407+0000 7f27971ea740 1 -- 172.21.15.138:0/2067985979 wait complete. 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 -- start start 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 Processor -- start 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701bea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1b0290 con 0x55b5701ae930 2024-08-20T01:14:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1b1700 con 0x55b5701bb1e0 2024-08-20T01:14:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1b0960 con 0x55b5701af250 2024-08-20T01:14:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.408+0000 7f27971ea740 10 client.0 _ll_register_callbacks cb 0x7f27840098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:35.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2790e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55008/0 (socket says 172.21.15.138:55008) 2024-08-20T01:14:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2792200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701bea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2790e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:50264/0 (socket says 172.21.15.138:50264) 2024-08-20T01:14:35.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 -- 172.21.15.138:0/1337938975 learned_addr learned my addr 172.21.15.138:0/1337938975 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 -- 172.21.15.138:0/1337938975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 msgr2=0x55b5701c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 -- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 msgr2=0x55b5701bea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701bea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 -- 172.21.15.138:0/1337938975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5701bf2a0 con 0x55b5701ae930 2024-08-20T01:14:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2792200640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2791800640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f27800a8260 tx=0x7f278000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2149ec5c73520c09 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2773400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:14:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27800ab6f0 con 0x55b5701ae930 2024-08-20T01:14:35.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f27800c03d0 con 0x55b5701ae930 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.409+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f278401fcb0 con 0x55b5701ae930 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f27800af3b0 con 0x55b5701ae930 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 10 client.16284 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55b5702358c0 con 0x55b5701ae930 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278401fcb0 con 0x55b5701ae930 2024-08-20T01:14:35.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f27800bf020 con 0x55b5701ae930 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 10 client.16284 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 20 client.16284 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b570236020 0x55b570238440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 4 client.16284 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.410+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b570121c00 con 0x55b570236020 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.411+0000 7f2790e00640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b570236020 0x55b570238440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.411+0000 7f2790e00640 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b570236020 0x55b570238440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=decb33c9624089c4 server_cookie=1e264a7b4f2f1ac7 in_seq=0 out_seq=0 2024-08-20T01:14:35.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.411+0000 7f2773400640 10 client.16284 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f2773400640 1 -- 172.21.15.138:0/1337938975 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b570121c00 con 0x55b570236020 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f2773400640 10 client.16284 handle_command_reply: tid=0 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 client.16284 shutdown 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 msgr2=0x55b5701c3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f27800a8260 tx=0x7f278000c7d0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 shutdown_connections 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b570236020 0x55b570238440 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b5701bb1e0 0x55b5701bea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b5701af250 0x55b5701c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f2773400640 0 client.16284 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 --2- 172.21.15.138:0/1337938975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b5701ae930 0x55b5701c3440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 >> 172.21.15.138:0/1337938975 conn(0x55b5701ac760 msgr2=0x55b5701ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 shutdown_connections 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.430+0000 7f27971ea740 1 -- 172.21.15.138:0/1337938975 wait complete. 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f27971ea740 20 client.16284 trim_cache size 0 max 16384 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:14:35.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:14:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 msgr2=0x7f27740b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 0x7f27740b6200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2784005fd0 tx=0x7f27840096a0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 msgr2=0x7f278c19b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:35.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f278000e9f0 tx=0x7f278000edf0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 shutdown_connections 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f27740b3d40 0x7f27740b6200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c108d70 0x7f278c1a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f278c1083f0 0x7f278c19b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 --2- 172.21.15.138:0/3159887026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f278c1029f0 0x7f278c19b1e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 >> 172.21.15.138:0/3159887026 conn(0x7f278c0fe430 msgr2=0x7f278c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 shutdown_connections 2024-08-20T01:14:35.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:35.431+0000 7f2794200640 1 -- 172.21.15.138:0/3159887026 wait complete. 2024-08-20T01:14:35.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:35 smithi167 ceph-mon[21965]: pgmap v556: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 510 KiB/s wr, 69 op/s 2024-08-20T01:14:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:35 smithi142 ceph-mon[21799]: pgmap v556: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 510 KiB/s wr, 69 op/s 2024-08-20T01:14:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:35 smithi138 ceph-mon[18076]: pgmap v556: 129 pgs: 129 active+clean; 161 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 642 KiB/s rd, 510 KiB/s wr, 69 op/s 2024-08-20T01:14:36.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.06 MB/sec execute 168 sec latency 12919.417 ms 2024-08-20T01:14:37.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.04 MB/sec execute 169 sec latency 13919.535 ms 2024-08-20T01:14:37.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:37 smithi167 ceph-mon[21965]: pgmap v557: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 154 KiB/s wr, 27 op/s 2024-08-20T01:14:37.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:37 smithi142 ceph-mon[21799]: pgmap v557: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 154 KiB/s wr, 27 op/s 2024-08-20T01:14:38.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:37 smithi138 ceph-mon[18076]: pgmap v557: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 154 KiB/s wr, 27 op/s 2024-08-20T01:14:38.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 4.01 MB/sec execute 170 sec latency 14919.649 ms 2024-08-20T01:14:39.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.99 MB/sec execute 171 sec latency 15919.777 ms 2024-08-20T01:14:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:39 smithi167 ceph-mon[21965]: pgmap v558: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 112 KiB/s wr, 15 op/s 2024-08-20T01:14:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:39 smithi142 ceph-mon[21799]: pgmap v558: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 112 KiB/s wr, 15 op/s 2024-08-20T01:14:40.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:39 smithi138 ceph-mon[18076]: pgmap v558: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 112 KiB/s wr, 15 op/s 2024-08-20T01:14:40.155 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.97 MB/sec execute 172 sec latency 16919.903 ms 2024-08-20T01:14:40.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:40 smithi167 ceph-mon[21965]: pgmap v559: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-08-20T01:14:40.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:40 smithi142 ceph-mon[21799]: pgmap v559: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-08-20T01:14:41.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:40 smithi138 ceph-mon[18076]: pgmap v559: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 123 KiB/s wr, 20 op/s 2024-08-20T01:14:41.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.94 MB/sec execute 173 sec latency 17919.972 ms 2024-08-20T01:14:42.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.92 MB/sec execute 174 sec latency 18920.070 ms 2024-08-20T01:14:43.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.90 MB/sec execute 175 sec latency 19920.186 ms 2024-08-20T01:14:43.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:43 smithi167 ceph-mon[21965]: pgmap v560: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:14:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:43 smithi142 ceph-mon[21799]: pgmap v560: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:14:44.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:43 smithi138 ceph-mon[18076]: pgmap v560: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 109 KiB/s wr, 14 op/s 2024-08-20T01:14:44.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.88 MB/sec execute 176 sec latency 20920.326 ms 2024-08-20T01:14:45.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.86 MB/sec execute 177 sec latency 21920.460 ms 2024-08-20T01:14:45.803 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:45 smithi138 ceph-mon[18076]: pgmap v561: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 116 KiB/s wr, 16 op/s 2024-08-20T01:14:45.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:45 smithi167 ceph-mon[21965]: pgmap v561: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 116 KiB/s wr, 16 op/s 2024-08-20T01:14:45.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:45 smithi142 ceph-mon[21799]: pgmap v561: 129 pgs: 129 active+clean; 162 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 116 KiB/s wr, 16 op/s 2024-08-20T01:14:46.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.83 MB/sec execute 178 sec latency 22920.546 ms 2024-08-20T01:14:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:46 smithi142 ceph-mon[21799]: pgmap v562: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-08-20T01:14:47.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:46 smithi138 ceph-mon[18076]: pgmap v562: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-08-20T01:14:47.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.81 MB/sec execute 179 sec latency 23920.658 ms 2024-08-20T01:14:47.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:46 smithi167 ceph-mon[21965]: pgmap v562: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 289 KiB/s wr, 16 op/s 2024-08-20T01:14:47.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:47 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:14:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:47 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:14:48.156 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.79 MB/sec execute 180 sec latency 24920.722 ms 2024-08-20T01:14:48.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:47 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 -- 172.21.15.138:0/1727728884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6944101520 msgr2=0x7f6944101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 --2- 172.21.15.138:0/1727728884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6944101520 0x7f6944101900 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6934009900 tx=0x7f693402a400 comp rx=0 tx=0).stop 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 -- 172.21.15.138:0/1727728884 shutdown_connections 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 --2- 172.21.15.138:0/1727728884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f694410cf50 0x7f694410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 --2- 172.21.15.138:0/1727728884 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694410ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 --2- 172.21.15.138:0/1727728884 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6944101520 0x7f6944101900 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 -- 172.21.15.138:0/1727728884 >> 172.21.15.138:0/1727728884 conn(0x7f6944076a70 msgr2=0x7f6944076e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 -- 172.21.15.138:0/1727728884 shutdown_connections 2024-08-20T01:14:48.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.549+0000 7f694ae00640 1 -- 172.21.15.138:0/1727728884 wait complete. 2024-08-20T01:14:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 Processor -- start 2024-08-20T01:14:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 -- start start 2024-08-20T01:14:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694419b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694410cf50 0x7f69441a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6944112ae0 con 0x7f6944101520 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6944112960 con 0x7f694410cf50 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.550+0000 7f694ae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6944112c60 con 0x7f694410a620 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6948e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694410cf50 0x7f69441a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56644/0 (socket says 172.21.15.138:56644) 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 -- 172.21.15.138:0/3851875329 learned_addr learned my addr 172.21.15.138:0/3851875329 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943400640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694419b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 -- 172.21.15.138:0/3851875329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 msgr2=0x7f694419b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694419b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 -- 172.21.15.138:0/3851875329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694410cf50 msgr2=0x7f69441a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694410cf50 0x7f69441a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 -- 172.21.15.138:0/3851875329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69441a12d0 con 0x7f6944101520 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943400640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.551+0000 7f6943e00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6934005ab0 tx=0x7f6934003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.552+0000 7f6940c00640 1 -- 172.21.15.138:0/3851875329 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693403f070 con 0x7f6944101520 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.552+0000 7f6940c00640 1 -- 172.21.15.138:0/3851875329 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6934031b60 con 0x7f6944101520 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.552+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69441a1560 con 0x7f6944101520 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.552+0000 7f6940c00640 1 -- 172.21.15.138:0/3851875329 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934035030 con 0x7f6944101520 2024-08-20T01:14:48.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.552+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69441a7c80 con 0x7f6944101520 2024-08-20T01:14:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 Processor -- start 2024-08-20T01:14:48.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 -- start start 2024-08-20T01:14:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6940c00640 1 -- 172.21.15.138:0/3851875329 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6934042070 con 0x7f6944101520 2024-08-20T01:14:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 0x55849307cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493088ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558492ff2210 con 0x55849307c930 2024-08-20T01:14:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558492fe1710 con 0x5584930891e0 2024-08-20T01:14:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f694dc43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558492fe5750 con 0x55849307d250 2024-08-20T01:14:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6948e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 0x55849307cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6943400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6943e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493088ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6943400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:40214/0 (socket says 172.21.15.138:40214) 2024-08-20T01:14:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.555+0000 7f6943400640 1 -- 172.21.15.138:0/1769838335 learned_addr learned my addr 172.21.15.138:0/1769838335 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 -- 172.21.15.138:0/1769838335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 msgr2=0x558493088ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493088ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 -- 172.21.15.138:0/1769838335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 msgr2=0x55849307cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 0x55849307cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 -- 172.21.15.138:0/1769838335 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558492fadde0 con 0x5584930891e0 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6948e00640 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 0x55849307cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6943400640 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69300114f0 tx=0x7f69300119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6d6f2497821c73 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6939600640 1 -- 172.21.15.138:0/1769838335 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930005020 con 0x5584930891e0 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6939600640 1 -- 172.21.15.138:0/1769838335 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69300047a0 con 0x5584930891e0 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.556+0000 7f6939600640 1 -- 172.21.15.138:0/1769838335 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693000f840 con 0x5584930891e0 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f6940c00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 0x7f69280b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f6940c00640 1 -- 172.21.15.138:0/3851875329 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f6934104a90 con 0x7f6944101520 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 -- 172.21.15.138:0/1769838335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 msgr2=0x55849308b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f69300114f0 tx=0x7f69300119b0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f6948e00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 0x7f69280b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 -- 172.21.15.138:0/1769838335 shutdown_connections 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308b5d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493088ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 --2- 172.21.15.138:0/1769838335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307c930 0x55849307cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 -- 172.21.15.138:0/1769838335 >> 172.21.15.138:0/1769838335 conn(0x55849307a760 msgr2=0x55849307b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 -- 172.21.15.138:0/1769838335 shutdown_connections 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f694dc43740 1 -- 172.21.15.138:0/1769838335 wait complete. 2024-08-20T01:14:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.557+0000 7f6948e00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 0x7f69280b6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f693c009fd0 tx=0x7f693c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 -- start start 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 Processor -- start 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493085e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584930891e0 0x558493086360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558492f1aab0 con 0x5584930891e0 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558492f88140 con 0x55849307c930 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55849308bec0 con 0x55849307d250 2024-08-20T01:14:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f694dc43740 10 client.0 _ll_register_callbacks cb 0x558492f9b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.558+0000 7f6943e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:40226/0 (socket says 172.21.15.138:40226) 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 -- 172.21.15.138:0/1802996149 learned_addr learned my addr 172.21.15.138:0/1802996149 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943400640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493085e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6948e00640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584930891e0 0x558493086360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 -- 172.21.15.138:0/1802996149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 msgr2=0x558493085e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493085e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 -- 172.21.15.138:0/1802996149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584930891e0 msgr2=0x558493086360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584930891e0 0x558493086360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 -- 172.21.15.138:0/1802996149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558492f89170 con 0x55849307c930 2024-08-20T01:14:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f6943e00640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6934000c00 tx=0x7f69340c7260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26f1390bb1ec0b11 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f692e000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6934049030 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f69341020b0 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69300047a0 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f693410d070 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f694dc43740 10 client.26351 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.559+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558492d44790 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69300047a0 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f69340cb600 con 0x55849307c930 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 10 client.26351 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 20 client.26351 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558493098dc0 0x55849309b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 4 client.26351 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558492ff4160 con 0x558493098dc0 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.560+0000 7f6943400640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558493098dc0 0x55849309b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.561+0000 7f6943400640 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558493098dc0 0x55849309b1f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f8ac136ff167f4 server_cookie=5dec6e6b22da5484 in_seq=0 out_seq=0 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.561+0000 7f692e000640 10 client.26351 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f692e000640 1 -- 172.21.15.138:0/1802996149 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558492ff4160 con 0x558493098dc0 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f692e000640 10 client.26351 handle_command_reply: tid=0 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 client.26351 shutdown 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 msgr2=0x5584930889e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6934000c00 tx=0x7f69340c7260 comp rx=0 tx=0).stop 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 shutdown_connections 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558493098dc0 0x55849309b1f0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5584930891e0 0x558493086360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307d250 0x558493085e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 --2- 172.21.15.138:0/1802996149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55849307c930 0x5584930889e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f692e000640 0 client.26351 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 >> 172.21.15.138:0/1802996149 conn(0x55849307a760 msgr2=0x558493080300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 shutdown_connections 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 1 -- 172.21.15.138:0/1802996149 wait complete. 2024-08-20T01:14:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.562+0000 7f694dc43740 20 client.26351 trim_cache size 0 max 16384 2024-08-20T01:14:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.577+0000 7f694dc43740 1 Processor -- start 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.577+0000 7f694dc43740 1 -- start start 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849307bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x5584930a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69441b4540 con 0x55849307d250 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69441b1de0 con 0x5584930891e0 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f694dc43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69441afc30 con 0x55849307c930 2024-08-20T01:14:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:40230/0 (socket says 172.21.15.138:40230) 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 -- 172.21.15.138:0/699959023 learned_addr learned my addr 172.21.15.138:0/699959023 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6943e00640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849307bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6943400640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x5584930a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 -- 172.21.15.138:0/699959023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 msgr2=0x55849307bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849307bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 -- 172.21.15.138:0/699959023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 msgr2=0x5584930a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x5584930a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6948e00640 1 -- 172.21.15.138:0/699959023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558492dbd620 con 0x5584930891e0 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6943400640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x5584930a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.578+0000 7f6943e00640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849307bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f6948e00640 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f693c009270 tx=0x7f693c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7da9c5903368d19 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f6938c00640 1 -- 172.21.15.138:0/699959023 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c01c530 con 0x5584930891e0 2024-08-20T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f6938c00640 1 -- 172.21.15.138:0/699959023 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693c02c070 con 0x5584930891e0 2024-08-20T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f6938c00640 1 -- 172.21.15.138:0/699959023 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c027be0 con 0x5584930891e0 2024-08-20T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f694dc43740 1 -- 172.21.15.138:0/699959023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 msgr2=0x5584930a1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f694dc43740 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f693c009270 tx=0x7f693c016000 comp rx=0 tx=0).stop 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.579+0000 7f694dc43740 1 -- 172.21.15.138:0/699959023 shutdown_connections 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x5584930a1040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x5584930a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 --2- 172.21.15.138:0/699959023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849307bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 -- 172.21.15.138:0/699959023 >> 172.21.15.138:0/699959023 conn(0x55849307a760 msgr2=0x55849307b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 -- 172.21.15.138:0/699959023 shutdown_connections 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 -- 172.21.15.138:0/699959023 wait complete. 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 -- start start 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.580+0000 7f694dc43740 1 Processor -- start 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x55849308f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f69441b02a0 con 0x55849307d250 2024-08-20T01:14:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f69441b1710 con 0x5584930891e0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f69441b0970 con 0x55849307c930 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f694dc43740 10 client.0 _ll_register_callbacks cb 0x7f6934108230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6943e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6943e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38796/0 (socket says 172.21.15.138:38796) 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:40242/0 (socket says 172.21.15.138:40242) 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6943e00640 1 -- 172.21.15.138:0/1114031124 learned_addr learned my addr 172.21.15.138:0/1114031124 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6943400640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x55849308f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 -- 172.21.15.138:0/1114031124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 msgr2=0x55849308f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 -- 172.21.15.138:0/1114031124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 msgr2=0x55849308f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x55849308f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 -- 172.21.15.138:0/1114031124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558493102c40 con 0x5584930891e0 2024-08-20T01:14:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6943e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f6948e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f693c018040 tx=0x7f693c005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26ebae5dda8cec4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.581+0000 7f692e000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c034020 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f693c02c070 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f693c027bf0 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f693c03f050 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f694dc43740 10 client.26357 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x558493103f50 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f693c02c070 con 0x5584930891e0 2024-08-20T01:14:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.582+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f693c01cca0 con 0x5584930891e0 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 10 client.26357 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 10 client.26357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 20 client.26357 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584931046b0 0x558493106ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 4 client.26357 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558492fefc00 con 0x5584931046b0 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f6943e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584931046b0 0x558493106ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f692e000640 10 client.26357 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.583+0000 7f6943e00640 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584931046b0 0x558493106ad0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d4a7f5df6ecd9a server_cookie=fed9c6771b865c98 in_seq=0 out_seq=0 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f692e000640 1 -- 172.21.15.138:0/1114031124 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558492fefc00 con 0x5584931046b0 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f692e000640 10 client.26357 handle_command_reply: tid=0 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 client.26357 shutdown 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 msgr2=0x55849308fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f693c018040 tx=0x7f693c005f50 comp rx=0 tx=0).stop 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 shutdown_connections 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584931046b0 0x558493106ad0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.593 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584930891e0 0x55849308fb50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f692e000640 0 client.26357 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55849307d250 0x55849308f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 --2- 172.21.15.138:0/1114031124 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55849307c930 0x55849308f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 >> 172.21.15.138:0/1114031124 conn(0x55849307a760 msgr2=0x558493078f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 shutdown_connections 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.594+0000 7f694dc43740 1 -- 172.21.15.138:0/1114031124 wait complete. 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694dc43740 20 client.26357 trim_cache size 0 max 16384 2024-08-20T01:14:48.594 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:14:48.595 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 msgr2=0x7f69280b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 0x7f69280b6200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f693c009fd0 tx=0x7f693c009500 comp rx=0 tx=0).stop 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 msgr2=0x7f694419b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6934005ab0 tx=0x7f6934003c10 comp rx=0 tx=0).stop 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 shutdown_connections 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f69280b3d40 0x7f69280b6200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f694410cf50 0x7f69441a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.596 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f694410a620 0x7f694419b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 --2- 172.21.15.138:0/3851875329 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6944101520 0x7f694419b230 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.595+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 >> 172.21.15.138:0/3851875329 conn(0x7f6944076a70 msgr2=0x7f69441089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.596+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 shutdown_connections 2024-08-20T01:14:48.597 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:14:48.596+0000 7f694ae00640 1 -- 172.21.15.138:0/3851875329 wait complete. 2024-08-20T01:14:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:14:48.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:14:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:14:48.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:48 smithi142 ceph-mon[21799]: pgmap v563: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 355 KiB/s wr, 15 op/s 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:14:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:14:49.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:14:49.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:14:49.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:48 smithi138 ceph-mon[18076]: pgmap v563: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 355 KiB/s wr, 15 op/s 2024-08-20T01:14:49.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.77 MB/sec execute 181 sec latency 25920.828 ms 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:14:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:14:49.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:14:49.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:14:49.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:14:49.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:14:49.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:48 smithi167 ceph-mon[21965]: pgmap v563: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 355 KiB/s wr, 15 op/s 2024-08-20T01:14:49.408 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:14:49.781 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:50.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.75 MB/sec execute 182 sec latency 26920.927 ms 2024-08-20T01:14:50.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.428+0000 7f4e2b800640 1 -- 172.21.15.138:0/880117253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 msgr2=0x7f4e2410aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.428+0000 7f4e2b800640 1 --2- 172.21.15.138:0/880117253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2410aaf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4e10009900 tx=0x7f4e1002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 -- 172.21.15.138:0/880117253 shutdown_connections 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 --2- 172.21.15.138:0/880117253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e2410b030 0x7f4e24111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 --2- 172.21.15.138:0/880117253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2410aaf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 --2- 172.21.15.138:0/880117253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e24104ca0 0x7f4e24105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 -- 172.21.15.138:0/880117253 >> 172.21.15.138:0/880117253 conn(0x7f4e241006f0 msgr2=0x7f4e24102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 -- 172.21.15.138:0/880117253 shutdown_connections 2024-08-20T01:14:50.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.429+0000 7f4e2b800640 1 -- 172.21.15.138:0/880117253 wait complete. 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 Processor -- start 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 -- start start 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 0x7f4e2419b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 0x7f4e241a0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24114cd0 con 0x7f4e2410a6b0 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24114b50 con 0x7f4e2410b030 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e2b800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e24114e50 con 0x7f4e24104ca0 2024-08-20T01:14:50.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e23e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56684/0 (socket says 172.21.15.138:56684) 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.430+0000 7f4e23e00640 1 -- 172.21.15.138:0/1892567666 learned_addr learned my addr 172.21.15.138:0/1892567666 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e28e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 0x7f4e2419b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e29800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 0x7f4e241a0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 -- 172.21.15.138:0/1892567666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 msgr2=0x7f4e2419b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 0x7f4e2419b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 -- 172.21.15.138:0/1892567666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 msgr2=0x7f4e241a0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 0x7f4e241a0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 -- 172.21.15.138:0/1892567666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e241a11f0 con 0x7f4e2410a6b0 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e28e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 0x7f4e2419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e29800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 0x7f4e241a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.431+0000 7f4e23e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4e10005ab0 tx=0x7f4e10003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:50.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.432+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1003f070 con 0x7f4e2410a6b0 2024-08-20T01:14:50.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.432+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e10031b60 con 0x7f4e2410a6b0 2024-08-20T01:14:50.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.432+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e241a1480 con 0x7f4e2410a6b0 2024-08-20T01:14:50.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.432+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e10035030 con 0x7f4e2410a6b0 2024-08-20T01:14:50.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.432+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e241a7ba0 con 0x7f4e2410a6b0 2024-08-20T01:14:50.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.433+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e24105080 con 0x7f4e2410a6b0 2024-08-20T01:14:50.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.435+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4e10042070 con 0x7f4e2410a6b0 2024-08-20T01:14:50.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.437+0000 7f4e21600640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 0x7f4e0c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:50.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.437+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f4e10104a10 con 0x7f4e2410a6b0 2024-08-20T01:14:50.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.438+0000 7f4e28e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 0x7f4e0c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:50.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.438+0000 7f4e28e00640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 0x7f4e0c0b6200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009fc0 tx=0x7f4e18005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:50.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.441+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4e100cb320 con 0x7f4e2410a6b0 2024-08-20T01:14:50.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.660+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e2410ce10 con 0x7f4e2410a6b0 2024-08-20T01:14:50.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.661+0000 7f4e21600640 1 -- 172.21.15.138:0/1892567666 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11936 (secure 0 0 0) 0x7f4e100cf870 con 0x7f4e2410a6b0 2024-08-20T01:14:50.661 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:14:50.661 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":34,"btime":"2024-08-20T01:13:24:677461+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:13:24.223334+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24694":{"gid":24694,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2113916191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2113916191},{"type":"v1","addr":"172.21.15.167:6839","nonce":2113916191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:14:50.661 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 34 2024-08-20T01:14:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 msgr2=0x7f4e0c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 0x7f4e0c0b6200 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4e18009fc0 tx=0x7f4e18005f90 comp rx=0 tx=0).stop 2024-08-20T01:14:50.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 msgr2=0x7f4e2419b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4e10005ab0 tx=0x7f4e10003c10 comp rx=0 tx=0).stop 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 shutdown_connections 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e0c0b3d40 0x7f4e0c0b6200 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e2410b030 0x7f4e241a0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e2410a6b0 0x7f4e2419b680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 --2- 172.21.15.138:0/1892567666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e24104ca0 0x7f4e2419b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 >> 172.21.15.138:0/1892567666 conn(0x7f4e241006f0 msgr2=0x7f4e24102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 shutdown_connections 2024-08-20T01:14:50.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:50.664+0000 7f4e2b800640 1 -- 172.21.15.138:0/1892567666 wait complete. 2024-08-20T01:14:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:50 smithi142 ceph-mon[21799]: pgmap v564: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 489 KiB/s wr, 27 op/s 2024-08-20T01:14:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:50 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1892567666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:51.039 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:50 smithi138 ceph-mon[18076]: pgmap v564: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 489 KiB/s wr, 27 op/s 2024-08-20T01:14:51.039 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1892567666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:51.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.73 MB/sec execute 183 sec latency 27920.964 ms 2024-08-20T01:14:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:50 smithi167 ceph-mon[21965]: pgmap v564: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 489 KiB/s wr, 27 op/s 2024-08-20T01:14:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:50 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1892567666' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:14:51.579 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 12f7d6 --release --await' 2024-08-20T01:14:51.579 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 12f7d6 --release --await 2024-08-20T01:14:51.950 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:14:52.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 262835 3.71 MB/sec execute 184 sec latency 28921.029 ms 2024-08-20T01:14:52.545 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 -- 172.21.15.138:0/2351657676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 msgr2=0x7f8238105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 --2- 172.21.15.138:0/2351657676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238105080 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8228009900 tx=0x7f822802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.546 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 -- 172.21.15.138:0/2351657676 shutdown_connections 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 --2- 172.21.15.138:0/2351657676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 0x7f8238111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 --2- 172.21.15.138:0/2351657676 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 --2- 172.21.15.138:0/2351657676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238105080 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 -- 172.21.15.138:0/2351657676 >> 172.21.15.138:0/2351657676 conn(0x7f8238100730 msgr2=0x7f8238102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 -- 172.21.15.138:0/2351657676 shutdown_connections 2024-08-20T01:14:52.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.546+0000 7f823ea00640 1 -- 172.21.15.138:0/2351657676 wait complete. 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 Processor -- start 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 -- start start 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 0x7f823810ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8238114d00 con 0x7f8238104ca0 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8238114b80 con 0x7f823810b030 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f823ea00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8238114e80 con 0x7f823810a6b0 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.548+0000 7f8237400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.548 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38830/0 (socket says 172.21.15.138:38830) 2024-08-20T01:14:52.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 -- 172.21.15.138:0/1454202497 learned_addr learned my addr 172.21.15.138:0/1454202497 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:52.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237e00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f823ca00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 0x7f823810ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 -- 172.21.15.138:0/1454202497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 msgr2=0x7f823810ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 0x7f823810ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 -- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 msgr2=0x7f8238111590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238111590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.549+0000 7f8237400640 1 -- 172.21.15.138:0/1454202497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823810f5c0 con 0x7f823810a6b0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f8237400640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f822400ea60 tx=0x7f822400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f8234c00640 1 -- 172.21.15.138:0/1454202497 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f822400cde0 con 0x7f823810a6b0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f8234c00640 1 -- 172.21.15.138:0/1454202497 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82240035f0 con 0x7f823810a6b0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381a1280 con 0x7f823810a6b0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f8234c00640 1 -- 172.21.15.138:0/1454202497 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224010690 con 0x7f823810a6b0 2024-08-20T01:14:52.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.550+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a1810 con 0x7f823810a6b0 2024-08-20T01:14:52.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.553+0000 7f82413fd740 1 Processor -- start 2024-08-20T01:14:52.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.553+0000 7f82413fd740 1 -- start start 2024-08-20T01:14:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc96060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b4dc2b670 con 0x564b4dc8a610 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564b4dbffd70 con 0x564b4dc965a0 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f82413fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b4dbef270 con 0x564b4dc89cf0 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f8237e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc96060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38842/0 (socket says 172.21.15.138:38842) 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 -- 172.21.15.138:0/3943415852 learned_addr learned my addr 172.21.15.138:0/3943415852 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 -- 172.21.15.138:0/3943415852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 msgr2=0x564b4dc98990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc98990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 -- 172.21.15.138:0/3943415852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 msgr2=0x564b4dc96060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc96060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.554+0000 7f823ca00640 1 -- 172.21.15.138:0/3943415852 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4db8b870 con 0x564b4dc89cf0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.555+0000 7f8234c00640 1 -- 172.21.15.138:0/1454202497 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f82240037a0 con 0x7f823810a6b0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.555+0000 7f823ca00640 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8230009fd0 tx=0x7f82300118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5defd4e2e183288 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.555+0000 7f822d200640 1 -- 172.21.15.138:0/3943415852 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230004d20 con 0x564b4dc89cf0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.555+0000 7f822d200640 1 -- 172.21.15.138:0/3943415852 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8230004050 con 0x564b4dc89cf0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.555+0000 7f822d200640 1 -- 172.21.15.138:0/3943415852 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8230005240 con 0x564b4dc89cf0 2024-08-20T01:14:52.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- 172.21.15.138:0/3943415852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 msgr2=0x564b4dc8a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8230009fd0 tx=0x7f82300118d0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- 172.21.15.138:0/3943415852 shutdown_connections 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc98990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc96060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 --2- 172.21.15.138:0/3943415852 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dc8a0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- 172.21.15.138:0/3943415852 >> 172.21.15.138:0/3943415852 conn(0x564b4dc87b20 msgr2=0x564b4dc88cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- 172.21.15.138:0/3943415852 shutdown_connections 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- 172.21.15.138:0/3943415852 wait complete. 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.556+0000 7f82413fd740 1 -- start start 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f8234c00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 0x7f821c0b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 Processor -- start 2024-08-20T01:14:52.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f8234c00640 1 -- 172.21.15.138:0/1454202497 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82240e45e0 con 0x7f823810a6b0 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f823ca00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 0x7f821c0b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc95df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc965a0 0x564b4dc937c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564b4db75ff0 con 0x564b4dc965a0 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x564b4dbf32b0 con 0x564b4dc89cf0 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x564b4da51bb0 con 0x564b4dc8a610 2024-08-20T01:14:52.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.557+0000 7f82413fd740 10 client.0 _ll_register_callbacks cb 0x564b4d8fad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38848/0 (socket says 172.21.15.138:38848) 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 -- 172.21.15.138:0/1239003479 learned_addr learned my addr 172.21.15.138:0/1239003479 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f823ca00640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc965a0 0x564b4dc937c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f823ca00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 0x7f821c0b61c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f823001b040 tx=0x7f8230005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 -- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 msgr2=0x564b4dc95df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc95df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 -- 172.21.15.138:0/1239003479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc965a0 msgr2=0x564b4dc937c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc965a0 0x564b4dc937c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 -- 172.21.15.138:0/1239003479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4d94eb00 con 0x564b4dc8a610 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8237400640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82240aab40 tx=0x7f8224003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcef0d1e70d3354e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8221600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:14:52.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.558+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240c5060 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8230004050 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f82240a70d0 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82240cc050 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 10 client.26338 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b4d9c5960 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8230004050 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f82240e41e0 con 0x564b4dc8a610 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 10 client.26338 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 10 client.26338 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 20 client.26338 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dca5c90 0x564b4dca8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 4 client.26338 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:14:52.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.559+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b4dc01cc0 con 0x564b4dca5c90 2024-08-20T01:14:52.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.560+0000 7f8237e00640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dca5c90 0x564b4dca8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.560+0000 7f8237e00640 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dca5c90 0x564b4dca8080 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc8e4665cf3d191 server_cookie=9be1444dd51413fc in_seq=0 out_seq=0 2024-08-20T01:14:52.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.560+0000 7f8221600640 10 client.26338 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.561+0000 7f8221600640 1 -- 172.21.15.138:0/1239003479 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564b4dc01cc0 con 0x564b4dca5c90 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.561+0000 7f8221600640 10 client.26338 handle_command_reply: tid=0 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.561+0000 7f82413fd740 1 client.26338 shutdown 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.561+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 msgr2=0x564b4dc93260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f82240aab40 tx=0x7f8224003cf0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 shutdown_connections 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dca5c90 0x564b4dca8080 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc965a0 0x564b4dc937c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc8a610 0x564b4dc93260 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f8221600640 0 client.26338 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 --2- 172.21.15.138:0/1239003479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc95df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 >> 172.21.15.138:0/1239003479 conn(0x564b4dc87b20 msgr2=0x564b4dc8d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 shutdown_connections 2024-08-20T01:14:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 1 -- 172.21.15.138:0/1239003479 wait complete. 2024-08-20T01:14:52.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.562+0000 7f82413fd740 20 client.26338 trim_cache size 0 max 16384 2024-08-20T01:14:52.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 Processor -- start 2024-08-20T01:14:52.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 -- start start 2024-08-20T01:14:52.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dca21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc92600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82381c11a0 con 0x564b4dc8a610 2024-08-20T01:14:52.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82381bea40 con 0x564b4dc965a0 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f82413fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82381bc890 con 0x564b4dc89cf0 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f8237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f8237e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dca21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f8237400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56726/0 (socket says 172.21.15.138:56726) 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f8237400640 1 -- 172.21.15.138:0/2994684983 learned_addr learned my addr 172.21.15.138:0/2994684983 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.580+0000 7f823ca00640 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc92600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 -- 172.21.15.138:0/2994684983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 msgr2=0x564b4dca21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dca21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 -- 172.21.15.138:0/2994684983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 msgr2=0x564b4dc92600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc92600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 -- 172.21.15.138:0/2994684983 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b4dc8a440 con 0x564b4dc8a610 2024-08-20T01:14:52.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8237400640 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82240a6e70 tx=0x7f82240aa6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1f0cf1909c0e00 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8223e00640 1 -- 172.21.15.138:0/2994684983 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8224088080 con 0x564b4dc8a610 2024-08-20T01:14:52.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8223e00640 1 -- 172.21.15.138:0/2994684983 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82240af050 con 0x564b4dc8a610 2024-08-20T01:14:52.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.581+0000 7f8223e00640 1 -- 172.21.15.138:0/2994684983 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82240ad070 con 0x564b4dc8a610 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- 172.21.15.138:0/2994684983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 msgr2=0x564b4dc920c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f82240a6e70 tx=0x7f82240aa6e0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- 172.21.15.138:0/2994684983 shutdown_connections 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc965a0 0x564b4dc92600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc920c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 --2- 172.21.15.138:0/2994684983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc89cf0 0x564b4dca21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- 172.21.15.138:0/2994684983 >> 172.21.15.138:0/2994684983 conn(0x564b4dc87b20 msgr2=0x564b4dc919b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- 172.21.15.138:0/2994684983 shutdown_connections 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- 172.21.15.138:0/2994684983 wait complete. 2024-08-20T01:14:52.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.582+0000 7f82413fd740 1 -- start start 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 Processor -- start 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc90950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc965a0 0x564b4dc90e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82381bcf00 con 0x564b4dc8a610 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82381be370 con 0x564b4dc89cf0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82381bd5d0 con 0x564b4dc965a0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f82413fd740 10 client.0 _ll_register_callbacks cb 0x7f8224003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f8237e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f823ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc965a0 0x564b4dc90e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f8237e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:40328/0 (socket says 172.21.15.138:40328) 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.583+0000 7f8237e00640 1 -- 172.21.15.138:0/923248998 learned_addr learned my addr 172.21.15.138:0/923248998 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237400640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc90950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 -- 172.21.15.138:0/923248998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc965a0 msgr2=0x564b4dc90e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc965a0 0x564b4dc90e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 -- 172.21.15.138:0/923248998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 msgr2=0x564b4dc90950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc90950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 -- 172.21.15.138:0/923248998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b4dd0fa10 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8237e00640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8228005ab0 tx=0x7f8228005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3e70522186a9724 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8221600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228035040 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f8228004780 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.584+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82240af050 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8228056050 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 10 client.26375 fetch_fsmap learned FSMap version 34 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564b4dd10db0 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82240af050 con 0x564b4dc89cf0 2024-08-20T01:14:52.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f8228046820 con 0x564b4dc89cf0 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 10 client.26375 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 10 client.26375 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 20 client.26375 populate_metadata read hostname 'smithi138' 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dd11570 0x564b4dd13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 4 client.26375 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "12f7d6", "release": true, "await": true}] 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.585+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "12f7d6", "release": true, "await": true}) -- 0x564b4dbfd760 con 0x564b4dd11570 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.586+0000 7f8237400640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dd11570 0x564b4dd13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.586+0000 7f8237400640 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dd11570 0x564b4dd13990 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d2f73cb17fab60b server_cookie=57e4ccdf7cfaf7dc in_seq=0 out_seq=0 2024-08-20T01:14:52.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.586+0000 7f8221600640 10 client.26375 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:52.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f8221600640 1 -- 172.21.15.138:0/923248998 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x564b4dbfd760 con 0x564b4dd11570 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f8221600640 10 client.26375 handle_command_reply: tid=0 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 client.26375 shutdown 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 msgr2=0x564b4dc90410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8228005ab0 tx=0x7f8228005270 comp rx=0 tx=0).stop 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 shutdown_connections 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x564b4dd11570 0x564b4dd13990 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564b4dc965a0 0x564b4dc90e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564b4dc8a610 0x564b4dc90950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f8221600640 0 client.26375 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 --2- 172.21.15.138:0/923248998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b4dc89cf0 0x564b4dc90410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 >> 172.21.15.138:0/923248998 conn(0x564b4dc87b20 msgr2=0x564b4dc86420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 shutdown_connections 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 1 -- 172.21.15.138:0/923248998 wait complete. 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.591+0000 7f82413fd740 20 client.26375 trim_cache size 0 max 16384 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 34, 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 32, 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:14:52.592 INFO:teuthology.orchestra.run.smithi138.stdout: "12f7d6": { 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 32, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 150.0, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:14:52.593 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 msgr2=0x7f821c0b61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 0x7f821c0b61c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f823001b040 tx=0x7f8230005fd0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 msgr2=0x7f823810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f822400ea60 tx=0x7f822400ef20 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 shutdown_connections 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f821c0b3d00 0x7f821c0b61c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f823810b030 0x7f823810ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f823810a6b0 0x7f823810e7e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 --2- 172.21.15.138:0/1454202497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8238104ca0 0x7f8238111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:14:52.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 >> 172.21.15.138:0/1454202497 conn(0x7f8238100730 msgr2=0x7f8238102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:14:52.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 shutdown_connections 2024-08-20T01:14:52.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:14:52.592+0000 7f823ea00640 1 -- 172.21.15.138:0/1454202497 wait complete. 2024-08-20T01:14:53.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 263234 3.69 MB/sec execute 185 sec latency 29461.480 ms 2024-08-20T01:14:53.853 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '12f7d6', total seconds elapsed: 34.4 2024-08-20T01:14:53.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:53 smithi167 ceph-mon[21965]: pgmap v565: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 477 KiB/s wr, 23 op/s 2024-08-20T01:14:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:53 smithi142 ceph-mon[21799]: pgmap v565: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 477 KiB/s wr, 23 op/s 2024-08-20T01:14:54.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:53 smithi138 ceph-mon[18076]: pgmap v565: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 477 KiB/s wr, 23 op/s 2024-08-20T01:14:54.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 264204 3.70 MB/sec execute 186 sec latency 532.415 ms 2024-08-20T01:14:55.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 266408 3.75 MB/sec execute 187 sec latency 607.847 ms 2024-08-20T01:14:55.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:55 smithi167 ceph-mon[21965]: pgmap v566: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 575 KiB/s wr, 38 op/s 2024-08-20T01:14:55.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:55 smithi142 ceph-mon[21799]: pgmap v566: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 575 KiB/s wr, 38 op/s 2024-08-20T01:14:56.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:55 smithi138 ceph-mon[18076]: pgmap v566: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 575 KiB/s wr, 38 op/s 2024-08-20T01:14:56.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 267752 3.76 MB/sec execute 188 sec latency 120.933 ms 2024-08-20T01:14:57.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 267752 3.74 MB/sec execute 189 sec latency 1121.028 ms 2024-08-20T01:14:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:57 smithi167 ceph-mon[21965]: pgmap v567: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 644 KiB/s wr, 44 op/s 2024-08-20T01:14:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:57 smithi142 ceph-mon[21799]: pgmap v567: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 644 KiB/s wr, 44 op/s 2024-08-20T01:14:58.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:57 smithi138 ceph-mon[18076]: pgmap v567: 129 pgs: 129 active+clean; 164 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 644 KiB/s wr, 44 op/s 2024-08-20T01:14:58.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 267752 3.72 MB/sec execute 190 sec latency 2121.113 ms 2024-08-20T01:14:58.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:14:58 smithi167 ceph-mon[21965]: pgmap v568: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 656 KiB/s wr, 63 op/s 2024-08-20T01:14:58.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:14:58 smithi142 ceph-mon[21799]: pgmap v568: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 656 KiB/s wr, 63 op/s 2024-08-20T01:14:59.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:14:58 smithi138 ceph-mon[18076]: pgmap v568: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 656 KiB/s wr, 63 op/s 2024-08-20T01:14:59.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 267752 3.70 MB/sec execute 191 sec latency 3121.161 ms 2024-08-20T01:15:00.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 269985 3.76 MB/sec execute 192 sec latency 3197.993 ms 2024-08-20T01:15:01.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 270632 3.74 MB/sec execute 193 sec latency 61.963 ms 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 -- 172.21.15.138:0/3938212837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01029f0 msgr2=0x7f1cf0102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/3938212837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01029f0 0x7f1cf0102dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009900 tx=0x7f1ce002a410 comp rx=0 tx=0).stop 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 -- 172.21.15.138:0/3938212837 shutdown_connections 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/3938212837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf0108d70 0x7f1cf010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/3938212837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf01083f0 0x7f1cf0108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/3938212837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01029f0 0x7f1cf0102dd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.726+0000 7f1cf5e00640 1 -- 172.21.15.138:0/3938212837 >> 172.21.15.138:0/3938212837 conn(0x7f1cf00fe430 msgr2=0x7f1cf0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.727+0000 7f1cf5e00640 1 -- 172.21.15.138:0/3938212837 shutdown_connections 2024-08-20T01:15:01.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.727+0000 7f1cf5e00640 1 -- 172.21.15.138:0/3938212837 wait complete. 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.727+0000 7f1cf5e00640 1 Processor -- start 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 -- start start 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01029f0 0x7f1cf019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0108d70 0x7f1cf01a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0112a80 con 0x7f1cf01029f0 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0112900 con 0x7f1cf01083f0 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cf5e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf0112c00 con 0x7f1cf0108d70 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cee000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cee000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35632/0 (socket says 172.21.15.138:35632) 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cee000640 1 -- 172.21.15.138:0/2759008873 learned_addr learned my addr 172.21.15.138:0/2759008873 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:01.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1ceea00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01029f0 0x7f1cf019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.728+0000 7f1cef400640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0108d70 0x7f1cf01a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 -- 172.21.15.138:0/2759008873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0108d70 msgr2=0x7f1cf01a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0108d70 0x7f1cf01a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 -- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01029f0 msgr2=0x7f1cf019b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01029f0 0x7f1cf019b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 -- 172.21.15.138:0/2759008873 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf01a1250 con 0x7f1cf01083f0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cee000640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e9f0 tx=0x7f1cdc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cf4a00640 1 -- 172.21.15.138:0/2759008873 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00cd80 con 0x7f1cf01083f0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf01a1540 con 0x7f1cf01083f0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cf4a00640 1 -- 172.21.15.138:0/2759008873 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc0054d0 con 0x7f1cf01083f0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cf4a00640 1 -- 172.21.15.138:0/2759008873 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc010690 con 0x7f1cf01083f0 2024-08-20T01:15:01.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.729+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf01a7c00 con 0x7f1cf01083f0 2024-08-20T01:15:01.731 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.732+0000 7f1cf8ca1740 1 Processor -- start 2024-08-20T01:15:01.732 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 -- start start 2024-08-20T01:15:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe6cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a63fdd6210 con 0x55a63fe6d1e0 2024-08-20T01:15:01.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a63fdc5710 con 0x55a63fe60930 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a63fdc9750 con 0x55a63fe61250 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35636/0 (socket says 172.21.15.138:35636) 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 -- 172.21.15.138:0/1493742702 learned_addr learned my addr 172.21.15.138:0/1493742702 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1ceea00640 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe6cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 -- 172.21.15.138:0/1493742702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 msgr2=0x55a63fe6cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe6cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 -- 172.21.15.138:0/1493742702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 msgr2=0x55a63fe6f5d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6f5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cef400640 1 -- 172.21.15.138:0/1493742702 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63fd91de0 con 0x55a63fe60930 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1ceea00640 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe6cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:01.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.733+0000 7f1cf4a00640 1 -- 172.21.15.138:0/2759008873 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1cdc004e90 con 0x7f1cf01083f0 2024-08-20T01:15:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cef400640 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009f50 tx=0x7f1ce80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f3902a93302f82 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cdbe00640 1 -- 172.21.15.138:0/1493742702 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8004d20 con 0x55a63fe60930 2024-08-20T01:15:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cdbe00640 1 -- 172.21.15.138:0/1493742702 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8004050 con 0x55a63fe60930 2024-08-20T01:15:01.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cdbe00640 1 -- 172.21.15.138:0/1493742702 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8005280 con 0x55a63fe60930 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1493742702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 msgr2=0x55a63fe60d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.734+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009f50 tx=0x7f1ce80118d0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1493742702 shutdown_connections 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe6cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1493742702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe60d10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1493742702 >> 172.21.15.138:0/1493742702 conn(0x55a63fe5e760 msgr2=0x55a63fe5f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1493742702 shutdown_connections 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1493742702 wait complete. 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf4a00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 0x7f1cd40b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf4a00640 1 -- 172.21.15.138:0/2759008873 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1cdc0e4790 con 0x7f1cf01083f0 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.735+0000 7f1cf8ca1740 1 -- start start 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cef400640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 0x7f1cd40b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 Processor -- start 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 0x55a63fe69e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55a63fcfeab0 con 0x55a63fe61250 2024-08-20T01:15:01.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55a63fd6c140 con 0x55a63fe6d1e0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55a63fe6fec0 con 0x55a63fe60930 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cf8ca1740 10 client.0 _ll_register_callbacks cb 0x55a63fd7f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33314/0 (socket says 172.21.15.138:33314) 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1ceea00640 1 -- 172.21.15.138:0/2383661463 learned_addr learned my addr 172.21.15.138:0/2383661463 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.736+0000 7f1cef400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cef400640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 0x7f1cd40b6110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d60 tx=0x7f1ce8000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cee000640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 0x55a63fe69e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 -- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe6d1e0 msgr2=0x55a63fe6a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 -- 172.21.15.138:0/2383661463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 msgr2=0x55a63fe69e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 0x55a63fe69e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 -- 172.21.15.138:0/2383661463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63fd6d170 con 0x55a63fe60930 2024-08-20T01:15:01.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cee000640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 0x55a63fe69e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1ceea00640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009230 tx=0x7f1ce0004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8d4e2aa5d898f91 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0035040 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1ce0005510 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.737+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce8004050 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ce004f050 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 10 client.26356 fetch_fsmap learned FSMap version 34 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a63fb28790 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce8004050 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1ce0003a00 con 0x55a63fe60930 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 10 client.26356 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 20 client.26356 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:01.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fe7cde0 0x55a63fe7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 4 client.26356 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:15:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a63fdd8160 con 0x55a63fe7cde0 2024-08-20T01:15:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.738+0000 7f1cee000640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fe7cde0 0x55a63fe7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.739+0000 7f1cee000640 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fe7cde0 0x55a63fe7f280 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519c10c373b23d52 server_cookie=203bfde3844372a0 in_seq=0 out_seq=0 2024-08-20T01:15:01.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.739+0000 7f1cd8c00640 10 client.26356 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.739+0000 7f1cd8c00640 1 -- 172.21.15.138:0/2383661463 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a63fdd8160 con 0x55a63fe7cde0 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cd8c00640 10 client.26356 handle_command_reply: tid=0 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 client.26356 shutdown 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 msgr2=0x55a63fe6ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009230 tx=0x7f1ce0004440 comp rx=0 tx=0).stop 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 shutdown_connections 2024-08-20T01:15:01.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fe7cde0 0x55a63fe7f280 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe6d1e0 0x55a63fe6a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe61250 0x55a63fe69e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/2383661463 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe60930 0x55a63fe6ca10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 >> 172.21.15.138:0/2383661463 conn(0x55a63fe5e760 msgr2=0x55a63fe64290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cd8c00640 0 client.26356 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 shutdown_connections 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/2383661463 wait complete. 2024-08-20T01:15:01.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.740+0000 7f1cf8ca1740 20 client.26356 trim_cache size 0 max 16384 2024-08-20T01:15:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 Processor -- start 2024-08-20T01:15:01.756 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 -- start start 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe5fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe84b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01b4570 con 0x55a63fe6d1e0 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01b1e10 con 0x55a63fe60930 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.756+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01afc60 con 0x55a63fe61250 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35664/0 (socket says 172.21.15.138:35664) 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 -- 172.21.15.138:0/376936568 learned_addr learned my addr 172.21.15.138:0/376936568 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1cef400640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe84b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1cee000640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe5fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 -- 172.21.15.138:0/376936568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 msgr2=0x55a63fe5fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe5fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 -- 172.21.15.138:0/376936568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 msgr2=0x55a63fe84b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe84b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 -- 172.21.15.138:0/376936568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a63fba1620 con 0x55a63fe60930 2024-08-20T01:15:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1cee000640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:15:01.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.757+0000 7f1ceea00640 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0006010 tx=0x7f1ce0031630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41c34f3a7400ded1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.758+0000 7f1cdb400640 1 -- 172.21.15.138:0/376936568 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0058070 con 0x55a63fe60930 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.758+0000 7f1cdb400640 1 -- 172.21.15.138:0/376936568 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce0041070 con 0x55a63fe60930 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.758+0000 7f1cdb400640 1 -- 172.21.15.138:0/376936568 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0035030 con 0x55a63fe60930 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.758+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/376936568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 msgr2=0x55a63fe5f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.758+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0006010 tx=0x7f1ce0031630 comp rx=0 tx=0).stop 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/376936568 shutdown_connections 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fe84b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe5fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/376936568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe5f870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/376936568 >> 172.21.15.138:0/376936568 conn(0x55a63fe5e760 msgr2=0x55a63fe69340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/376936568 shutdown_connections 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/376936568 wait complete. 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 -- start start 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.759+0000 7f1cf8ca1740 1 Processor -- start 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe70a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fee5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01b02d0 con 0x55a63fe6d1e0 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01b1740 con 0x55a63fe60930 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf01b09a0 con 0x55a63fe61250 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cf8ca1740 10 client.0 _ll_register_callbacks cb 0x7f1ce0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:01.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35668/0 (socket says 172.21.15.138:35668) 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 -- 172.21.15.138:0/1471696086 learned_addr learned my addr 172.21.15.138:0/1471696086 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cef400640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fee5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cee000640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe70a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 -- 172.21.15.138:0/1471696086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 msgr2=0x55a63fe70a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe70a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 -- 172.21.15.138:0/1471696086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 msgr2=0x55a63fee5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fee5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 -- 172.21.15.138:0/1471696086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a63fee6050 con 0x55a63fe60930 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cee000640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe70a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1cef400640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fee5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.760+0000 7f1ceea00640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0056040 tx=0x7f1ce0005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bdd3dc2e48f53a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:15:01.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0058040 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce0041070 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 9365+0+0 (secure 0 0 0) 0x7f1ce00035f0 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ce0004420 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cf8ca1740 10 client.26393 fetch_fsmap learned FSMap version 34 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55a63fee7310 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ce0041070 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.761+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 34) ==== 8366+0+0 (secure 0 0 0) 0x7f1ce005b070 con 0x55a63fe60930 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 10 client.26393 fetch_fsmap finished waiting for FSMap version 34 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 10 client.26393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 20 client.26393 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fee77d0 0x55a63fee9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 4 client.26393 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a63fdd3c00 con 0x55a63fee77d0 2024-08-20T01:15:01.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cee000640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fee77d0 0x55a63fee9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cee000640 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fee77d0 0x55a63fee9bf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5367914f9b4d0ba1 server_cookie=dcde29918780eb9a in_seq=0 out_seq=0 2024-08-20T01:15:01.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.762+0000 7f1cd8c00640 10 client.26393 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cd8c00640 1 -- 172.21.15.138:0/1471696086 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a63fdd3c00 con 0x55a63fee77d0 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cd8c00640 10 client.26393 handle_command_reply: tid=0 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 client.26393 shutdown 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 msgr2=0x55a63fe70550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0056040 tx=0x7f1ce0005540 comp rx=0 tx=0).stop 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 shutdown_connections 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55a63fee77d0 0x55a63fee9bf0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55a63fe6d1e0 0x55a63fee5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cd8c00640 0 client.26393 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:01.795 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55a63fe61250 0x55a63fe70a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 --2- 172.21.15.138:0/1471696086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a63fe60930 0x55a63fe70550 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.793+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 >> 172.21.15.138:0/1471696086 conn(0x55a63fe5e760 msgr2=0x55a63fe5f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 shutdown_connections 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf8ca1740 1 -- 172.21.15.138:0/1471696086 wait complete. 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf8ca1740 20 client.26393 trim_cache size 0 max 16384 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:15:01.796 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 msgr2=0x7f1cd40b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 0x7f1cd40b6110 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8009d60 tx=0x7f1ce8000f40 comp rx=0 tx=0).stop 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 msgr2=0x7f1cf019b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.794+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00e9f0 tx=0x7f1cdc00edf0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 shutdown_connections 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f1cd40b3c50 0x7f1cd40b6110 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1cf0108d70 0x7f1cf01a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.797 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cf01083f0 0x7f1cf019b6f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 --2- 172.21.15.138:0/2759008873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1cf01029f0 0x7f1cf019b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 >> 172.21.15.138:0/2759008873 conn(0x7f1cf00fe430 msgr2=0x7f1cf01049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 shutdown_connections 2024-08-20T01:15:01.798 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:01.795+0000 7f1cf5e00640 1 -- 172.21.15.138:0/2759008873 wait complete. 2024-08-20T01:15:01.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:01 smithi167 ceph-mon[21965]: pgmap v569: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 650 KiB/s wr, 80 op/s 2024-08-20T01:15:01.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:01 smithi142 ceph-mon[21799]: pgmap v569: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 650 KiB/s wr, 80 op/s 2024-08-20T01:15:02.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:01 smithi138 ceph-mon[18076]: pgmap v569: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 650 KiB/s wr, 80 op/s 2024-08-20T01:15:02.157 INFO:tasks.workunit.client.0.smithi138.stdout: 1 271294 3.74 MB/sec execute 194 sec latency 358.388 ms 2024-08-20T01:15:02.708 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:15:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: ignoring --setuser ceph since I am not root 2024-08-20T01:15:02.708 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:15:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: ignoring --setgroup ceph since I am not root 2024-08-20T01:15:02.708 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:15:02 smithi167 ceph-mds[53569]: -- 172.21.15.167:0/1351615294 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563ffcfbd860 con 0x563ffcf9a800 2024-08-20T01:15:02.708 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:15:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: starting mds.i at 2024-08-20T01:15:02.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:02 smithi167 ceph-mon[21965]: daemon mds.i restarted 2024-08-20T01:15:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:02 smithi142 ceph-mon[21799]: daemon mds.i restarted 2024-08-20T01:15:03.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:02 smithi138 ceph-mon[18076]: daemon mds.i restarted 2024-08-20T01:15:03.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 271294 3.72 MB/sec execute 195 sec latency 1358.541 ms 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: pgmap v570: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 676 KiB/s wr, 82 op/s 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: osdmap e89: 12 total, 12 up, 12 in 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6838/2422640849,v1:172.21.15.167:6839/2422640849] up:boot 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-08-20T01:15:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:03 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: pgmap v570: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 676 KiB/s wr, 82 op/s 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: osdmap e89: 12 total, 12 up, 12 in 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6838/2422640849,v1:172.21.15.167:6839/2422640849] up:boot 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-08-20T01:15:04.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:03 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:04.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 271294 3.71 MB/sec execute 196 sec latency 2358.679 ms 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: pgmap v570: 129 pgs: 129 active+clean; 169 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 676 KiB/s wr, 82 op/s 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: osdmap e89: 12 total, 12 up, 12 in 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6838/2422640849,v1:172.21.15.167:6839/2422640849] up:boot 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-08-20T01:15:04.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:03 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:04 smithi142 ceph-mon[21799]: osdmap e90: 12 total, 12 up, 12 in 2024-08-20T01:15:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:04 smithi142 ceph-mon[21799]: pgmap v573: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1001 KiB/s wr, 152 op/s 2024-08-20T01:15:05.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:04 smithi138 ceph-mon[18076]: osdmap e90: 12 total, 12 up, 12 in 2024-08-20T01:15:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:04 smithi138 ceph-mon[18076]: pgmap v573: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1001 KiB/s wr, 152 op/s 2024-08-20T01:15:05.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 273455 3.76 MB/sec execute 197 sec latency 2431.047 ms 2024-08-20T01:15:05.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:04 smithi167 ceph-mon[21965]: osdmap e90: 12 total, 12 up, 12 in 2024-08-20T01:15:05.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:04 smithi167 ceph-mon[21965]: pgmap v573: 129 pgs: 129 active+clean; 166 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1001 KiB/s wr, 152 op/s 2024-08-20T01:15:06.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 274741 3.76 MB/sec execute 198 sec latency 21.599 ms 2024-08-20T01:15:07.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 274843 3.75 MB/sec execute 199 sec latency 934.411 ms 2024-08-20T01:15:07.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:07 smithi167 ceph-mon[21965]: pgmap v574: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-08-20T01:15:07.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:07 smithi142 ceph-mon[21799]: pgmap v574: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-08-20T01:15:08.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:07 smithi138 ceph-mon[18076]: pgmap v574: 129 pgs: 129 active+clean; 168 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.1 MiB/s wr, 181 op/s 2024-08-20T01:15:08.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 274843 3.73 MB/sec execute 200 sec latency 1934.503 ms 2024-08-20T01:15:09.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:08 smithi138 ceph-mon[18076]: pgmap v575: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 159 op/s 2024-08-20T01:15:09.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 274843 3.71 MB/sec execute 201 sec latency 2934.617 ms 2024-08-20T01:15:09.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:08 smithi167 ceph-mon[21965]: pgmap v575: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 159 op/s 2024-08-20T01:15:09.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:08 smithi142 ceph-mon[21799]: pgmap v575: 129 pgs: 129 active+clean; 168 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 975 KiB/s wr, 159 op/s 2024-08-20T01:15:10.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 276989 3.76 MB/sec execute 202 sec latency 3007.828 ms 2024-08-20T01:15:10.595 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:10 smithi138 ceph-mon[18076]: osdmap e91: 12 total, 12 up, 12 in 2024-08-20T01:15:10.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:10 smithi167 ceph-mon[21965]: osdmap e91: 12 total, 12 up, 12 in 2024-08-20T01:15:10.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:10 smithi142 ceph-mon[21799]: osdmap e91: 12 total, 12 up, 12 in 2024-08-20T01:15:11.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 278264 3.77 MB/sec execute 203 sec latency 27.500 ms 2024-08-20T01:15:11.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:11 smithi138 ceph-mon[18076]: pgmap v577: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 953 KiB/s wr, 164 op/s 2024-08-20T01:15:11.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:11 smithi167 ceph-mon[21965]: pgmap v577: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 953 KiB/s wr, 164 op/s 2024-08-20T01:15:11.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:11 smithi142 ceph-mon[21799]: pgmap v577: 129 pgs: 129 active+clean; 170 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 953 KiB/s wr, 164 op/s 2024-08-20T01:15:12.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 278385 3.75 MB/sec execute 204 sec latency 908.931 ms 2024-08-20T01:15:12.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:12 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:12.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:12 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:12 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:13.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 278385 3.73 MB/sec execute 205 sec latency 1909.048 ms 2024-08-20T01:15:13.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:13 smithi167 ceph-mon[21965]: pgmap v578: 129 pgs: 129 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-08-20T01:15:13.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:13 smithi142 ceph-mon[21799]: pgmap v578: 129 pgs: 129 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-08-20T01:15:13.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:13 smithi138 ceph-mon[18076]: pgmap v578: 129 pgs: 129 active+clean; 171 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 163 op/s 2024-08-20T01:15:14.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 278385 3.71 MB/sec execute 206 sec latency 2909.207 ms 2024-08-20T01:15:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 -- 172.21.15.138:0/310123905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 msgr2=0x7f593410ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.927 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 --2- 172.21.15.138:0/310123905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593410ca10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5920009900 tx=0x7f592002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 -- 172.21.15.138:0/310123905 shutdown_connections 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 --2- 172.21.15.138:0/310123905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f593410cf50 0x7f593410f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 --2- 172.21.15.138:0/310123905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593410ca10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 --2- 172.21.15.138:0/310123905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5934101520 0x7f5934101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 -- 172.21.15.138:0/310123905 >> 172.21.15.138:0/310123905 conn(0x7f5934076a70 msgr2=0x7f5934076e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 -- 172.21.15.138:0/310123905 shutdown_connections 2024-08-20T01:15:14.928 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.929+0000 7f593a200640 1 -- 172.21.15.138:0/310123905 wait complete. 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.930+0000 7f593a200640 1 Processor -- start 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.930+0000 7f593a200640 1 -- start start 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5934101520 0x7f593419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f593410cf50 0x7f59341a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5934112ae0 con 0x7f5934101520 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5934112960 con 0x7f593410a620 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f593a200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5934112c60 con 0x7f593410cf50 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48516/0 (socket says 172.21.15.138:48516) 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 -- 172.21.15.138:0/4012937458 learned_addr learned my addr 172.21.15.138:0/4012937458 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:14.930 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5933400640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5934101520 0x7f593419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5933e00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f593410cf50 0x7f59341a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 -- 172.21.15.138:0/4012937458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f593410cf50 msgr2=0x7f59341a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f593410cf50 0x7f59341a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 -- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5934101520 msgr2=0x7f593419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5934101520 0x7f593419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.931+0000 7f5932a00640 1 -- 172.21.15.138:0/4012937458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59341a12f0 con 0x7f593410a620 2024-08-20T01:15:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f5932a00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5920005ab0 tx=0x7f5920003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f592be00640 1 -- 172.21.15.138:0/4012937458 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592003f070 con 0x7f593410a620 2024-08-20T01:15:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f592be00640 1 -- 172.21.15.138:0/4012937458 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920031b50 con 0x7f593410a620 2024-08-20T01:15:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59341a1580 con 0x7f593410a620 2024-08-20T01:15:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f592be00640 1 -- 172.21.15.138:0/4012937458 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920035030 con 0x7f593410a620 2024-08-20T01:15:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.932+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341a7ca0 con 0x7f593410a620 2024-08-20T01:15:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 Processor -- start 2024-08-20T01:15:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 -- start start 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd559d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd565ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd5685d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd4cf210 con 0x55cbdd559930 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd4be710 con 0x55cbdd5661e0 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f593d1bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd4c2750 con 0x55cbdd55a250 2024-08-20T01:15:14.935 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.936+0000 7f592be00640 1 -- 172.21.15.138:0/4012937458 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5920042070 con 0x7f593410a620 2024-08-20T01:15:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5933e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd559d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5933e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd559d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59272/0 (socket says 172.21.15.138:59272) 2024-08-20T01:15:14.936 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5933e00640 1 -- 172.21.15.138:0/4176495530 learned_addr learned my addr 172.21.15.138:0/4176495530 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd5685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 -- 172.21.15.138:0/4176495530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 msgr2=0x55cbdd565ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd565ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 -- 172.21.15.138:0/4176495530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 msgr2=0x55cbdd559d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd559d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.937+0000 7f5932a00640 1 -- 172.21.15.138:0/4176495530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbdd48ade0 con 0x55cbdd5661e0 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f5932a00640 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd5685d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5920101de0 tx=0x7f5920108c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e25a96f1cad45e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f591fe00640 1 -- 172.21.15.138:0/4176495530 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592010b070 con 0x55cbdd5661e0 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f591fe00640 1 -- 172.21.15.138:0/4176495530 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920112410 con 0x55cbdd5661e0 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f591fe00640 1 -- 172.21.15.138:0/4176495530 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920111610 con 0x55cbdd5661e0 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f592be00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 0x7f59180b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.938+0000 7f592be00640 1 -- 172.21.15.138:0/4012937458 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f5920104a30 con 0x7f593410a620 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f5933e00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 0x7f59180b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 -- 172.21.15.138:0/4176495530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 msgr2=0x55cbdd5685d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd5685d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5920101de0 tx=0x7f5920108c40 comp rx=0 tx=0).stop 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 -- 172.21.15.138:0/4176495530 shutdown_connections 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd5685d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd565ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 --2- 172.21.15.138:0/4176495530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd559d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 -- 172.21.15.138:0/4176495530 >> 172.21.15.138:0/4176495530 conn(0x55cbdd557760 msgr2=0x55cbdd558900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 -- 172.21.15.138:0/4176495530 shutdown_connections 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f5933e00640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 0x7f59180b6200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f592c005f40 tx=0x7f592c0096c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.939+0000 7f593d1bc740 1 -- 172.21.15.138:0/4176495530 wait complete. 2024-08-20T01:15:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 -- start start 2024-08-20T01:15:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 Processor -- start 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd565ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd562f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd3f7ab0 con 0x55cbdd559930 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd465140 con 0x55cbdd5661e0 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.940+0000 7f593d1bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cbdd568c80 con 0x55cbdd55a250 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f593d1bc740 10 client.0 _ll_register_callbacks cb 0x55cbdd4785d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48530/0 (socket says 172.21.15.138:48530) 2024-08-20T01:15:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd565ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 -- 172.21.15.138:0/2700860612 learned_addr learned my addr 172.21.15.138:0/2700860612 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5932a00640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd562f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 -- 172.21.15.138:0/2700860612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 msgr2=0x55cbdd562f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd562f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 -- 172.21.15.138:0/2700860612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 msgr2=0x55cbdd565ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd565ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 -- 172.21.15.138:0/2700860612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbdd466170 con 0x55cbdd5661e0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f5933e00640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f592c0049d0 tx=0x7f592c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7885c4c74a5ebc0a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.941+0000 7f591cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c02c060 con 0x55cbdd5661e0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5920112410 con 0x55cbdd5661e0 2024-08-20T01:15:14.945 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f592c017030 con 0x55cbdd5661e0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f592c033050 con 0x55cbdd5661e0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f593d1bc740 10 client.26411 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbdd221790 con 0x55cbdd5661e0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.942+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5920112410 con 0x55cbdd5661e0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f592c027620 con 0x55cbdd5661e0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 10 client.26411 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 10 client.26411 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 20 client.26411 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd575c70 0x55cbdd578060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 4 client.26411 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbdd4d1160 con 0x55cbdd575c70 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.943+0000 7f5933400640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd575c70 0x55cbdd578060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.946 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.944+0000 7f5933400640 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd575c70 0x55cbdd578060 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48e50d1a47eb68e2 server_cookie=dbb6da461492eca in_seq=0 out_seq=0 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.944+0000 7f591cc00640 10 client.26411 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.944+0000 7f591cc00640 1 -- 172.21.15.138:0/2700860612 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbdd4d1160 con 0x55cbdd575c70 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.944+0000 7f591cc00640 10 client.26411 handle_command_reply: tid=0 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.944+0000 7f593d1bc740 1 client.26411 shutdown 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 msgr2=0x55cbdd563540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f592c0049d0 tx=0x7f592c004ab0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.947 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 shutdown_connections 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd575c70 0x55cbdd578060 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd5661e0 0x55cbdd563540 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd55a250 0x55cbdd562f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f591cc00640 0 client.26411 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2700860612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd559930 0x55cbdd565ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 >> 172.21.15.138:0/2700860612 conn(0x55cbdd557760 msgr2=0x55cbdd55d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 shutdown_connections 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 1 -- 172.21.15.138:0/2700860612 wait complete. 2024-08-20T01:15:14.948 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.945+0000 7f593d1bc740 20 client.26411 trim_cache size 0 max 16384 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.961+0000 7f593d1bc740 1 Processor -- start 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.961+0000 7f593d1bc740 1 -- start start 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd559930 0x55cbdd556650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd5661e0 0x55cbdd57deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b4700 con 0x55cbdd5661e0 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b1f70 con 0x55cbdd55a250 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f593d1bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59341afde0 con 0x55cbdd559930 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5933e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd5661e0 0x55cbdd57deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48534/0 (socket says 172.21.15.138:48534) 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 -- 172.21.15.138:0/3580539560 learned_addr learned my addr 172.21.15.138:0/3580539560 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5933400640 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd559930 0x55cbdd556650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 -- 172.21.15.138:0/3580539560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd559930 msgr2=0x55cbdd556650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd559930 0x55cbdd556650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 -- 172.21.15.138:0/3580539560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd5661e0 msgr2=0x55cbdd57deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.961 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd5661e0 0x55cbdd57deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.962 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 -- 172.21.15.138:0/3580539560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbdd29a620 con 0x55cbdd55a250 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.962+0000 7f5932a00640 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59200cb2e0 tx=0x7f592002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c1ff3b39af3038a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f591f400640 1 -- 172.21.15.138:0/3580539560 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592010b070 con 0x55cbdd55a250 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f591f400640 1 -- 172.21.15.138:0/3580539560 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59200cae60 con 0x55cbdd55a250 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f591f400640 1 -- 172.21.15.138:0/3580539560 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5920049070 con 0x55cbdd55a250 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f593d1bc740 1 -- 172.21.15.138:0/3580539560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 msgr2=0x55cbdd57d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f593d1bc740 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f59200cb2e0 tx=0x7f592002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.963+0000 7f593d1bc740 1 -- 172.21.15.138:0/3580539560 shutdown_connections 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd5661e0 0x55cbdd57deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd55a250 0x55cbdd57d970 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.965 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 --2- 172.21.15.138:0/3580539560 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd559930 0x55cbdd556650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 -- 172.21.15.138:0/3580539560 >> 172.21.15.138:0/3580539560 conn(0x55cbdd557760 msgr2=0x55cbdd555f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 -- 172.21.15.138:0/3580539560 shutdown_connections 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 -- 172.21.15.138:0/3580539560 wait complete. 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 -- start start 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.964+0000 7f593d1bc740 1 Processor -- start 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd55a250 0x55cbdd56bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 0x55cbdd5de940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b0430 con 0x55cbdd55a250 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b18a0 con 0x55cbdd559930 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b0b00 con 0x55cbdd5661e0 2024-08-20T01:15:14.966 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f593d1bc740 10 client.0 _ll_register_callbacks cb 0x7f592c004860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5932a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd55a250 0x55cbdd56bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48536/0 (socket says 172.21.15.138:48536) 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 -- 172.21.15.138:0/2860943374 learned_addr learned my addr 172.21.15.138:0/2860943374 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933e00640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 0x55cbdd5de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 -- 172.21.15.138:0/2860943374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 msgr2=0x55cbdd5de940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 0x55cbdd5de940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 -- 172.21.15.138:0/2860943374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd55a250 msgr2=0x55cbdd56bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd55a250 0x55cbdd56bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933400640 1 -- 172.21.15.138:0/2860943374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbdd5df050 con 0x55cbdd559930 2024-08-20T01:15:14.967 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.965+0000 7f5933e00640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 0x55cbdd5de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f5933400640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55cbdd578910 tx=0x7f59240044d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aeba180dbc1c9cd1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f591cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592401d380 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f592401d520 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59200cae60 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f592402d050 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f593d1bc740 10 client.26417 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cbdd5e03c0 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.966+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59200cae60 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f5924010b30 con 0x55cbdd559930 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 10 client.26417 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:14.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 20 client.26417 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd5e0b80 0x55cbdd5e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 4 client.26417 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbdd4ccc00 con 0x55cbdd5e0b80 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f5932a00640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd5e0b80 0x55cbdd5e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f5932a00640 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd5e0b80 0x55cbdd5e2fa0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e51366dd25428ffe server_cookie=9cde1395fc39d7cf in_seq=0 out_seq=0 2024-08-20T01:15:14.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.967+0000 7f591cc00640 10 client.26417 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f591cc00640 1 -- 172.21.15.138:0/2860943374 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cbdd4ccc00 con 0x55cbdd5e0b80 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f591cc00640 10 client.26417 handle_command_reply: tid=0 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 client.26417 shutdown 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 msgr2=0x55cbdd56b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x55cbdd578910 tx=0x7f59240044d0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 shutdown_connections 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cbdd5e0b80 0x55cbdd5e2fa0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cbdd5661e0 0x55cbdd5de940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cbdd55a250 0x55cbdd56bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 --2- 172.21.15.138:0/2860943374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cbdd559930 0x55cbdd56b5c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f591cc00640 0 client.26417 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 >> 172.21.15.138:0/2860943374 conn(0x55cbdd557760 msgr2=0x55cbdd556760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 shutdown_connections 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 1 -- 172.21.15.138:0/2860943374 wait complete. 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:14.999+0000 7f593d1bc740 20 client.26417 trim_cache size 0 max 16384 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:15:14.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 msgr2=0x7f59180b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 0x7f59180b6200 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f592c005f40 tx=0x7f592c0096c0 comp rx=0 tx=0).stop 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 msgr2=0x7f593419b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5920005ab0 tx=0x7f5920003b90 comp rx=0 tx=0).stop 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 shutdown_connections 2024-08-20T01:15:15.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59180b3d40 0x7f59180b6200 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f593410cf50 0x7f59341a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f593410a620 0x7f593419b790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 --2- 172.21.15.138:0/4012937458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5934101520 0x7f593419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 >> 172.21.15.138:0/4012937458 conn(0x7f5934076a70 msgr2=0x7f59341089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 shutdown_connections 2024-08-20T01:15:15.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:15.000+0000 7f593a200640 1 -- 172.21.15.138:0/4012937458 wait complete. 2024-08-20T01:15:15.158 INFO:tasks.workunit.client.0.smithi138.stdout: 1 279071 3.70 MB/sec execute 207 sec latency 2988.631 ms 2024-08-20T01:15:15.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:15 smithi167 ceph-mon[21965]: pgmap v579: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1021 KiB/s wr, 136 op/s 2024-08-20T01:15:15.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:15 smithi142 ceph-mon[21799]: pgmap v579: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1021 KiB/s wr, 136 op/s 2024-08-20T01:15:16.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:15 smithi138 ceph-mon[18076]: pgmap v579: 129 pgs: 129 active+clean; 174 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1021 KiB/s wr, 136 op/s 2024-08-20T01:15:16.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 280931 3.74 MB/sec execute 208 sec latency 22.759 ms 2024-08-20T01:15:17.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:16 smithi138 ceph-mon[18076]: pgmap v580: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 830 KiB/s wr, 97 op/s 2024-08-20T01:15:17.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 281934 3.75 MB/sec execute 209 sec latency 334.112 ms 2024-08-20T01:15:17.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:16 smithi167 ceph-mon[21965]: pgmap v580: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 830 KiB/s wr, 97 op/s 2024-08-20T01:15:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:16 smithi142 ceph-mon[21799]: pgmap v580: 129 pgs: 129 active+clean; 175 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 830 KiB/s wr, 97 op/s 2024-08-20T01:15:18.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 281934 3.73 MB/sec execute 210 sec latency 1334.202 ms 2024-08-20T01:15:19.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 281934 3.71 MB/sec execute 211 sec latency 2334.313 ms 2024-08-20T01:15:19.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:19 smithi167 ceph-mon[21965]: pgmap v581: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 882 KiB/s wr, 98 op/s 2024-08-20T01:15:19.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:19 smithi142 ceph-mon[21799]: pgmap v581: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 882 KiB/s wr, 98 op/s 2024-08-20T01:15:20.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:19 smithi138 ceph-mon[18076]: pgmap v581: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 882 KiB/s wr, 98 op/s 2024-08-20T01:15:20.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 284024 3.76 MB/sec execute 212 sec latency 2407.036 ms 2024-08-20T01:15:20.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:20 smithi142 ceph-mon[21799]: pgmap v582: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-08-20T01:15:21.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:20 smithi138 ceph-mon[18076]: pgmap v582: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-08-20T01:15:21.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 285292 3.77 MB/sec execute 213 sec latency 43.050 ms 2024-08-20T01:15:21.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:20 smithi167 ceph-mon[21965]: pgmap v582: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 1.1 MiB/s wr, 168 op/s 2024-08-20T01:15:22.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 285476 3.75 MB/sec execute 214 sec latency 810.224 ms 2024-08-20T01:15:23.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 287312 3.79 MB/sec execute 215 sec latency 1203.178 ms 2024-08-20T01:15:23.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:23 smithi142 ceph-mon[21799]: pgmap v583: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 921 KiB/s wr, 142 op/s 2024-08-20T01:15:24.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:23 smithi138 ceph-mon[18076]: pgmap v583: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 921 KiB/s wr, 142 op/s 2024-08-20T01:15:24.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 288091 3.78 MB/sec execute 216 sec latency 136.023 ms 2024-08-20T01:15:24.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:23 smithi167 ceph-mon[21965]: pgmap v583: 129 pgs: 129 active+clean; 179 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 921 KiB/s wr, 142 op/s 2024-08-20T01:15:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:24 smithi142 ceph-mon[21799]: pgmap v584: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-08-20T01:15:25.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:24 smithi138 ceph-mon[18076]: pgmap v584: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-08-20T01:15:25.159 INFO:tasks.workunit.client.0.smithi138.stdout: 1 289024 3.79 MB/sec execute 217 sec latency 503.407 ms 2024-08-20T01:15:25.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:24 smithi167 ceph-mon[21965]: pgmap v584: 129 pgs: 129 active+clean; 185 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-08-20T01:15:26.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 289024 3.77 MB/sec execute 218 sec latency 1503.523 ms 2024-08-20T01:15:27.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 289024 3.75 MB/sec execute 219 sec latency 2503.647 ms 2024-08-20T01:15:27.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:27 smithi167 ceph-mon[21965]: pgmap v585: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-08-20T01:15:27.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:27 smithi142 ceph-mon[21799]: pgmap v585: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-08-20T01:15:28.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:27 smithi138 ceph-mon[18076]: pgmap v585: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.2 MiB/s wr, 161 op/s 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.122+0000 7fe5ba600640 1 -- 172.21.15.138:0/3998917610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410a410 msgr2=0x7fe5b410a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.122+0000 7fe5ba600640 1 --2- 172.21.15.138:0/3998917610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410a410 0x7fe5b410a850 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009900 tx=0x7fe5a002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 -- 172.21.15.138:0/3998917610 shutdown_connections 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 --2- 172.21.15.138:0/3998917610 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410ad90 0x7fe5b4111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 --2- 172.21.15.138:0/3998917610 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410a410 0x7fe5b410a850 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 --2- 172.21.15.138:0/3998917610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b4104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 -- 172.21.15.138:0/3998917610 >> 172.21.15.138:0/3998917610 conn(0x7fe5b4100410 msgr2=0x7fe5b4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 -- 172.21.15.138:0/3998917610 shutdown_connections 2024-08-20T01:15:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.123+0000 7fe5ba600640 1 -- 172.21.15.138:0/3998917610 wait complete. 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 Processor -- start 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 -- start start 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410a410 0x7fe5b419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4114a60 con 0x7fe5b410ad90 2024-08-20T01:15:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41148e0 con 0x7fe5b4104a10 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5ba600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b4114be0 con 0x7fe5b410a410 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5b9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46994/0 (socket says 172.21.15.138:46994) 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5b9c00640 1 -- 172.21.15.138:0/1769062001 learned_addr learned my addr 172.21.15.138:0/1769062001 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.124+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410a410 0x7fe5b419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 -- 172.21.15.138:0/1769062001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410a410 msgr2=0x7fe5b419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410a410 0x7fe5b419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 -- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 msgr2=0x7fe5b41a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 -- 172.21.15.138:0/1769062001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b41a1000 con 0x7fe5b4104a10 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b9200640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b419af60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9f0 tx=0x7fe5a800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b1600640 1 -- 172.21.15.138:0/1769062001 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a800cd80 con 0x7fe5b4104a10 2024-08-20T01:15:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b41a12f0 con 0x7fe5b4104a10 2024-08-20T01:15:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b1600640 1 -- 172.21.15.138:0/1769062001 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a80035f0 con 0x7fe5b4104a10 2024-08-20T01:15:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.125+0000 7fe5b1600640 1 -- 172.21.15.138:0/1769062001 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a8010690 con 0x7fe5b4104a10 2024-08-20T01:15:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.126+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b41a7b90 con 0x7fe5b4104a10 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.128+0000 7fe5bebcd740 1 Processor -- start 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.128+0000 7fe5bebcd740 1 -- start start 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec9899e3c0 0x55ec989a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec98907210 con 0x55ec98992430 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec988f6710 con 0x55ec98991b10 2024-08-20T01:15:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec988fa750 con 0x55ec9899e3c0 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5b9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5b9c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:49506/0 (socket says 172.21.15.138:49506) 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5b9c00640 1 -- 172.21.15.138:0/1203863808 learned_addr learned my addr 172.21.15.138:0/1203863808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec9899e3c0 0x55ec989a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.129+0000 7fe5b9c00640 1 -- 172.21.15.138:0/1203863808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec9899e3c0 msgr2=0x55ec989a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec9899e3c0 0x55ec989a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b9c00640 1 -- 172.21.15.138:0/1203863808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 msgr2=0x55ec9899de80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899de80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b9c00640 1 -- 172.21.15.138:0/1203863808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec988c2de0 con 0x55ec98991b10 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b1600640 1 -- 172.21.15.138:0/1769062001 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe5a80037a0 con 0x7fe5b4104a10 2024-08-20T01:15:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0118b0 tx=0x7fe5ac011d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=315ec1e9c2a2122 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5a6000640 1 -- 172.21.15.138:0/1203863808 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac01e070 con 0x55ec98991b10 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5a6000640 1 -- 172.21.15.138:0/1203863808 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac00fe70 con 0x55ec98991b10 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.130+0000 7fe5a6000640 1 -- 172.21.15.138:0/1203863808 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac019d40 con 0x55ec98991b10 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- 172.21.15.138:0/1203863808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 msgr2=0x55ec98991ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac0118b0 tx=0x7fe5ac011d70 comp rx=0 tx=0).stop 2024-08-20T01:15:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- 172.21.15.138:0/1203863808 shutdown_connections 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec9899e3c0 0x55ec989a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/1203863808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec98991b10 0x55ec98991ef0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- 172.21.15.138:0/1203863808 >> 172.21.15.138:0/1203863808 conn(0x55ec9898f940 msgr2=0x55ec98990ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- 172.21.15.138:0/1203863808 shutdown_connections 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- 172.21.15.138:0/1203863808 wait complete. 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5b1600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 0x7fe59c0b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5b1600640 1 -- 172.21.15.138:0/1769062001 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7fe5a80e46c0 con 0x7fe5b4104a10 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 0x7fe59c0b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.131+0000 7fe5bebcd740 1 -- start start 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 Processor -- start 2024-08-20T01:15:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 0x7fe59c0b6110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ab0 tx=0x7fe5a0005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec9899b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ec9882fab0 con 0x55ec98992430 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ec9889d140 con 0x55ec9899e3c0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ec989a10a0 con 0x55ec98991b10 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5bebcd740 10 client.0 _ll_register_callbacks cb 0x55ec988b05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59314/0 (socket says 172.21.15.138:59314) 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b9200640 1 -- 172.21.15.138:0/501528413 learned_addr learned my addr 172.21.15.138:0/501528413 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.132+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec9899b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 -- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 msgr2=0x55ec9899b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec9899b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 -- 172.21.15.138:0/501528413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 msgr2=0x55ec9899af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 -- 172.21.15.138:0/501528413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9889e170 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5b9200640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80e1ad0 tx=0x7fe5a8003cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2787b31dde148c6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe59aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80af2f0 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7fe5a80e1cf0 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5ac00fe70 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe5a8086050 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.133+0000 7fe5bebcd740 10 client.26395 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec98659790 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac00fe70 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a80a71c0 con 0x55ec98991b10 2024-08-20T01:15:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 10 client.26395 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 10 client.26395 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 20 client.26395 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec989adf00 0x55ec989b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 4 client.26395 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec98909160 con 0x55ec989adf00 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.134+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec989adf00 0x55ec989b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.135+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec989adf00 0x55ec989b02f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a758fc42eb985 server_cookie=36abe72444f9396e in_seq=0 out_seq=0 2024-08-20T01:15:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.135+0000 7fe59aa00640 10 client.26395 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe59aa00640 1 -- 172.21.15.138:0/501528413 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ec98909160 con 0x55ec989adf00 2024-08-20T01:15:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe59aa00640 10 client.26395 handle_command_reply: tid=0 2024-08-20T01:15:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 client.26395 shutdown 2024-08-20T01:15:28.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 msgr2=0x55ec9899dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80e1ad0 tx=0x7fe5a8003cf0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 shutdown_connections 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec989adf00 0x55ec989b02f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec9899b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec9899af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/501528413 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec9899dae0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe59aa00640 0 client.26395 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 >> 172.21.15.138:0/501528413 conn(0x55ec9898f940 msgr2=0x55ec989954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 shutdown_connections 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 1 -- 172.21.15.138:0/501528413 wait complete. 2024-08-20T01:15:28.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.136+0000 7fe5bebcd740 20 client.26395 trim_cache size 0 max 16384 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.151+0000 7fe5bebcd740 1 Processor -- start 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.151+0000 7fe5bebcd740 1 -- start start 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98992430 0x55ec989b5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec989b6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b4490 con 0x55ec98991b10 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b1d00 con 0x55ec9899e3c0 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41afb70 con 0x55ec98992430 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47002/0 (socket says 172.21.15.138:47002) 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 -- 172.21.15.138:0/2827810949 learned_addr learned my addr 172.21.15.138:0/2827810949 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec989b6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98992430 0x55ec989b5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 -- 172.21.15.138:0/2827810949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98992430 msgr2=0x55ec989b5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98992430 0x55ec989b5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 -- 172.21.15.138:0/2827810949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 msgr2=0x55ec989b6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec989b6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9200640 1 -- 172.21.15.138:0/2827810949 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec98992260 con 0x55ec98991b10 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.152+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec989b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80a6fb0 tx=0x7fe5a80ab0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae8709098cbceb4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5a5600640 1 -- 172.21.15.138:0/2827810949 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80af2f0 con 0x55ec98991b10 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5a5600640 1 -- 172.21.15.138:0/2827810949 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a80af530 con 0x55ec98991b10 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5a5600640 1 -- 172.21.15.138:0/2827810949 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80cc1e0 con 0x55ec98991b10 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2827810949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 msgr2=0x55ec98991140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe5a80a6fb0 tx=0x7fe5a80ab0b0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2827810949 shutdown_connections 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec989b6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98992430 0x55ec989b5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2827810949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98991b10 0x55ec98991140 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2827810949 >> 172.21.15.138:0/2827810949 conn(0x55ec9898f940 msgr2=0x55ec98990a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.153+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2827810949 shutdown_connections 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2827810949 wait complete. 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 -- start start 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 Processor -- start 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec989a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec98998090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b01c0 con 0x55ec98992430 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b1630 con 0x55ec9899e3c0 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b41b0890 con 0x55ec98991b10 2024-08-20T01:15:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.154+0000 7fe5bebcd740 10 client.0 _ll_register_callbacks cb 0x7fe5a80aa6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47016/0 (socket says 172.21.15.138:47016) 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 -- 172.21.15.138:0/2439761590 learned_addr learned my addr 172.21.15.138:0/2439761590 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec989a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b9c00640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec98998090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 -- 172.21.15.138:0/2439761590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 msgr2=0x55ec989a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec989a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 -- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 msgr2=0x55ec98998090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec98998090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 -- 172.21.15.138:0/2439761590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec986d2620 con 0x55ec98992430 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe5b3e00640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0054fd0 tx=0x7fe5a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b4f58233f388da server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.155+0000 7fe59aa00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0035040 con 0x55ec98992430 2024-08-20T01:15:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7fe5a00035f0 con 0x55ec98992430 2024-08-20T01:15:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a80af530 con 0x55ec98992430 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe5a0066050 con 0x55ec98992430 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 10 client.16377 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ec989a30f0 con 0x55ec98992430 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a80af530 con 0x55ec98992430 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7fe5a0032810 con 0x55ec98992430 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 10 client.16377 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 20 client.16377 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec98a19150 0x55ec98a1b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 4 client.16377 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:15:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.156+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec98904c00 con 0x55ec98a19150 2024-08-20T01:15:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.157+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec98a19150 0x55ec98a1b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.157+0000 7fe5b9200640 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec98a19150 0x55ec98a1b570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b7b8d2f2af67c13 server_cookie=c2201bd68b8c760 in_seq=0 out_seq=0 2024-08-20T01:15:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.157+0000 7fe59aa00640 10 client.16377 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:28.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 289024 3.74 MB/sec execute 220 sec latency 3503.762 ms 2024-08-20T01:15:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.198+0000 7fe59aa00640 1 -- 172.21.15.138:0/2439761590 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec98904c00 con 0x55ec98a19150 2024-08-20T01:15:28.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.198+0000 7fe59aa00640 10 client.16377 handle_command_reply: tid=0 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.198+0000 7fe5bebcd740 1 client.16377 shutdown 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 msgr2=0x55ec98997b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0054fd0 tx=0x7fe5a0009260 comp rx=0 tx=0).stop 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 shutdown_connections 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ec98a19150 0x55ec98a1b570 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec9899e3c0 0x55ec98998090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ec98992430 0x55ec98997b50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 --2- 172.21.15.138:0/2439761590 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ec98991b10 0x55ec989a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 >> 172.21.15.138:0/2439761590 conn(0x55ec9898f940 msgr2=0x55ec989964f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe59aa00640 0 client.16377 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 shutdown_connections 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 1 -- 172.21.15.138:0/2439761590 wait complete. 2024-08-20T01:15:28.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.199+0000 7fe5bebcd740 20 client.16377 trim_cache size 0 max 16384 2024-08-20T01:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:15:28.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 msgr2=0x7fe59c0b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 0x7fe59c0b6110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0005ab0 tx=0x7fe5a0005a20 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 msgr2=0x7fe5b419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b419af60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe5a800e9f0 tx=0x7fe5a800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 shutdown_connections 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe59c0b3c50 0x7fe59c0b6110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe5b410ad90 0x7fe5b41a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe5b410a410 0x7fe5b419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 --2- 172.21.15.138:0/1769062001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5b4104a10 0x7fe5b419af60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:28.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 >> 172.21.15.138:0/1769062001 conn(0x7fe5b4100410 msgr2=0x7fe5b4102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 shutdown_connections 2024-08-20T01:15:28.201 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:28.200+0000 7fe5ba600640 1 -- 172.21.15.138:0/1769062001 wait complete. 2024-08-20T01:15:29.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 290859 3.77 MB/sec execute 221 sec latency 3508.291 ms 2024-08-20T01:15:29.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:29 smithi142 ceph-mon[21799]: pgmap v586: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-08-20T01:15:30.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:29 smithi138 ceph-mon[18076]: pgmap v586: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-08-20T01:15:30.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 291868 3.77 MB/sec execute 222 sec latency 522.585 ms 2024-08-20T01:15:30.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:29 smithi167 ceph-mon[21965]: pgmap v586: 129 pgs: 129 active+clean; 186 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.1 MiB/s wr, 154 op/s 2024-08-20T01:15:31.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 292570 3.77 MB/sec execute 223 sec latency 519.949 ms 2024-08-20T01:15:31.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:31 smithi142 ceph-mon[21799]: pgmap v587: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-08-20T01:15:32.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:31 smithi138 ceph-mon[18076]: pgmap v587: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-08-20T01:15:32.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 294830 3.82 MB/sec execute 224 sec latency 13.909 ms 2024-08-20T01:15:32.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:31 smithi167 ceph-mon[21965]: pgmap v587: 129 pgs: 129 active+clean; 188 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.3 MiB/s wr, 186 op/s 2024-08-20T01:15:33.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:32 smithi138 ceph-mon[18076]: pgmap v588: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-08-20T01:15:33.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 296117 3.82 MB/sec execute 225 sec latency 71.173 ms 2024-08-20T01:15:33.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:32 smithi167 ceph-mon[21965]: pgmap v588: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-08-20T01:15:33.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:32 smithi142 ceph-mon[21799]: pgmap v588: 129 pgs: 129 active+clean; 190 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-08-20T01:15:34.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 296117 3.81 MB/sec execute 226 sec latency 1071.257 ms 2024-08-20T01:15:35.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 298268 3.85 MB/sec execute 227 sec latency 1167.856 ms 2024-08-20T01:15:35.962 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:35 smithi167 ceph-mon[21965]: pgmap v589: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-08-20T01:15:35.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:35 smithi142 ceph-mon[21799]: pgmap v589: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-08-20T01:15:36.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:35 smithi138 ceph-mon[18076]: pgmap v589: 129 pgs: 129 active+clean; 191 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-08-20T01:15:36.160 INFO:tasks.workunit.client.0.smithi138.stdout: 1 299532 3.86 MB/sec execute 228 sec latency 22.555 ms 2024-08-20T01:15:37.162 INFO:tasks.workunit.client.0.smithi138.stdout: 1 299665 3.84 MB/sec execute 229 sec latency 890.867 ms 2024-08-20T01:15:37.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:37 smithi142 ceph-mon[21799]: pgmap v590: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 110 op/s 2024-08-20T01:15:38.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:37 smithi138 ceph-mon[18076]: pgmap v590: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 110 op/s 2024-08-20T01:15:38.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 301502 3.88 MB/sec execute 230 sec latency 1286.688 ms 2024-08-20T01:15:38.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:37 smithi167 ceph-mon[21965]: pgmap v590: 129 pgs: 129 active+clean; 192 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1002 KiB/s wr, 110 op/s 2024-08-20T01:15:39.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 302436 3.87 MB/sec execute 231 sec latency 160.146 ms 2024-08-20T01:15:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:39 smithi142 ceph-mon[21799]: pgmap v591: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-08-20T01:15:40.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:39 smithi138 ceph-mon[18076]: pgmap v591: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-08-20T01:15:40.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 303214 3.87 MB/sec execute 232 sec latency 357.062 ms 2024-08-20T01:15:40.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:39 smithi167 ceph-mon[21965]: pgmap v591: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.0 MiB/s wr, 118 op/s 2024-08-20T01:15:41.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:40 smithi138 ceph-mon[18076]: pgmap v592: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-08-20T01:15:41.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 303214 3.86 MB/sec execute 233 sec latency 1109.124 ms 2024-08-20T01:15:41.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:40 smithi167 ceph-mon[21965]: pgmap v592: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-08-20T01:15:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:40 smithi142 ceph-mon[21799]: pgmap v592: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 174 op/s 2024-08-20T01:15:41.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 -- 172.21.15.138:0/1997593265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00101790 msgr2=0x7f7d00101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 --2- 172.21.15.138:0/1997593265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00101790 0x7f7d00101b70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009ca0 tx=0x7f7cf002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 -- 172.21.15.138:0/1997593265 shutdown_connections 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 --2- 172.21.15.138:0/1997593265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d0010e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 --2- 172.21.15.138:0/1997593265 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00107190 0x7f7d001075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 --2- 172.21.15.138:0/1997593265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00101790 0x7f7d00101b70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 -- 172.21.15.138:0/1997593265 >> 172.21.15.138:0/1997593265 conn(0x7f7d000fd1f0 msgr2=0x7f7d000ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 -- 172.21.15.138:0/1997593265 shutdown_connections 2024-08-20T01:15:41.335 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.335+0000 7f7d05a00640 1 -- 172.21.15.138:0/1997593265 wait complete. 2024-08-20T01:15:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.336+0000 7f7d05a00640 1 Processor -- start 2024-08-20T01:15:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.336+0000 7f7d05a00640 1 -- start start 2024-08-20T01:15:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00101790 0x7f7d0019b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.337 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d001a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00111790 con 0x7f7d00107190 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00111610 con 0x7f7d00107b10 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d00111910 con 0x7f7d00101790 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d001a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42876/0 (socket says 172.21.15.138:42876) 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00101790 0x7f7d0019b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 -- 172.21.15.138:0/2225821450 learned_addr learned my addr 172.21.15.138:0/2225821450 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 -- 172.21.15.138:0/2225821450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00101790 msgr2=0x7f7d0019b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00101790 0x7f7d0019b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 -- 172.21.15.138:0/2225821450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 msgr2=0x7f7d001a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d001a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.338 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7cf7400640 1 -- 172.21.15.138:0/2225821450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d001a10e0 con 0x7f7d00107190 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.337+0000 7f7d05000640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d001a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7cf7400640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9f0 tx=0x7f7ce800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7cfd600640 1 -- 172.21.15.138:0/2225821450 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce800ccc0 con 0x7f7d00107190 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7cfd600640 1 -- 172.21.15.138:0/2225821450 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce80108e0 con 0x7f7d00107190 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7cfd600640 1 -- 172.21.15.138:0/2225821450 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8018880 con 0x7f7d00107190 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d001a13d0 con 0x7f7d00107190 2024-08-20T01:15:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.338+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d001a7a90 con 0x7f7d00107190 2024-08-20T01:15:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.341+0000 7f7d0a009740 1 Processor -- start 2024-08-20T01:15:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 -- start start 2024-08-20T01:15:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 0x55c7b52cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7b5233210 con 0x55c7b52bdb10 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7b5222710 con 0x55c7b52be430 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d0a009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c7b5226750 con 0x55c7b52ca3c0 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7cfd600640 1 -- 172.21.15.138:0/2225821450 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7ce8010430 con 0x7f7d00107190 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35726/0 (socket says 172.21.15.138:35726) 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7cffe00640 1 -- 172.21.15.138:0/780351643 learned_addr learned my addr 172.21.15.138:0/780351643 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7d05000640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.342+0000 7f7cf7400640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 0x55c7b52cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 -- 172.21.15.138:0/780351643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 msgr2=0x55c7b52cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 0x55c7b52cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 -- 172.21.15.138:0/780351643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 msgr2=0x55c7b52bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52bdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 -- 172.21.15.138:0/780351643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b51eede0 con 0x55c7b52be430 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cf7400640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 0x55c7b52cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cffe00640 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7cf002ade0 tx=0x7f7cf0005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c08f314f6fa303d7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cf5600640 1 -- 172.21.15.138:0/780351643 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0031cf0 con 0x55c7b52be430 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cf5600640 1 -- 172.21.15.138:0/780351643 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf0036030 con 0x55c7b52be430 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.343+0000 7f7cf5600640 1 -- 172.21.15.138:0/780351643 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00042a0 con 0x55c7b52be430 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.344+0000 7f7cfd600640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 0x7f7ce40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.344+0000 7f7d05000640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 0x7f7ce40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.344+0000 7f7cfd600640 1 -- 172.21.15.138:0/2225821450 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f7ce80e4830 con 0x7f7d00107190 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.344+0000 7f7d0a009740 1 -- 172.21.15.138:0/780351643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 msgr2=0x55c7b52c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.344+0000 7f7d0a009740 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7cf002ade0 tx=0x7f7cf0005fa0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 -- 172.21.15.138:0/780351643 shutdown_connections 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52ca3c0 0x55c7b52cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52c9e80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 --2- 172.21.15.138:0/780351643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 -- 172.21.15.138:0/780351643 >> 172.21.15.138:0/780351643 conn(0x55c7b52bb940 msgr2=0x55c7b52bcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 -- 172.21.15.138:0/780351643 shutdown_connections 2024-08-20T01:15:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 -- 172.21.15.138:0/780351643 wait complete. 2024-08-20T01:15:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d05000640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 0x7f7ce40b6200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8012640 tx=0x7f7cf8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 -- start start 2024-08-20T01:15:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.345+0000 7f7d0a009740 1 Processor -- start 2024-08-20T01:15:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52be430 0x55c7b52c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b52c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7b515bab0 con 0x55c7b52bdb10 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7b51c9140 con 0x55c7b52ca3c0 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c7b52ccec0 con 0x55c7b52be430 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d0a009740 10 client.0 _ll_register_callbacks cb 0x55c7b51dc5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7d05000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b52c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42896/0 (socket says 172.21.15.138:42896) 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7cffe00640 1 -- 172.21.15.138:0/2363964201 learned_addr learned my addr 172.21.15.138:0/2363964201 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.346+0000 7f7cf7400640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52be430 0x55c7b52c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 -- 172.21.15.138:0/2363964201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52be430 msgr2=0x55c7b52c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52be430 0x55c7b52c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 -- 172.21.15.138:0/2363964201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 msgr2=0x55c7b52c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 -- 172.21.15.138:0/2363964201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b51ca170 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d05000640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b52c74f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009fd0 tx=0x7f7cf8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cb01d6469f558d1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7cee000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf802e060 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f7cf8018030 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.347+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cf0036030 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7cf8004020 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 10 client.26447 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7b4f85790 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cf0036030 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f7cf801e5a0 con 0x55c7b52ca3c0 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 10 client.26447 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 10 client.26447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 20 client.26447 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b52d9e10 0x55c7b52dc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 4 client.26447 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:15:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7b5235160 con 0x55c7b52d9e10 2024-08-20T01:15:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.348+0000 7f7cffe00640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b52d9e10 0x55c7b52dc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.349+0000 7f7cee000640 10 client.26447 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.349+0000 7f7cffe00640 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b52d9e10 0x55c7b52dc240 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2201b440b3bbc253 server_cookie=8c587bcdd8f08f0d in_seq=0 out_seq=0 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7cee000640 1 -- 172.21.15.138:0/2363964201 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7b5235160 con 0x55c7b52d9e10 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7cee000640 10 client.26447 handle_command_reply: tid=0 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 client.26447 shutdown 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 msgr2=0x55c7b52c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b52c74f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009fd0 tx=0x7f7cf8004730 comp rx=0 tx=0).stop 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 shutdown_connections 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b52d9e10 0x55c7b52dc240 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b52c74f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7cee000640 0 client.26447 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52be430 0x55c7b52c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 --2- 172.21.15.138:0/2363964201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52bdb10 0x55c7b52c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 >> 172.21.15.138:0/2363964201 conn(0x55c7b52bb940 msgr2=0x55c7b52c14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 shutdown_connections 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 1 -- 172.21.15.138:0/2363964201 wait complete. 2024-08-20T01:15:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.350+0000 7f7d0a009740 20 client.26447 trim_cache size 0 max 16384 2024-08-20T01:15:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 Processor -- start 2024-08-20T01:15:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 -- start start 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52ca3c0 0x55c7b52e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001b44d0 con 0x55c7b52ca3c0 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001b1d40 con 0x55c7b52be430 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d0a009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001afbb0 con 0x55c7b52bdb10 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7cf7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35736/0 (socket says 172.21.15.138:35736) 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7cf7400640 1 -- 172.21.15.138:0/1046497404 learned_addr learned my addr 172.21.15.138:0/1046497404 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.368+0000 7f7d05000640 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52ca3c0 0x55c7b52e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 -- 172.21.15.138:0/1046497404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 msgr2=0x55c7b52bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52bcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 -- 172.21.15.138:0/1046497404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52ca3c0 msgr2=0x55c7b52e20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52ca3c0 0x55c7b52e20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 -- 172.21.15.138:0/1046497404 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b4ffe620 con 0x55c7b52be430 2024-08-20T01:15:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf7400640 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80a82b0 tx=0x7f7ce80aa870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82b38de5a985c673 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf4c00640 1 -- 172.21.15.138:0/1046497404 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80ab040 con 0x55c7b52be430 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf4c00640 1 -- 172.21.15.138:0/1046497404 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce80af640 con 0x55c7b52be430 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.369+0000 7f7cf4c00640 1 -- 172.21.15.138:0/1046497404 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80e43d0 con 0x55c7b52be430 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 -- 172.21.15.138:0/1046497404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 msgr2=0x55c7b52e1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80a82b0 tx=0x7f7ce80aa870 comp rx=0 tx=0).stop 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 -- 172.21.15.138:0/1046497404 shutdown_connections 2024-08-20T01:15:41.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52ca3c0 0x55c7b52e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52be430 0x55c7b52e1ba0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 --2- 172.21.15.138:0/1046497404 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 -- 172.21.15.138:0/1046497404 >> 172.21.15.138:0/1046497404 conn(0x55c7b52bb940 msgr2=0x55c7b52bc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 -- 172.21.15.138:0/1046497404 shutdown_connections 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.370+0000 7f7d0a009740 1 -- 172.21.15.138:0/1046497404 wait complete. 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 -- start start 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 Processor -- start 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52be430 0x55c7b52cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001b0200 con 0x55c7b52be430 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001b1670 con 0x55c7b52ca3c0 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7d001b08d0 con 0x55c7b52bdb10 2024-08-20T01:15:41.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.371+0000 7f7d0a009740 10 client.0 _ll_register_callbacks cb 0x7f7cf8009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7d05000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7d05000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35746/0 (socket says 172.21.15.138:35746) 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:53520/0 (socket says 172.21.15.138:53520) 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7d05000640 1 -- 172.21.15.138:0/650293838 learned_addr learned my addr 172.21.15.138:0/650293838 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cf7400640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52be430 0x55c7b52cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 -- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 msgr2=0x55c7b5342b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 -- 172.21.15.138:0/650293838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52be430 msgr2=0x55c7b52cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52be430 0x55c7b52cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 -- 172.21.15.138:0/650293838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b53431b0 con 0x55c7b52bdb10 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7d05000640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cffe00640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000f80 tx=0x7f7cf0003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4b339bbd27266a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:41.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cee000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.372+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf004a040 con 0x55c7b52bdb10 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f7cf0031db0 con 0x55c7b52bdb10 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ce80af640 con 0x55c7b52bdb10 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7cf0031450 con 0x55c7b52bdb10 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 10 client.26419 fetch_fsmap learned FSMap version 37 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55c7b5344520 con 0x55c7b52bdb10 2024-08-20T01:15:41.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ce80af640 con 0x55c7b52bdb10 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 37) ==== 8362+0+0 (secure 0 0 0) 0x7f7cf0043070 con 0x55c7b52bdb10 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 10 client.26419 fetch_fsmap finished waiting for FSMap version 37 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 10 client.26419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 20 client.26419 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b5344ac0 0x55c7b5346ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 4 client.26419 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.373+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7b5230c00 con 0x55c7b5344ac0 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.374+0000 7f7cf7400640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b5344ac0 0x55c7b5346ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.374+0000 7f7cf7400640 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b5344ac0 0x55c7b5346ee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43ba5f2472907a1f server_cookie=66fff77641a09c01 in_seq=0 out_seq=0 2024-08-20T01:15:41.376 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.374+0000 7f7cee000640 10 client.26419 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7cee000640 1 -- 172.21.15.138:0/650293838 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7b5230c00 con 0x55c7b5344ac0 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7cee000640 10 client.26419 handle_command_reply: tid=0 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 client.26419 shutdown 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 msgr2=0x55c7b52cf780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0000f80 tx=0x7f7cf0003970 comp rx=0 tx=0).stop 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 shutdown_connections 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c7b5344ac0 0x55c7b5346ee0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7b52ca3c0 0x55c7b5342b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7cee000640 0 client.26419 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7b52be430 0x55c7b52cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 --2- 172.21.15.138:0/650293838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7b52bdb10 0x55c7b52cf780 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 >> 172.21.15.138:0/650293838 conn(0x55c7b52bb940 msgr2=0x55c7b52c8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 shutdown_connections 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 1 -- 172.21.15.138:0/650293838 wait complete. 2024-08-20T01:15:41.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.425+0000 7f7d0a009740 20 client.26419 trim_cache size 0 max 16384 2024-08-20T01:15:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:15:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:15:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:15:41.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 msgr2=0x7f7ce40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 0x7f7ce40b6200 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8012640 tx=0x7f7cf8005f90 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 msgr2=0x7f7d0019b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800e9f0 tx=0x7f7ce800edf0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 shutdown_connections 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7ce40b3d40 0x7f7ce40b6200 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d00107b10 0x7f7d001a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d00107190 0x7f7d0019b580 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 --2- 172.21.15.138:0/2225821450 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7d00101790 0x7f7d0019b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 >> 172.21.15.138:0/2225821450 conn(0x7f7d000fd1f0 msgr2=0x7f7d00104320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 shutdown_connections 2024-08-20T01:15:41.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:41.426+0000 7f7d05a00640 1 -- 172.21.15.138:0/2225821450 wait complete. 2024-08-20T01:15:42.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 305452 3.90 MB/sec execute 234 sec latency 1109.884 ms 2024-08-20T01:15:43.161 INFO:tasks.workunit.client.0.smithi138.stdout: 1 306757 3.90 MB/sec execute 235 sec latency 68.446 ms 2024-08-20T01:15:43.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:43 smithi142 ceph-mon[21799]: pgmap v593: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:15:44.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:43 smithi138 ceph-mon[18076]: pgmap v593: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:15:44.168 INFO:tasks.workunit.client.0.smithi138.stdout: 1 306757 3.89 MB/sec execute 236 sec latency 1075.847 ms 2024-08-20T01:15:44.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:43 smithi167 ceph-mon[21965]: pgmap v593: 129 pgs: 129 active+clean; 198 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 146 op/s 2024-08-20T01:15:45.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 308231 3.90 MB/sec execute 237 sec latency 1195.807 ms 2024-08-20T01:15:45.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:45 smithi142 ceph-mon[21799]: pgmap v594: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-20T01:15:46.014 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:45 smithi167 ceph-mon[21965]: pgmap v594: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-20T01:15:46.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:45 smithi138 ceph-mon[18076]: pgmap v594: 129 pgs: 129 active+clean; 203 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-08-20T01:15:46.168 INFO:tasks.workunit.client.0.smithi138.stdout: 1 309439 3.92 MB/sec execute 238 sec latency 58.288 ms 2024-08-20T01:15:47.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:46 smithi138 ceph-mon[18076]: pgmap v595: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-20T01:15:47.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 310305 3.92 MB/sec execute 239 sec latency 446.689 ms 2024-08-20T01:15:47.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:46 smithi167 ceph-mon[21965]: pgmap v595: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-20T01:15:47.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:46 smithi142 ceph-mon[21799]: pgmap v595: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-08-20T01:15:48.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 310756 3.90 MB/sec execute 240 sec latency 830.177 ms 2024-08-20T01:15:48.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:48 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:15:48.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:48 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:15:49.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:48 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:15:49.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 312625 3.94 MB/sec execute 241 sec latency 12.268 ms 2024-08-20T01:15:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:49 smithi142 ceph-mon[21799]: pgmap v596: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-08-20T01:15:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:49 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:15:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:49 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:15:49.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:49 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:15:50.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:49 smithi138 ceph-mon[18076]: pgmap v596: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-08-20T01:15:50.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:49 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:15:50.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:49 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:15:50.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:49 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:15:50.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 313406 3.94 MB/sec execute 242 sec latency 353.320 ms 2024-08-20T01:15:50.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:49 smithi167 ceph-mon[21965]: pgmap v596: 129 pgs: 129 active+clean; 204 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 161 op/s 2024-08-20T01:15:50.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:49 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:15:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:49 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:15:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:49 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:15:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:50 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:50.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:50 smithi142 ceph-mon[21799]: pgmap v597: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-08-20T01:15:51.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:50 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:51.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:50 smithi138 ceph-mon[18076]: pgmap v597: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-08-20T01:15:51.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 313850 3.93 MB/sec execute 243 sec latency 718.578 ms 2024-08-20T01:15:51.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:50 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:15:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:50 smithi167 ceph-mon[21965]: pgmap v597: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 2.2 MiB/s wr, 235 op/s 2024-08-20T01:15:52.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 316127 3.97 MB/sec execute 244 sec latency 14.871 ms 2024-08-20T01:15:53.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 317396 3.98 MB/sec execute 245 sec latency 119.076 ms 2024-08-20T01:15:53.990 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:53 smithi142 ceph-mon[21799]: pgmap v598: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-08-20T01:15:54.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:53 smithi138 ceph-mon[18076]: pgmap v598: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-08-20T01:15:54.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 317396 3.96 MB/sec execute 246 sec latency 1119.176 ms 2024-08-20T01:15:54.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:53 smithi167 ceph-mon[21965]: pgmap v598: 129 pgs: 129 active+clean; 208 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-08-20T01:15:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- 172.21.15.138:0/621403974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075b90 msgr2=0x7f7810113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 --2- 172.21.15.138:0/621403974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075b90 0x7f7810113010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009900 tx=0x7f77fc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- 172.21.15.138:0/621403974 shutdown_connections 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 --2- 172.21.15.138:0/621403974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810113550 0x7f7810115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 --2- 172.21.15.138:0/621403974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075b90 0x7f7810113010 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 --2- 172.21.15.138:0/621403974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7810075270 0x7f7810075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- 172.21.15.138:0/621403974 >> 172.21.15.138:0/621403974 conn(0x7f78100fe280 msgr2=0x7f78101006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- 172.21.15.138:0/621403974 shutdown_connections 2024-08-20T01:15:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- 172.21.15.138:0/621403974 wait complete. 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 Processor -- start 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 -- start start 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.541+0000 7f7816e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7816e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810075b90 0x7f781019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7816e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7810113550 0x7f78101a4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7816e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7810118e00 con 0x7f7810075270 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7816e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7810118c80 con 0x7f7810075b90 2024-08-20T01:15:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7816e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7810118f80 con 0x7f7810113550 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7815a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7815a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56254/0 (socket says 172.21.15.138:56254) 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.545+0000 7f7815a00640 1 -- 172.21.15.138:0/748011521 learned_addr learned my addr 172.21.15.138:0/748011521 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815000640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810075b90 0x7f781019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/748011521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7810113550 msgr2=0x7f78101a4c50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7810113550 0x7f78101a4c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/748011521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810075b90 msgr2=0x7f781019f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810075b90 0x7f781019f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/748011521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78101a5420 con 0x7f7810075270 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f780400e9f0 tx=0x7f780400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f780e000640 1 -- 172.21.15.138:0/748011521 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780400cd80 con 0x7f7810075270 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f780e000640 1 -- 172.21.15.138:0/748011521 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78040108e0 con 0x7f7810075270 2024-08-20T01:15:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f780e000640 1 -- 172.21.15.138:0/748011521 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804018980 con 0x7f7810075270 2024-08-20T01:15:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78101a5710 con 0x7f7810075270 2024-08-20T01:15:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78101abdd0 con 0x7f7810075270 2024-08-20T01:15:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 Processor -- start 2024-08-20T01:15:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 -- start start 2024-08-20T01:15:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b5febb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 0x55e68b60ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 0x55e68b60d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.549 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e68b574210 con 0x55e68b5ff0f0 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e68b563710 con 0x55e68b5fe7d0 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f781b2b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e68b567750 con 0x55e68b60b080 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f780e000640 1 -- 172.21.15.138:0/748011521 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7804010430 con 0x7f7810075270 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 0x55e68b60ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 0x55e68b60d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 0x55e68b60d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41264/0 (socket says 172.21.15.138:41264) 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815000640 1 -- 172.21.15.138:0/3520992918 learned_addr learned my addr 172.21.15.138:0/3520992918 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/3520992918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 msgr2=0x55e68b60d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 0x55e68b60d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/3520992918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 msgr2=0x55e68b5febb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b5febb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 -- 172.21.15.138:0/3520992918 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68b52fde0 con 0x55e68b5ff0f0 2024-08-20T01:15:54.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.546+0000 7f7815a00640 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 0x55e68b60ab40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f78040e1b50 tx=0x7f78040ec670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ca69955e8ab2db server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f780e000640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 0x7f77f80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7802a00640 1 -- 172.21.15.138:0/3520992918 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040f3070 con 0x55e68b5ff0f0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7802a00640 1 -- 172.21.15.138:0/3520992918 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7804010720 con 0x55e68b5ff0f0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7802a00640 1 -- 172.21.15.138:0/3520992918 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7804026060 con 0x55e68b5ff0f0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f780e000640 1 -- 172.21.15.138:0/748011521 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f78040e4800 con 0x7f7810075270 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7816400640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 0x7f77f80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7816400640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 0x7f77f80b6280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f781006bd80 tx=0x7f7808009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/3520992918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 msgr2=0x55e68b60ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 0x55e68b60ab40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f78040e1b50 tx=0x7f78040ec670 comp rx=0 tx=0).stop 2024-08-20T01:15:54.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/3520992918 shutdown_connections 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b60b080 0x55e68b60d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5ff0f0 0x55e68b60ab40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3520992918 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b5febb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/3520992918 >> 172.21.15.138:0/3520992918 conn(0x55e68b5fc600 msgr2=0x55e68b5fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/3520992918 shutdown_connections 2024-08-20T01:15:54.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/3520992918 wait complete. 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- start start 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 Processor -- start 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 0x55e68b608080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b6085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b60b080 0x55e68b608b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.556 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e68b49cab0 con 0x55e68b5fe7d0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e68b50a140 con 0x55e68b60b080 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e68b60dd60 con 0x55e68b5ff0f0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 10 client.0 _ll_register_callbacks cb 0x55e68b51d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b6085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b6085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41280/0 (socket says 172.21.15.138:41280) 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815000640 1 -- 172.21.15.138:0/2115280507 learned_addr learned my addr 172.21.15.138:0/2115280507 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 0x55e68b608080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7816400640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b60b080 0x55e68b608b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 -- 172.21.15.138:0/2115280507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 msgr2=0x55e68b6085c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b6085c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 -- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b60b080 msgr2=0x55e68b608b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b60b080 0x55e68b608b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 -- 172.21.15.138:0/2115280507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68b50b170 con 0x55e68b5fe7d0 2024-08-20T01:15:54.557 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f7815a00640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 0x55e68b608080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f78040a8280 tx=0x7f78040ea380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34dc8a1322685c03 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78040f3030 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f78040a3750 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e68b53aa30 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f78040aa960 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 10 client.16407 fetch_fsmap learned FSMap version 38 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e68b2c6790 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e68b53aa30 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f78040e4060 con 0x55e68b5fe7d0 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 10 client.16407 fetch_fsmap finished waiting for FSMap version 38 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 10 client.16407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:54.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 20 client.16407 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b61af50 0x55e68b61d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 4 client.16407 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.551+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e68b576160 con 0x55e68b61af50 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f7815000640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b61af50 0x55e68b61d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f77f7400640 10 client.16407 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f7815000640 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b61af50 0x55e68b61d340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b7225ebb868d327 server_cookie=ee3501c27b3cd033 in_seq=0 out_seq=0 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f77f7400640 1 -- 172.21.15.138:0/2115280507 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e68b576160 con 0x55e68b61af50 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f77f7400640 10 client.16407 handle_command_reply: tid=0 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 client.16407 shutdown 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 msgr2=0x55e68b608080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 0x55e68b608080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f78040a8280 tx=0x7f78040ea380 comp rx=0 tx=0).stop 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 shutdown_connections 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b61af50 0x55e68b61d340 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b60b080 0x55e68b608b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b6085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2115280507 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b5fe7d0 0x55e68b608080 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f77f7400640 0 client.16407 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 >> 172.21.15.138:0/2115280507 conn(0x55e68b5fc600 msgr2=0x55e68b6021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 shutdown_connections 2024-08-20T01:15:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 1 -- 172.21.15.138:0/2115280507 wait complete. 2024-08-20T01:15:54.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.556+0000 7f781b2b0740 20 client.16407 trim_cache size 0 max 16384 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 Processor -- start 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- start start 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 0x55e68b609dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5ff0f0 0x55e68b605270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b8990 con 0x55e68b60b080 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b6230 con 0x55e68b5ff0f0 2024-08-20T01:15:54.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b4080 con 0x55e68b5fe7d0 2024-08-20T01:15:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56288/0 (socket says 172.21.15.138:56288) 2024-08-20T01:15:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 -- 172.21.15.138:0/3726680817 learned_addr learned my addr 172.21.15.138:0/3726680817 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:54.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7815a00640 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 0x55e68b609dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 -- 172.21.15.138:0/3726680817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 msgr2=0x55e68b609dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 0x55e68b609dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 -- 172.21.15.138:0/3726680817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5ff0f0 msgr2=0x55e68b605270 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5ff0f0 0x55e68b605270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 -- 172.21.15.138:0/3726680817 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e68b5fef20 con 0x55e68b60b080 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7815a00640 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 0x55e68b609dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7808009750 tx=0x7f7808011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8586e0c69fd9a7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7802000640 1 -- 172.21.15.138:0/3726680817 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780801d510 con 0x55e68b60b080 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7802000640 1 -- 172.21.15.138:0/3726680817 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780801db60 con 0x55e68b60b080 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7802000640 1 -- 172.21.15.138:0/3726680817 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780801ecd0 con 0x55e68b60b080 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- 172.21.15.138:0/3726680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 msgr2=0x55e68b6057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7808009750 tx=0x7f7808011000 comp rx=0 tx=0).stop 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- 172.21.15.138:0/3726680817 shutdown_connections 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b6057b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5ff0f0 0x55e68b605270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- 172.21.15.138:0/3726680817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5fe7d0 0x55e68b609dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- 172.21.15.138:0/3726680817 >> 172.21.15.138:0/3726680817 conn(0x55e68b5fc600 msgr2=0x55e68b6096b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- 172.21.15.138:0/3726680817 shutdown_connections 2024-08-20T01:15:54.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- 172.21.15.138:0/3726680817 wait complete. 2024-08-20T01:15:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- start start 2024-08-20T01:15:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 Processor -- start 2024-08-20T01:15:54.587 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b620ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b61efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b46f0 con 0x55e68b60b080 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b5b60 con 0x55e68b5fe7d0 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f78101b4dc0 con 0x55e68b5ff0f0 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f781b2b0740 10 client.0 _ll_register_callbacks cb 0x7f78040eaa70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.580+0000 7f7816400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56300/0 (socket says 172.21.15.138:56300) 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 -- 172.21.15.138:0/2438845903 learned_addr learned my addr 172.21.15.138:0/2438845903 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 -- 172.21.15.138:0/2438845903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 msgr2=0x55e68b61efa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b61efa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 -- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 msgr2=0x55e68b620ab0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:15:54.588 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b620ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 -- 172.21.15.138:0/2438845903 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e68b33f620 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7816400640 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7808010fd0 tx=0x7f780801e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b73d64d9642a0f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7808035020 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9453+0+0 (secure 0 0 0) 0x7f7808009040 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780801db60 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7808040050 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 10 client.16413 fetch_fsmap learned FSMap version 38 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55e68b6847d0 con 0x55e68b60b080 2024-08-20T01:15:54.589 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780801db60 con 0x55e68b60b080 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f7808030450 con 0x55e68b60b080 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 10 client.16413 fetch_fsmap finished waiting for FSMap version 38 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 10 client.16413 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 20 client.16413 populate_metadata read hostname 'smithi138' 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b684f70 0x55e68b687390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 4 client.16413 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e68b571c00 con 0x55e68b684f70 2024-08-20T01:15:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7815a00640 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b684f70 0x55e68b687390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:15:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f7815a00640 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b684f70 0x55e68b687390 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51ebfa826f688bda server_cookie=8216754e9887652c in_seq=0 out_seq=0 2024-08-20T01:15:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.586+0000 7f77f7400640 10 client.16413 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:54.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.677+0000 7f77f7400640 1 -- 172.21.15.138:0/2438845903 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e68b571c00 con 0x55e68b684f70 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.677+0000 7f77f7400640 10 client.16413 handle_command_reply: tid=0 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.677+0000 7f781b2b0740 1 client.16413 shutdown 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.677+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 msgr2=0x55e68b61f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.677+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7808010fd0 tx=0x7f780801e720 comp rx=0 tx=0).stop 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 shutdown_connections 2024-08-20T01:15:54.678 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e68b684f70 0x55e68b687390 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e68b60b080 0x55e68b61f580 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e68b5ff0f0 0x55e68b61efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 --2- 172.21.15.138:0/2438845903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e68b5fe7d0 0x55e68b620ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 >> 172.21.15.138:0/2438845903 conn(0x55e68b5fc600 msgr2=0x55e68b609ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f77f7400640 0 client.16413 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 shutdown_connections 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 1 -- 172.21.15.138:0/2438845903 wait complete. 2024-08-20T01:15:54.679 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f781b2b0740 20 client.16413 trim_cache size 0 max 16384 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 msgr2=0x7f77f80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 0x7f77f80b6280 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f781006bd80 tx=0x7f7808009500 comp rx=0 tx=0).stop 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 msgr2=0x7f781019f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:15:54.680 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f780400e9f0 tx=0x7f780400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 shutdown_connections 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f77f80b3dc0 0x7f77f80b6280 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7810113550 0x7f78101a4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7810075b90 0x7f781019f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 --2- 172.21.15.138:0/748011521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7810075270 0x7f781019f380 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:15:54.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 >> 172.21.15.138:0/748011521 conn(0x7f78100fe280 msgr2=0x7f78100fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:15:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 shutdown_connections 2024-08-20T01:15:54.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:15:54.679+0000 7f7816e00640 1 -- 172.21.15.138:0/748011521 wait complete. 2024-08-20T01:15:55.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 317468 3.95 MB/sec execute 247 sec latency 1182.242 ms 2024-08-20T01:15:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:55 smithi142 ceph-mon[21799]: pgmap v599: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-08-20T01:15:56.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:55 smithi138 ceph-mon[18076]: pgmap v599: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-08-20T01:15:56.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 319078 3.97 MB/sec execute 248 sec latency 812.506 ms 2024-08-20T01:15:56.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:55 smithi167 ceph-mon[21965]: pgmap v599: 129 pgs: 129 active+clean; 209 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-08-20T01:15:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:56 smithi142 ceph-mon[21799]: pgmap v600: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-20T01:15:57.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:56 smithi138 ceph-mon[18076]: pgmap v600: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-20T01:15:57.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 320467 3.98 MB/sec execute 249 sec latency 17.411 ms 2024-08-20T01:15:57.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:56 smithi167 ceph-mon[21965]: pgmap v600: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 2.0 MiB/s wr, 201 op/s 2024-08-20T01:15:58.169 INFO:tasks.workunit.client.0.smithi138.stdout: 1 320938 3.98 MB/sec execute 250 sec latency 739.647 ms 2024-08-20T01:15:59.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 320938 3.96 MB/sec execute 251 sec latency 1739.781 ms 2024-08-20T01:15:59.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:15:59 smithi138 ceph-mon[18076]: pgmap v601: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:15:59.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:15:59 smithi142 ceph-mon[21799]: pgmap v601: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:16:00.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 322420 3.98 MB/sec execute 252 sec latency 1803.301 ms 2024-08-20T01:16:00.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:15:59 smithi167 ceph-mon[21965]: pgmap v601: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:16:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:00 smithi142 ceph-mon[21799]: pgmap v602: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-08-20T01:16:01.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:00 smithi138 ceph-mon[18076]: pgmap v602: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-08-20T01:16:01.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 322987 3.98 MB/sec execute 253 sec latency 66.906 ms 2024-08-20T01:16:01.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:00 smithi167 ceph-mon[21965]: pgmap v602: 129 pgs: 129 active+clean; 213 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 230 op/s 2024-08-20T01:16:02.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 324180 3.99 MB/sec execute 254 sec latency 55.597 ms 2024-08-20T01:16:03.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 326316 4.02 MB/sec execute 255 sec latency 152.647 ms 2024-08-20T01:16:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:03 smithi142 ceph-mon[21799]: pgmap v603: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-20T01:16:04.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:03 smithi138 ceph-mon[18076]: pgmap v603: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-20T01:16:04.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 327479 4.02 MB/sec execute 256 sec latency 23.868 ms 2024-08-20T01:16:04.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:03 smithi167 ceph-mon[21965]: pgmap v603: 129 pgs: 129 active+clean; 215 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 1.6 MiB/s wr, 159 op/s 2024-08-20T01:16:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:04 smithi138 ceph-mon[18076]: pgmap v604: 129 pgs: 129 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-08-20T01:16:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:04 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:16:05.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:04 smithi138 ceph-mon[18076]: osdmap e92: 12 total, 12 up, 12 in 2024-08-20T01:16:05.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 328029 4.02 MB/sec execute 257 sec latency 618.781 ms 2024-08-20T01:16:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:04 smithi167 ceph-mon[21965]: pgmap v604: 129 pgs: 129 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-08-20T01:16:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:04 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:16:05.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:04 smithi167 ceph-mon[21965]: osdmap e92: 12 total, 12 up, 12 in 2024-08-20T01:16:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:04 smithi142 ceph-mon[21799]: pgmap v604: 129 pgs: 129 active+clean; 220 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.1 MiB/s wr, 220 op/s 2024-08-20T01:16:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:04 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:16:05.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:04 smithi142 ceph-mon[21799]: osdmap e92: 12 total, 12 up, 12 in 2024-08-20T01:16:06.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 328029 4.00 MB/sec execute 258 sec latency 1618.911 ms 2024-08-20T01:16:07.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:06 smithi138 ceph-mon[18076]: pgmap v606: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-08-20T01:16:07.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 328029 3.99 MB/sec execute 259 sec latency 2619.050 ms 2024-08-20T01:16:07.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:06 smithi167 ceph-mon[21965]: pgmap v606: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-08-20T01:16:07.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:06 smithi142 ceph-mon[21799]: pgmap v606: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 -- 172.21.15.138:0/2247505095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 msgr2=0x7f06c8104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 --2- 172.21.15.138:0/2247505095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c8104df0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f06bc009900 tx=0x7f06bc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 -- 172.21.15.138:0/2247505095 shutdown_connections 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 --2- 172.21.15.138:0/2247505095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810ad90 0x7f06c8111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 --2- 172.21.15.138:0/2247505095 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810a410 0x7f06c810a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 --2- 172.21.15.138:0/2247505095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c8104df0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.804+0000 7f06cee00640 1 -- 172.21.15.138:0/2247505095 >> 172.21.15.138:0/2247505095 conn(0x7f06c8100450 msgr2=0x7f06c8102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.805+0000 7f06cee00640 1 -- 172.21.15.138:0/2247505095 shutdown_connections 2024-08-20T01:16:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.805+0000 7f06cee00640 1 -- 172.21.15.138:0/2247505095 wait complete. 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.805+0000 7f06cee00640 1 Processor -- start 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 -- start start 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 0x7f06c819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 0x7f06c81a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8114aa0 con 0x7f06c8104a10 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8114920 con 0x7f06c810a410 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06c8114c20 con 0x7f06c810ad90 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50924/0 (socket says 172.21.15.138:50924) 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cda00640 1 -- 172.21.15.138:0/3873103318 learned_addr learned my addr 172.21.15.138:0/3873103318 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06cd000640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 0x7f06c819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.806+0000 7f06ce400640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 0x7f06c81a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 -- 172.21.15.138:0/3873103318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 msgr2=0x7f06c81a0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 0x7f06c81a0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 -- 172.21.15.138:0/3873103318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 msgr2=0x7f06c819b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 0x7f06c819b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 -- 172.21.15.138:0/3873103318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c81a1040 con 0x7f06c8104a10 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06ce400640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 0x7f06c81a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cd000640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 0x7f06c819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cda00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005ab0 tx=0x7f06bc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06c6000640 1 -- 172.21.15.138:0/3873103318 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc03f070 con 0x7f06c8104a10 2024-08-20T01:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06c6000640 1 -- 172.21.15.138:0/3873103318 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06bc031b60 con 0x7f06c8104a10 2024-08-20T01:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c81a12d0 con 0x7f06c8104a10 2024-08-20T01:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06c6000640 1 -- 172.21.15.138:0/3873103318 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc035030 con 0x7f06c8104a10 2024-08-20T01:16:07.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.807+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c81a79f0 con 0x7f06c8104a10 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.810+0000 7f06d328d740 1 Processor -- start 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 -- start start 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06c6000640 1 -- 172.21.15.138:0/3873103318 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f06bc042070 con 0x7f06c8104a10 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafb0f0 0x559dceb06b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb09470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dcea70210 con 0x559dceafb0f0 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559dcea5f710 con 0x559dceafa7d0 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06d328d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559dcea63750 con 0x559dceb07080 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06cda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafb0f0 0x559dceb06b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06ce400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06ce400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43342/0 (socket says 172.21.15.138:43342) 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06ce400640 1 -- 172.21.15.138:0/3059048269 learned_addr learned my addr 172.21.15.138:0/3059048269 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.811+0000 7f06cd000640 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb09470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 -- 172.21.15.138:0/3059048269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 msgr2=0x559dceb09470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb09470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 -- 172.21.15.138:0/3059048269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafb0f0 msgr2=0x559dceb06b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafb0f0 0x559dceb06b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 -- 172.21.15.138:0/3059048269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcea2bde0 con 0x559dceafa7d0 2024-08-20T01:16:07.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06ce400640 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06c0010450 tx=0x7f06c0010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbc7dcd42ddeba0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06baa00640 1 -- 172.21.15.138:0/3059048269 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0017070 con 0x559dceafa7d0 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06baa00640 1 -- 172.21.15.138:0/3059048269 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c0011c10 con 0x559dceafa7d0 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06baa00640 1 -- 172.21.15.138:0/3059048269 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001ad00 con 0x559dceafa7d0 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.812+0000 7f06c6000640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 0x7f06b00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06c6000640 1 -- 172.21.15.138:0/3873103318 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9541+0+0 (secure 0 0 0) 0x7f06bc104b90 con 0x7f06c8104a10 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06cda00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 0x7f06b00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 -- 172.21.15.138:0/3059048269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 msgr2=0x559dceafabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06c0010450 tx=0x7f06c0010910 comp rx=0 tx=0).stop 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 -- 172.21.15.138:0/3059048269 shutdown_connections 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb09470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafb0f0 0x559dceb06b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 --2- 172.21.15.138:0/3059048269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafa7d0 0x559dceafabb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 -- 172.21.15.138:0/3059048269 >> 172.21.15.138:0/3059048269 conn(0x559dceaf8600 msgr2=0x559dceaf97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06cda00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 0x7f06b00b6200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06bc101f90 tx=0x7f06bc1086d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 -- 172.21.15.138:0/3059048269 shutdown_connections 2024-08-20T01:16:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.813+0000 7f06d328d740 1 -- 172.21.15.138:0/3059048269 wait complete. 2024-08-20T01:16:07.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 -- start start 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 Processor -- start 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafa7d0 0x559dceb04040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 0x559dceb04ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559dce998ab0 con 0x559dceb07080 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559dcea06140 con 0x559dceafb0f0 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.814+0000 7f06d328d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559dceb09d60 con 0x559dceafa7d0 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06d328d740 10 client.0 _ll_register_callbacks cb 0x559dcea195d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06ce400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafa7d0 0x559dceb04040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43344/0 (socket says 172.21.15.138:43344) 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 -- 172.21.15.138:0/3476059675 learned_addr learned my addr 172.21.15.138:0/3476059675 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06ce400640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 0x559dceb04ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 -- 172.21.15.138:0/3476059675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafa7d0 msgr2=0x559dceb04040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafa7d0 0x559dceb04040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 -- 172.21.15.138:0/3476059675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 msgr2=0x559dceb04ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 0x559dceb04ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 -- 172.21.15.138:0/3476059675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcea07170 con 0x559dceafb0f0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06ce400640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 0x559dceb04ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06cd000640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06b4012d20 tx=0x7f06b400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a73dbcb8ef75a2b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.815+0000 7f06af400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4012110 con 0x559dceafb0f0 2024-08-20T01:16:07.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559dcea36a30 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9541+0+0 (secure 0 0 0) 0x7f06b4004050 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06b40122b0 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 10 client.26474 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559dce7c2790 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559dcea36a30 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f06b4010430 con 0x559dceafb0f0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 10 client.26474 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 20 client.26474 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb170a0 0x559dceb19490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 4 client.26474 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.816+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dcea72160 con 0x559dceb170a0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.817+0000 7f06ce400640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb170a0 0x559dceb19490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.817+0000 7f06ce400640 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb170a0 0x559dceb19490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2033459f8b9e364 server_cookie=5bc60e43e08e5064 in_seq=0 out_seq=0 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.817+0000 7f06af400640 10 client.26474 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06af400640 1 -- 172.21.15.138:0/3476059675 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559dcea72160 con 0x559dceb170a0 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06af400640 10 client.26474 handle_command_reply: tid=0 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 client.26474 shutdown 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 msgr2=0x559dceb04580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f06b4012d20 tx=0x7f06b400f790 comp rx=0 tx=0).stop 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 shutdown_connections 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb170a0 0x559dceb19490 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceb07080 0x559dceb04ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb04580 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 --2- 172.21.15.138:0/3476059675 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafa7d0 0x559dceb04040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06af400640 0 client.26474 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 >> 172.21.15.138:0/3476059675 conn(0x559dceaf8600 msgr2=0x559dceafe130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.818+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 shutdown_connections 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.819+0000 7f06d328d740 1 -- 172.21.15.138:0/3476059675 wait complete. 2024-08-20T01:16:07.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.819+0000 7f06d328d740 20 client.26474 trim_cache size 0 max 16384 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 Processor -- start 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 -- start start 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafb0f0 0x559dceb1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceb07080 0x559dceb1f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b4510 con 0x559dceafa7d0 2024-08-20T01:16:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b1de0 con 0x559dceb07080 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06d328d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81afc50 con 0x559dceafb0f0 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06ce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06ce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50950/0 (socket says 172.21.15.138:50950) 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06ce400640 1 -- 172.21.15.138:0/2976152368 learned_addr learned my addr 172.21.15.138:0/2976152368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.834+0000 7f06cd000640 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafb0f0 0x559dceb1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 -- 172.21.15.138:0/2976152368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafb0f0 msgr2=0x559dceb1ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafb0f0 0x559dceb1ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 -- 172.21.15.138:0/2976152368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceb07080 msgr2=0x559dceb1f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceb07080 0x559dceb1f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 -- 172.21.15.138:0/2976152368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dce83b620 con 0x559dceafa7d0 2024-08-20T01:16:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ce400640 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009e20 tx=0x7f06c0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790551ab34562275 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ba000640 1 -- 172.21.15.138:0/2976152368 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0017040 con 0x559dceafa7d0 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ba000640 1 -- 172.21.15.138:0/2976152368 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c002c070 con 0x559dceafa7d0 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.835+0000 7f06ba000640 1 -- 172.21.15.138:0/2976152368 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c0003970 con 0x559dceafa7d0 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- 172.21.15.138:0/2976152368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 msgr2=0x559dceafe920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f06c0009e20 tx=0x7f06c0009500 comp rx=0 tx=0).stop 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- 172.21.15.138:0/2976152368 shutdown_connections 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceb07080 0x559dceb1f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceafb0f0 0x559dceb1ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 --2- 172.21.15.138:0/2976152368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceafe920 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- 172.21.15.138:0/2976152368 >> 172.21.15.138:0/2976152368 conn(0x559dceaf8600 msgr2=0x559dceb05e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- 172.21.15.138:0/2976152368 shutdown_connections 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- 172.21.15.138:0/2976152368 wait complete. 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.836+0000 7f06d328d740 1 -- start start 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 Processor -- start 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb0a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b02a0 con 0x559dceafa7d0 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b1710 con 0x559dceafb0f0 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f06c81b0970 con 0x559dceb07080 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06d328d740 10 client.0 _ll_register_callbacks cb 0x7f06b4009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06ce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06ce400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50954/0 (socket says 172.21.15.138:50954) 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06ce400640 1 -- 172.21.15.138:0/1411616445 learned_addr learned my addr 172.21.15.138:0/1411616445 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06cd000640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb0a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.837+0000 7f06ce400640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 -- 172.21.15.138:0/1411616445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 msgr2=0x559dceb0ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb0ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 -- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 msgr2=0x559dceb0a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb0a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 -- 172.21.15.138:0/1411616445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dceb7fff0 con 0x559dceafa7d0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06ce400640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f06c0029040 tx=0x7f06c0009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a049a0e3029f5c39 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06af400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:16:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06c001a960 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9541+0+0 (secure 0 0 0) 0x7f06c0004a00 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06c002c070 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06c0004c00 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06d328d740 10 client.16431 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.838+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559dceb81300 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06c002c070 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f06c0026040 con 0x559dceafa7d0 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 10 client.16431 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 10 client.16431 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 20 client.16431 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb81820 0x559dceb83c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 4 client.16431 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dcea6dc00 con 0x559dceb81820 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.839+0000 7f06cd000640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb81820 0x559dceb83c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.840+0000 7f06cd000640 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb81820 0x559dceb83c10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c67ad1d43b6830 server_cookie=95db5d1c9210e4ff in_seq=0 out_seq=0 2024-08-20T01:16:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.840+0000 7f06af400640 10 client.16431 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06af400640 1 -- 172.21.15.138:0/1411616445 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559dcea6dc00 con 0x559dceb81820 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06af400640 10 client.16431 handle_command_reply: tid=0 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 client.16431 shutdown 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 msgr2=0x559dceb0a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f06c0029040 tx=0x7f06c0009940 comp rx=0 tx=0).stop 2024-08-20T01:16:07.887 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 shutdown_connections 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559dceb81820 0x559dceb83c10 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.888+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559dceb07080 0x559dceb0ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559dceafb0f0 0x559dceb0a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 1 --2- 172.21.15.138:0/1411616445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559dceafa7d0 0x559dceb0a0e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06af400640 0 client.16431 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 >> 172.21.15.138:0/1411616445 conn(0x559dceaf8600 msgr2=0x559dceaf92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 shutdown_connections 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 1 -- 172.21.15.138:0/1411616445 wait complete. 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06d328d740 20 client.16431 trim_cache size 0 max 16384 2024-08-20T01:16:07.888 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:16:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:16:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:16:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:16:07.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 msgr2=0x7f06b00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.889+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 0x7f06b00b6200 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f06bc101f90 tx=0x7f06bc1086d0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 msgr2=0x7f06c819afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f06bc005ab0 tx=0x7f06bc003c10 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 shutdown_connections 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f06b00b3d40 0x7f06b00b6200 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f06c810ad90 0x7f06c81a0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06c810a410 0x7f06c819b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 --2- 172.21.15.138:0/3873103318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f06c8104a10 0x7f06c819afa0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 >> 172.21.15.138:0/3873103318 conn(0x7f06c8100450 msgr2=0x7f06c8102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 shutdown_connections 2024-08-20T01:16:07.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:07.890+0000 7f06cee00640 1 -- 172.21.15.138:0/3873103318 wait complete. 2024-08-20T01:16:08.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 328817 3.98 MB/sec execute 260 sec latency 3300.245 ms 2024-08-20T01:16:09.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:08 smithi138 ceph-mon[18076]: pgmap v607: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-08-20T01:16:09.170 INFO:tasks.workunit.client.0.smithi138.stdout: 1 330562 4.01 MB/sec execute 261 sec latency 23.336 ms 2024-08-20T01:16:09.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:08 smithi167 ceph-mon[21965]: pgmap v607: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-08-20T01:16:09.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:08 smithi142 ceph-mon[21799]: pgmap v607: 129 pgs: 2 active+clean+snaptrim, 127 active+clean; 223 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-08-20T01:16:10.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 331475 4.01 MB/sec execute 262 sec latency 362.096 ms 2024-08-20T01:16:11.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:10 smithi138 ceph-mon[18076]: pgmap v608: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-08-20T01:16:11.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 331589 4.00 MB/sec execute 263 sec latency 778.866 ms 2024-08-20T01:16:11.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:10 smithi167 ceph-mon[21965]: pgmap v608: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-08-20T01:16:11.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:10 smithi142 ceph-mon[21799]: pgmap v608: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-08-20T01:16:12.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 333811 4.03 MB/sec execute 264 sec latency 21.328 ms 2024-08-20T01:16:13.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:12 smithi138 ceph-mon[18076]: pgmap v609: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-08-20T01:16:13.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 335120 4.04 MB/sec execute 265 sec latency 173.786 ms 2024-08-20T01:16:13.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:12 smithi167 ceph-mon[21965]: pgmap v609: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-08-20T01:16:13.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:12 smithi142 ceph-mon[21799]: pgmap v609: 129 pgs: 129 active+clean; 226 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 208 op/s 2024-08-20T01:16:14.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 335120 4.02 MB/sec execute 266 sec latency 1173.888 ms 2024-08-20T01:16:15.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 336980 4.05 MB/sec execute 267 sec latency 1244.292 ms 2024-08-20T01:16:15.605 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:15 smithi138 ceph-mon[18076]: osdmap e93: 12 total, 12 up, 12 in 2024-08-20T01:16:15.605 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:15 smithi138 ceph-mon[18076]: pgmap v611: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-08-20T01:16:15.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:15 smithi167 ceph-mon[21965]: osdmap e93: 12 total, 12 up, 12 in 2024-08-20T01:16:15.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:15 smithi167 ceph-mon[21965]: pgmap v611: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-08-20T01:16:15.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:15 smithi142 ceph-mon[21799]: osdmap e93: 12 total, 12 up, 12 in 2024-08-20T01:16:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:15 smithi142 ceph-mon[21799]: pgmap v611: 129 pgs: 129 active+clean; 229 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 176 op/s 2024-08-20T01:16:16.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 337767 4.05 MB/sec execute 268 sec latency 50.835 ms 2024-08-20T01:16:17.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:16 smithi138 ceph-mon[18076]: pgmap v612: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-08-20T01:16:17.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 338670 4.05 MB/sec execute 269 sec latency 578.691 ms 2024-08-20T01:16:17.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:16 smithi167 ceph-mon[21965]: pgmap v612: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-08-20T01:16:17.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:16 smithi142 ceph-mon[21799]: pgmap v612: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 158 op/s 2024-08-20T01:16:18.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 340532 4.08 MB/sec execute 270 sec latency 961.480 ms 2024-08-20T01:16:19.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:18 smithi138 ceph-mon[18076]: pgmap v613: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T01:16:19.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 341928 4.09 MB/sec execute 271 sec latency 25.655 ms 2024-08-20T01:16:19.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:18 smithi167 ceph-mon[21965]: pgmap v613: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T01:16:19.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:18 smithi142 ceph-mon[21799]: pgmap v613: 129 pgs: 129 active+clean; 228 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 156 op/s 2024-08-20T01:16:20.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 342210 4.08 MB/sec execute 272 sec latency 782.219 ms 2024-08-20T01:16:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 -- 172.21.15.138:0/1522816073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c104c00 msgr2=0x7f662c104fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 --2- 172.21.15.138:0/1522816073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c104c00 0x7f662c104fe0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f661c009900 tx=0x7f661c02a410 comp rx=0 tx=0).stop 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 -- 172.21.15.138:0/1522816073 shutdown_connections 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 --2- 172.21.15.138:0/1522816073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c10af80 0x7f662c111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 --2- 172.21.15.138:0/1522816073 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10a600 0x7f662c10aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 --2- 172.21.15.138:0/1522816073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c104c00 0x7f662c104fe0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 -- 172.21.15.138:0/1522816073 >> 172.21.15.138:0/1522816073 conn(0x7f662c100640 msgr2=0x7f662c102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 -- 172.21.15.138:0/1522816073 shutdown_connections 2024-08-20T01:16:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.002+0000 7f6632400640 1 -- 172.21.15.138:0/1522816073 wait complete. 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 Processor -- start 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 -- start start 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c104c00 0x7f662c19b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10af80 0x7f662c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f662c114c60 con 0x7f662c10a600 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f662c114ae0 con 0x7f662c104c00 2024-08-20T01:16:21.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.003+0000 7f6632400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f662c114de0 con 0x7f662c10af80 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47210/0 (socket says 172.21.15.138:47210) 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 -- 172.21.15.138:0/2038624065 learned_addr learned my addr 172.21.15.138:0/2038624065 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f662be00640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10af80 0x7f662c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f662b400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c104c00 0x7f662c19b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 -- 172.21.15.138:0/2038624065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10af80 msgr2=0x7f662c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10af80 0x7f662c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 -- 172.21.15.138:0/2038624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c104c00 msgr2=0x7f662c19b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c104c00 0x7f662c19b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 -- 172.21.15.138:0/2038624065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f662c1a1240 con 0x7f662c10a600 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.004+0000 7f6623e00640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f661400d950 tx=0x7f661400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.005+0000 7f6628c00640 1 -- 172.21.15.138:0/2038624065 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6614017030 con 0x7f662c10a600 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.005+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f662c1a1530 con 0x7f662c10a600 2024-08-20T01:16:21.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.005+0000 7f6628c00640 1 -- 172.21.15.138:0/2038624065 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6614004020 con 0x7f662c10a600 2024-08-20T01:16:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.005+0000 7f6628c00640 1 -- 172.21.15.138:0/2038624065 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66140047a0 con 0x7f662c10a600 2024-08-20T01:16:21.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.005+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f662c1a7bf0 con 0x7f662c10a600 2024-08-20T01:16:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 Processor -- start 2024-08-20T01:16:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 -- start start 2024-08-20T01:16:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x56071827dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718289ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x56071828c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5607181f3210 con 0x56071827e250 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5607181e2710 con 0x56071827d930 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6635220740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5607181e6750 con 0x56071828a1e0 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f6628c00640 1 -- 172.21.15.138:0/2038624065 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6614009a30 con 0x7f662c10a600 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718289ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718289ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47216/0 (socket says 172.21.15.138:47216) 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f662be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x56071827dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.008+0000 7f662b400640 1 -- 172.21.15.138:0/4288515642 learned_addr learned my addr 172.21.15.138:0/4288515642 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f6623e00640 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x56071828c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 -- 172.21.15.138:0/4288515642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 msgr2=0x56071828c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x56071828c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 -- 172.21.15.138:0/4288515642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 msgr2=0x560718289ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718289ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 -- 172.21.15.138:0/4288515642 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607181aede0 con 0x56071827d930 2024-08-20T01:16:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f662be00640 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x56071827dd10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6624009f00 tx=0x7f6624011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc37adb18619e2c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f6620c00640 1 -- 172.21.15.138:0/4288515642 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624004d20 con 0x56071827d930 2024-08-20T01:16:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f6620c00640 1 -- 172.21.15.138:0/4288515642 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6624004050 con 0x56071827d930 2024-08-20T01:16:21.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.009+0000 7f6620c00640 1 -- 172.21.15.138:0/4288515642 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6624005280 con 0x56071827d930 2024-08-20T01:16:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6628c00640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 0x7f66100b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 -- 172.21.15.138:0/4288515642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 msgr2=0x56071827dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x56071827dd10 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6624009f00 tx=0x7f6624011810 comp rx=0 tx=0).stop 2024-08-20T01:16:21.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 -- 172.21.15.138:0/4288515642 shutdown_connections 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x56071828c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718289ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 --2- 172.21.15.138:0/4288515642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x56071827dd10 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 -- 172.21.15.138:0/4288515642 >> 172.21.15.138:0/4288515642 conn(0x56071827b760 msgr2=0x56071827c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6628c00640 1 -- 172.21.15.138:0/2038624065 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f66140e3da0 con 0x7f662c10a600 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 -- 172.21.15.138:0/4288515642 shutdown_connections 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f6635220740 1 -- 172.21.15.138:0/4288515642 wait complete. 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.010+0000 7f662b400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 0x7f66100b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 -- start start 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 Processor -- start 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f662b400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 0x7f66100b6200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f661c02a930 tx=0x7f661c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x560718289a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718287390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56071811bab0 con 0x56071827e250 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560718189140 con 0x56071827d930 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.011+0000 7f6635220740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56071828cec0 con 0x56071828a1e0 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6635220740 10 client.0 _ll_register_callbacks cb 0x56071819c5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47230/0 (socket says 172.21.15.138:47230) 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 -- 172.21.15.138:0/215159031 learned_addr learned my addr 172.21.15.138:0/215159031 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f662b400640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x560718289a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f662be00640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718287390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 -- 172.21.15.138:0/215159031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 msgr2=0x560718287390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718287390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 -- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 msgr2=0x560718289a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x560718289a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 -- 172.21.15.138:0/215159031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56071818a170 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6623e00640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66140aa2b0 tx=0x7f66140a9ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d10ea732f06993 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.012+0000 7f6619600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66140aa5b0 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f66140041d0 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6624004050 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f66140aebb0 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 10 client.16443 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560717f45790 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6624004050 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f66140e1400 con 0x56071827e250 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 10 client.16443 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:21.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 20 client.16443 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718299d90 0x56071829c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 4 client.16443 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.013+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607181f5160 con 0x560718299d90 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.014+0000 7f662b400640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718299d90 0x56071829c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.014+0000 7f662b400640 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718299d90 0x56071829c180 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0311f217146f79 server_cookie=ec9effedc2f04800 in_seq=0 out_seq=0 2024-08-20T01:16:21.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.014+0000 7f6619600640 10 client.16443 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6619600640 1 -- 172.21.15.138:0/215159031 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607181f5160 con 0x560718299d90 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6619600640 10 client.16443 handle_command_reply: tid=0 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 client.16443 shutdown 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 msgr2=0x560718286e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f66140aa2b0 tx=0x7f66140a9ff0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 shutdown_connections 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718299d90 0x56071829c180 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718287390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6619600640 0 client.16443 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827e250 0x560718286e50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 --2- 172.21.15.138:0/215159031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827d930 0x560718289a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 >> 172.21.15.138:0/215159031 conn(0x56071827b760 msgr2=0x560718281320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 shutdown_connections 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 1 -- 172.21.15.138:0/215159031 wait complete. 2024-08-20T01:16:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.015+0000 7f6635220740 20 client.16443 trim_cache size 0 max 16384 2024-08-20T01:16:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.030+0000 7f6635220740 1 Processor -- start 2024-08-20T01:16:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.030+0000 7f6635220740 1 -- start start 2024-08-20T01:16:21.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.030+0000 7f6635220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.030+0000 7f6635220740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.030+0000 7f6635220740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6635220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1b46c0 con 0x56071827d930 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6635220740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1b1f90 con 0x56071827e250 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6635220740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1afe00 con 0x56071828a1e0 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6623e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47240/0 (socket says 172.21.15.138:47240) 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6623e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36548/0 (socket says 172.21.15.138:36548) 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 -- 172.21.15.138:0/255672871 learned_addr learned my addr 172.21.15.138:0/255672871 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662be00640 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 -- 172.21.15.138:0/255672871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 msgr2=0x560718284060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 -- 172.21.15.138:0/255672871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 msgr2=0x5607182a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 -- 172.21.15.138:0/255672871 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560717fbe620 con 0x56071827d930 2024-08-20T01:16:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f6623e00640 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:21.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.031+0000 7f662b400640 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f661c009230 tx=0x7f661c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=972b0217b483498b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f661be00640 1 -- 172.21.15.138:0/255672871 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c035040 con 0x56071827d930 2024-08-20T01:16:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f661be00640 1 -- 172.21.15.138:0/255672871 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f661c004e20 con 0x56071827d930 2024-08-20T01:16:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f661be00640 1 -- 172.21.15.138:0/255672871 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c054330 con 0x56071827d930 2024-08-20T01:16:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 -- 172.21.15.138:0/255672871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 msgr2=0x5607182a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f661c009230 tx=0x7f661c03f000 comp rx=0 tx=0).stop 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 -- 172.21.15.138:0/255672871 shutdown_connections 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x5607182a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 --2- 172.21.15.138:0/255672871 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x5607182a1a50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 -- 172.21.15.138:0/255672871 >> 172.21.15.138:0/255672871 conn(0x56071827b760 msgr2=0x56071827c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 -- 172.21.15.138:0/255672871 shutdown_connections 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.032+0000 7f6635220740 1 -- 172.21.15.138:0/255672871 wait complete. 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 -- start start 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 Processor -- start 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x560718283b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1b0450 con 0x56071827d930 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1b18c0 con 0x56071827e250 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f662c1b0b20 con 0x56071828a1e0 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.033+0000 7f6635220740 10 client.0 _ll_register_callbacks cb 0x7f66140046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47250/0 (socket says 172.21.15.138:47250) 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 -- 172.21.15.138:0/3247920728 learned_addr learned my addr 172.21.15.138:0/3247920728 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662be00640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f6623e00640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x560718283b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 -- 172.21.15.138:0/3247920728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 msgr2=0x560718284080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 -- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 msgr2=0x560718283b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x560718283b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 -- 172.21.15.138:0/3247920728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560718284810 con 0x56071827d930 2024-08-20T01:16:21.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f662b400640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f661c0092a0 tx=0x7f661c003eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2923915697a7f400 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f6619600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.034+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c004770 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f661c004910 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f661c004e20 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f661c06f050 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 10 client.16449 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560718304a00 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f661c004e20 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f661c054ad0 con 0x56071827d930 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 10 client.16449 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 10 client.16449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 20 client.16449 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:21.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718305110 0x560718307530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 4 client.16449 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:16:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.035+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5607181f0c00 con 0x560718305110 2024-08-20T01:16:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.036+0000 7f6623e00640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718305110 0x560718307530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.036+0000 7f6623e00640 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718305110 0x560718307530 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea81b7ebaaa06f41 server_cookie=303e03ca2dde2b2c in_seq=0 out_seq=0 2024-08-20T01:16:21.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.036+0000 7f6619600640 10 client.16449 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6619600640 1 -- 172.21.15.138:0/3247920728 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5607181f0c00 con 0x560718305110 2024-08-20T01:16:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6619600640 10 client.16449 handle_command_reply: tid=0 2024-08-20T01:16:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 client.16449 shutdown 2024-08-20T01:16:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 msgr2=0x560718292780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f661c0092a0 tx=0x7f661c003eb0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 shutdown_connections 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560718305110 0x560718307530 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56071828a1e0 0x560718284080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56071827e250 0x560718283b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 --2- 172.21.15.138:0/3247920728 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56071827d930 0x560718292780 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6619600640 0 client.16449 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 >> 172.21.15.138:0/3247920728 conn(0x56071827b760 msgr2=0x560718281bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 shutdown_connections 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 1 -- 172.21.15.138:0/3247920728 wait complete. 2024-08-20T01:16:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.093+0000 7f6635220740 20 client.16449 trim_cache size 0 max 16384 2024-08-20T01:16:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:16:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:16:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:16:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 msgr2=0x7f66100b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 0x7f66100b6200 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f661c02a930 tx=0x7f661c005a20 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 msgr2=0x7f662c19b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f661400d950 tx=0x7f661400de10 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 shutdown_connections 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f66100b3d40 0x7f66100b6200 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f662c10af80 0x7f662c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f662c10a600 0x7f662c19b6e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 --2- 172.21.15.138:0/2038624065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f662c104c00 0x7f662c19b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 >> 172.21.15.138:0/2038624065 conn(0x7f662c100640 msgr2=0x7f662c102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.094+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 shutdown_connections 2024-08-20T01:16:21.095 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:21.095+0000 7f6632400640 1 -- 172.21.15.138:0/2038624065 wait complete. 2024-08-20T01:16:21.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 342392 4.06 MB/sec execute 273 sec latency 1678.129 ms 2024-08-20T01:16:21.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:20 smithi167 ceph-mon[21965]: pgmap v614: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-08-20T01:16:21.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:20 smithi142 ceph-mon[21799]: pgmap v614: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-08-20T01:16:21.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:20 smithi138 ceph-mon[18076]: pgmap v614: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.5 MiB/s wr, 179 op/s 2024-08-20T01:16:22.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 344428 4.10 MB/sec execute 274 sec latency 161.080 ms 2024-08-20T01:16:23.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:22 smithi138 ceph-mon[18076]: pgmap v615: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-08-20T01:16:23.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 345759 4.10 MB/sec execute 275 sec latency 20.814 ms 2024-08-20T01:16:23.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:22 smithi167 ceph-mon[21965]: pgmap v615: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-08-20T01:16:23.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:22 smithi142 ceph-mon[21799]: pgmap v615: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 177 op/s 2024-08-20T01:16:24.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 345759 4.09 MB/sec execute 276 sec latency 1002.002 ms 2024-08-20T01:16:25.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:24 smithi138 ceph-mon[18076]: pgmap v616: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-08-20T01:16:25.171 INFO:tasks.workunit.client.0.smithi138.stdout: 1 346102 4.07 MB/sec execute 277 sec latency 1062.921 ms 2024-08-20T01:16:25.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:24 smithi167 ceph-mon[21965]: pgmap v616: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-08-20T01:16:25.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:24 smithi142 ceph-mon[21799]: pgmap v616: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.9 MiB/s wr, 234 op/s 2024-08-20T01:16:26.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 346508 4.06 MB/sec execute 278 sec latency 610.966 ms 2024-08-20T01:16:27.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:26 smithi138 ceph-mon[18076]: pgmap v617: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-08-20T01:16:27.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 348287 4.09 MB/sec execute 279 sec latency 20.183 ms 2024-08-20T01:16:27.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:26 smithi167 ceph-mon[21965]: pgmap v617: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-08-20T01:16:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:26 smithi142 ceph-mon[21799]: pgmap v617: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.2 MiB/s wr, 187 op/s 2024-08-20T01:16:28.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 349302 4.10 MB/sec execute 280 sec latency 232.467 ms 2024-08-20T01:16:29.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:28 smithi138 ceph-mon[18076]: pgmap v618: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-08-20T01:16:29.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 349302 4.08 MB/sec execute 281 sec latency 1232.508 ms 2024-08-20T01:16:29.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:28 smithi167 ceph-mon[21965]: pgmap v618: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-08-20T01:16:29.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:28 smithi142 ceph-mon[21799]: pgmap v618: 129 pgs: 129 active+clean; 237 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-08-20T01:16:30.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 351529 4.12 MB/sec execute 282 sec latency 1298.080 ms 2024-08-20T01:16:31.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:30 smithi138 ceph-mon[18076]: pgmap v619: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 249 op/s 2024-08-20T01:16:31.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 351874 4.10 MB/sec execute 283 sec latency 653.641 ms 2024-08-20T01:16:31.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:30 smithi167 ceph-mon[21965]: pgmap v619: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 249 op/s 2024-08-20T01:16:31.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:30 smithi142 ceph-mon[21799]: pgmap v619: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.5 MiB/s wr, 249 op/s 2024-08-20T01:16:32.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 352850 4.11 MB/sec execute 284 sec latency 504.247 ms 2024-08-20T01:16:33.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:32 smithi138 ceph-mon[18076]: pgmap v620: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:16:33.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 354774 4.13 MB/sec execute 285 sec latency 888.121 ms 2024-08-20T01:16:33.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:32 smithi167 ceph-mon[21965]: pgmap v620: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:16:33.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:32 smithi142 ceph-mon[21799]: pgmap v620: 129 pgs: 129 active+clean; 240 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:16:34.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 355959 4.13 MB/sec execute 286 sec latency 15.802 ms 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.219+0000 7f87b5000640 1 -- 172.21.15.138:0/2791097645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 msgr2=0x7f87b0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.219+0000 7f87b5000640 1 --2- 172.21.15.138:0/2791097645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0102c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f87a0009900 tx=0x7f87a002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 -- 172.21.15.138:0/2791097645 shutdown_connections 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 --2- 172.21.15.138:0/2791097645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b0108c30 0x7f87b010f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 --2- 172.21.15.138:0/2791097645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b01082b0 0x7f87b01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 --2- 172.21.15.138:0/2791097645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0102c90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 -- 172.21.15.138:0/2791097645 >> 172.21.15.138:0/2791097645 conn(0x7f87b00fe2d0 msgr2=0x7f87b01006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 -- 172.21.15.138:0/2791097645 shutdown_connections 2024-08-20T01:16:34.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.220+0000 7f87b5000640 1 -- 172.21.15.138:0/2791097645 wait complete. 2024-08-20T01:16:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 Processor -- start 2024-08-20T01:16:34.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 -- start start 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b01082b0 0x7f87b01993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b0108c30 0x7f87b019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0112940 con 0x7f87b0108c30 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01127c0 con 0x7f87b01028b0 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.221+0000 7f87b5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b0112ac0 con 0x7f87b01082b0 2024-08-20T01:16:34.222 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56858/0 (socket says 172.21.15.138:56858) 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 -- 172.21.15.138:0/4281961359 learned_addr learned my addr 172.21.15.138:0/4281961359 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87afe00640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b0108c30 0x7f87b019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87aea00640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b01082b0 0x7f87b01993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 -- 172.21.15.138:0/4281961359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b01082b0 msgr2=0x7f87b01993a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b01082b0 0x7f87b01993a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 -- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b0108c30 msgr2=0x7f87b019e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b0108c30 0x7f87b019e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 -- 172.21.15.138:0/4281961359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87b019ef00 con 0x7f87b01028b0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.222+0000 7f87af400640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005ab0 tx=0x7f87a0003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.223+0000 7f87a7e00640 1 -- 172.21.15.138:0/4281961359 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a003f070 con 0x7f87b01028b0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.223+0000 7f87a7e00640 1 -- 172.21.15.138:0/4281961359 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a0031d60 con 0x7f87b01028b0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.223+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87b019f190 con 0x7f87b01028b0 2024-08-20T01:16:34.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.223+0000 7f87a7e00640 1 -- 172.21.15.138:0/4281961359 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a0035030 con 0x7f87b01028b0 2024-08-20T01:16:34.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.223+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87b01a58b0 con 0x7f87b01028b0 2024-08-20T01:16:34.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.226+0000 7f87b9440740 1 Processor -- start 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 -- start start 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b41f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b41f7d0 0x55d00b421bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d00b384210 con 0x55d00b413840 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d00b373710 con 0x55d00b41f7d0 2024-08-20T01:16:34.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87b9440740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d00b377750 con 0x55d00b412f20 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87a7e00640 1 -- 172.21.15.138:0/4281961359 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f87a0042070 con 0x7f87b01028b0 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58762/0 (socket says 172.21.15.138:58762) 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 -- 172.21.15.138:0/2615893025 learned_addr learned my addr 172.21.15.138:0/2615893025 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87aea00640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b41f7d0 0x55d00b421bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87af400640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b41f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 -- 172.21.15.138:0/2615893025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b41f7d0 msgr2=0x55d00b421bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b41f7d0 0x55d00b421bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.227+0000 7f87afe00640 1 -- 172.21.15.138:0/2615893025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 msgr2=0x55d00b41f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87afe00640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b41f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87afe00640 1 -- 172.21.15.138:0/2615893025 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00b33fde0 con 0x55d00b412f20 2024-08-20T01:16:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87af400640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b41f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:16:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87afe00640 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87a8010370 tx=0x7f87a8010830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e58f13ba9298b41 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87a4c00640 1 -- 172.21.15.138:0/2615893025 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8016070 con 0x55d00b412f20 2024-08-20T01:16:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87a4c00640 1 -- 172.21.15.138:0/2615893025 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8004d20 con 0x55d00b412f20 2024-08-20T01:16:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.228+0000 7f87a4c00640 1 -- 172.21.15.138:0/2615893025 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8005350 con 0x55d00b412f20 2024-08-20T01:16:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87a7e00640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 0x7f87940b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 -- 172.21.15.138:0/2615893025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 msgr2=0x55d00b413300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87a8010370 tx=0x7f87a8010830 comp rx=0 tx=0).stop 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87a7e00640 1 -- 172.21.15.138:0/4281961359 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f87a0104b10 con 0x7f87b01028b0 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 -- 172.21.15.138:0/2615893025 shutdown_connections 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b41f7d0 0x55d00b421bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b41f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 --2- 172.21.15.138:0/2615893025 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b413300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87af400640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 0x7f87940b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 -- 172.21.15.138:0/2615893025 >> 172.21.15.138:0/2615893025 conn(0x55d00b410d50 msgr2=0x55d00b411ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.229+0000 7f87b9440740 1 -- 172.21.15.138:0/2615893025 shutdown_connections 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.230+0000 7f87b9440740 1 -- 172.21.15.138:0/2615893025 wait complete. 2024-08-20T01:16:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.230+0000 7f87af400640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 0x7f87940b6280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d00b4160a0 tx=0x7f87a0031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.230+0000 7f87b9440740 1 -- start start 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.230+0000 7f87b9440740 1 Processor -- start 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 0x55d00b41c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b41c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d00b2acab0 con 0x55d00b41f7d0 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d00b31a140 con 0x55d00b413840 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d00b4224b0 con 0x55d00b412f20 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87b9440740 10 client.0 _ll_register_callbacks cb 0x55d00b32d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58776/0 (socket says 172.21.15.138:58776) 2024-08-20T01:16:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87aea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 0x55d00b41c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87afe00640 1 -- 172.21.15.138:0/1489554485 learned_addr learned my addr 172.21.15.138:0/1489554485 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.231+0000 7f87afe00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b41c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 -- 172.21.15.138:0/1489554485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 msgr2=0x55d00b41efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 -- 172.21.15.138:0/1489554485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 msgr2=0x55d00b41c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b41c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 -- 172.21.15.138:0/1489554485 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00b31b170 con 0x55d00b413840 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87afe00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87aea00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 0x55d00b41c410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f879c0126d0 tx=0x7f879c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=790a0a5ea03760dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f8799600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c004050 con 0x55d00b413840 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87a8004d20 con 0x55d00b413840 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f879c0041f0 con 0x55d00b413840 2024-08-20T01:16:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f879c025050 con 0x55d00b413840 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87b9440740 10 client.26507 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.232+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d00b0d6790 con 0x55d00b413840 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87a8004d20 con 0x55d00b413840 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f879c003960 con 0x55d00b413840 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 10 client.26507 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 20 client.26507 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b42f410 0x55d00b431800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 4 client.26507 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d00b386160 con 0x55d00b42f410 2024-08-20T01:16:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.233+0000 7f87afe00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b42f410 0x55d00b431800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.234+0000 7f87afe00640 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b42f410 0x55d00b431800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=550dd2bc28d67c2f server_cookie=49ae57b7834993b8 in_seq=0 out_seq=0 2024-08-20T01:16:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.234+0000 7f8799600640 10 client.26507 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.234+0000 7f8799600640 1 -- 172.21.15.138:0/1489554485 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d00b386160 con 0x55d00b42f410 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.234+0000 7f8799600640 10 client.26507 handle_command_reply: tid=0 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 client.26507 shutdown 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 msgr2=0x55d00b41c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 0x55d00b41c410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f879c0126d0 tx=0x7f879c012b90 comp rx=0 tx=0).stop 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 shutdown_connections 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b42f410 0x55d00b431800 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b41c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b413840 0x55d00b41c410 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f8799600640 0 client.26507 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 --2- 172.21.15.138:0/1489554485 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b412f20 0x55d00b41efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 >> 172.21.15.138:0/1489554485 conn(0x55d00b410d50 msgr2=0x55d00b4168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 shutdown_connections 2024-08-20T01:16:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 1 -- 172.21.15.138:0/1489554485 wait complete. 2024-08-20T01:16:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.235+0000 7f87b9440740 20 client.26507 trim_cache size 0 max 16384 2024-08-20T01:16:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 Processor -- start 2024-08-20T01:16:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 -- start start 2024-08-20T01:16:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b4124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b41f7d0 0x55d00b437610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01b00a0 con 0x55d00b413840 2024-08-20T01:16:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01ad970 con 0x55d00b412f20 2024-08-20T01:16:34.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.253+0000 7f87b9440740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01ab7e0 con 0x55d00b41f7d0 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87afe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b41f7d0 0x55d00b437610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60758/0 (socket says 172.21.15.138:60758) 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 -- 172.21.15.138:0/3742232072 learned_addr learned my addr 172.21.15.138:0/3742232072 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87afe00640 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b4124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 -- 172.21.15.138:0/3742232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b41f7d0 msgr2=0x55d00b437610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b41f7d0 0x55d00b437610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 -- 172.21.15.138:0/3742232072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 msgr2=0x55d00b4124b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b4124b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 -- 172.21.15.138:0/3742232072 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00b431b70 con 0x55d00b413840 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f87aea00640 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f78d0cb6bc4df42 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.254+0000 7f879be00640 1 -- 172.21.15.138:0/3742232072 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0035f0 con 0x55d00b413840 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f879be00640 1 -- 172.21.15.138:0/3742232072 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879c02e070 con 0x55d00b413840 2024-08-20T01:16:34.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f879be00640 1 -- 172.21.15.138:0/3742232072 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0050e0 con 0x55d00b413840 2024-08-20T01:16:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 -- 172.21.15.138:0/3742232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 msgr2=0x55d00b4370d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f879c000c00 tx=0x7f879c0106c0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 -- 172.21.15.138:0/3742232072 shutdown_connections 2024-08-20T01:16:34.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b41f7d0 0x55d00b437610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b413840 0x55d00b4370d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 --2- 172.21.15.138:0/3742232072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b4124b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.255+0000 7f87b9440740 1 -- 172.21.15.138:0/3742232072 >> 172.21.15.138:0/3742232072 conn(0x55d00b410d50 msgr2=0x55d00b411da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.256+0000 7f87b9440740 1 -- 172.21.15.138:0/3742232072 shutdown_connections 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.256+0000 7f87b9440740 1 -- 172.21.15.138:0/3742232072 wait complete. 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.256+0000 7f87b9440740 1 -- start start 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 Processor -- start 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b40e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b40f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01abe30 con 0x55d00b41f7d0 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01ad2a0 con 0x55d00b412f20 2024-08-20T01:16:34.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f87b01ac500 con 0x55d00b413840 2024-08-20T01:16:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87b9440740 10 client.0 _ll_register_callbacks cb 0x7f879c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87aea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58790/0 (socket says 172.21.15.138:58790) 2024-08-20T01:16:34.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.257+0000 7f87aea00640 1 -- 172.21.15.138:0/2535079193 learned_addr learned my addr 172.21.15.138:0/2535079193 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87afe00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b40e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87afe00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b40f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 -- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 msgr2=0x55d00b40e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b40e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 -- 172.21.15.138:0/2535079193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 msgr2=0x55d00b40f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b40f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 -- 172.21.15.138:0/2535079193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00b14f620 con 0x55d00b413840 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87afe00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b40f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87aea00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f879c00fe30 tx=0x7f879c00fb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73dcf47e0efce583 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f8799600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:16:34.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c003800 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879c02e070 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.258+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f879c018040 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f879c03e050 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 10 client.26485 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d00b427b70 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f879c02e070 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f879c010730 con 0x55d00b413840 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 10 client.26485 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 10 client.26485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 20 client.26485 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b49e890 0x55d00b4a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 4 client.26485 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d00b381c00 con 0x55d00b49e890 2024-08-20T01:16:34.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.259+0000 7f87afe00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b49e890 0x55d00b4a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.260+0000 7f87afe00640 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b49e890 0x55d00b4a0c80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e46feaf0cd96b779 server_cookie=54fffac70f53b04f in_seq=0 out_seq=0 2024-08-20T01:16:34.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.260+0000 7f8799600640 10 client.26485 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.303+0000 7f8799600640 1 -- 172.21.15.138:0/2535079193 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d00b381c00 con 0x55d00b49e890 2024-08-20T01:16:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.303+0000 7f8799600640 10 client.26485 handle_command_reply: tid=0 2024-08-20T01:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.303+0000 7f87b9440740 1 client.26485 shutdown 2024-08-20T01:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 msgr2=0x55d00b40ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f879c00fe30 tx=0x7f879c00fb30 comp rx=0 tx=0).stop 2024-08-20T01:16:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 shutdown_connections 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d00b49e890 0x55d00b4a0c80 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d00b41f7d0 0x55d00b40f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d00b413840 0x55d00b40ee30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 --2- 172.21.15.138:0/2535079193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d00b412f20 0x55d00b40e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f8799600640 0 client.26485 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 >> 172.21.15.138:0/2535079193 conn(0x55d00b410d50 msgr2=0x55d00b41da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 shutdown_connections 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 1 -- 172.21.15.138:0/2535079193 wait complete. 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.304+0000 7f87b9440740 20 client.26485 trim_cache size 0 max 16384 2024-08-20T01:16:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:16:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 msgr2=0x7f87940b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 0x7f87940b6280 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55d00b4160a0 tx=0x7f87a0031400 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 msgr2=0x7f87b0198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87a0005ab0 tx=0x7f87a0003c80 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 shutdown_connections 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f87940b3dc0 0x7f87940b6280 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87b0108c30 0x7f87b019e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f87b01082b0 0x7f87b01993a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 --2- 172.21.15.138:0/4281961359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f87b01028b0 0x7f87b0198e60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 >> 172.21.15.138:0/4281961359 conn(0x7f87b00fe2d0 msgr2=0x7f87b01006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 shutdown_connections 2024-08-20T01:16:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:34.305+0000 7f87b5000640 1 -- 172.21.15.138:0/4281961359 wait complete. 2024-08-20T01:16:35.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:34 smithi138 ceph-mon[18076]: pgmap v621: 129 pgs: 129 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2024-08-20T01:16:35.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 356392 4.13 MB/sec execute 287 sec latency 575.080 ms 2024-08-20T01:16:35.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:34 smithi167 ceph-mon[21965]: pgmap v621: 129 pgs: 129 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2024-08-20T01:16:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:34 smithi142 ceph-mon[21799]: pgmap v621: 129 pgs: 129 active+clean; 247 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 242 op/s 2024-08-20T01:16:36.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 356392 4.11 MB/sec execute 288 sec latency 1575.171 ms 2024-08-20T01:16:37.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:36 smithi138 ceph-mon[18076]: pgmap v622: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-20T01:16:37.172 INFO:tasks.workunit.client.0.smithi138.stdout: 1 356392 4.10 MB/sec execute 289 sec latency 2575.284 ms 2024-08-20T01:16:37.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:36 smithi167 ceph-mon[21965]: pgmap v622: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-20T01:16:37.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:36 smithi142 ceph-mon[21799]: pgmap v622: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-08-20T01:16:38.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 356392 4.09 MB/sec execute 290 sec latency 3575.451 ms 2024-08-20T01:16:39.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:38 smithi138 ceph-mon[18076]: pgmap v623: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-08-20T01:16:39.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 356392 4.07 MB/sec execute 291 sec latency 4575.609 ms 2024-08-20T01:16:39.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:38 smithi167 ceph-mon[21965]: pgmap v623: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-08-20T01:16:39.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:38 smithi142 ceph-mon[21799]: pgmap v623: 129 pgs: 129 active+clean; 249 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-08-20T01:16:40.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 358569 4.10 MB/sec execute 292 sec latency 4640.385 ms 2024-08-20T01:16:41.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:40 smithi138 ceph-mon[18076]: pgmap v624: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-20T01:16:41.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 358969 4.09 MB/sec execute 293 sec latency 471.176 ms 2024-08-20T01:16:41.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:40 smithi167 ceph-mon[21965]: pgmap v624: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-20T01:16:41.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:40 smithi142 ceph-mon[21799]: pgmap v624: 129 pgs: 129 active+clean; 248 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-08-20T01:16:42.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 359941 4.10 MB/sec execute 294 sec latency 363.124 ms 2024-08-20T01:16:43.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:42 smithi138 ceph-mon[18076]: pgmap v625: 129 pgs: 129 active+clean; 247 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-20T01:16:43.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 361800 4.12 MB/sec execute 295 sec latency 750.668 ms 2024-08-20T01:16:43.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:42 smithi167 ceph-mon[21965]: pgmap v625: 129 pgs: 129 active+clean; 247 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-20T01:16:43.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:42 smithi142 ceph-mon[21799]: pgmap v625: 129 pgs: 129 active+clean; 247 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-08-20T01:16:44.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 363191 4.13 MB/sec execute 296 sec latency 25.560 ms 2024-08-20T01:16:45.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:44 smithi138 ceph-mon[18076]: pgmap v626: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-08-20T01:16:45.042 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:44 smithi142 ceph-mon[21799]: pgmap v626: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-08-20T01:16:45.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 363483 4.12 MB/sec execute 297 sec latency 396.287 ms 2024-08-20T01:16:45.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:44 smithi167 ceph-mon[21965]: pgmap v626: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 181 op/s 2024-08-20T01:16:45.282 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-20T01:16:45.283 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.85 2024-08-20T01:16:45.283 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:16:45.718 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:16:46.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 363483 4.11 MB/sec execute 298 sec latency 1396.351 ms 2024-08-20T01:16:46.819 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:46 smithi138 ceph-mon[18076]: pgmap v627: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:16:46.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 -- 172.21.15.138:0/1730784860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe388102ab0 msgr2=0x7fe388102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:46.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 --2- 172.21.15.138:0/1730784860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe388102ab0 0x7fe388102e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe378009860 tx=0x7fe37802a340 comp rx=0 tx=0).stop 2024-08-20T01:16:46.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 -- 172.21.15.138:0/1730784860 shutdown_connections 2024-08-20T01:16:46.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 --2- 172.21.15.138:0/1730784860 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388108e40 0x7fe38810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 --2- 172.21.15.138:0/1730784860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3881084c0 0x7fe388108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 --2- 172.21.15.138:0/1730784860 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe388102ab0 0x7fe388102e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 -- 172.21.15.138:0/1730784860 >> 172.21.15.138:0/1730784860 conn(0x7fe3880fe500 msgr2=0x7fe388100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 -- 172.21.15.138:0/1730784860 shutdown_connections 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.820+0000 7fe38d400640 1 -- 172.21.15.138:0/1730784860 wait complete. 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.821+0000 7fe38d400640 1 Processor -- start 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 -- start start 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 0x7fe38819b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe388112b00 con 0x7fe388108e40 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe388112980 con 0x7fe388102ab0 2024-08-20T01:16:46.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe38d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe388112c80 con 0x7fe3881084c0 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe387200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe385e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe385e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33410/0 (socket says 172.21.15.138:33410) 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe387200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37200/0 (socket says 172.21.15.138:37200) 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.822+0000 7fe385e00640 1 -- 172.21.15.138:0/2126304270 learned_addr learned my addr 172.21.15.138:0/2126304270 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe386800640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 0x7fe38819b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 -- 172.21.15.138:0/2126304270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 msgr2=0x7fe38819b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 0x7fe38819b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 -- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 msgr2=0x7fe3881a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 -- 172.21.15.138:0/2126304270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3881a1240 con 0x7fe3881084c0 2024-08-20T01:16:46.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe386800640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 0x7fe38819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe387200640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe385e00640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe37400e9c0 tx=0x7fe37400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37400ccc0 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3881a1530 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3740054d0 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374010430 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.823+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3881a7bf0 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.824+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe388102e90 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.826+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe374004f80 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.828+0000 7fe37f400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 0x7fe36c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.828+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7fe3740e48c0 con 0x7fe3881084c0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.828+0000 7fe386800640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 0x7fe36c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.829+0000 7fe386800640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 0x7fe36c0b6200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe37802a930 tx=0x7fe378030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:46.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:46.832+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fe3740ab7d0 con 0x7fe3881084c0 2024-08-20T01:16:47.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.061+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe38819c2c0 con 0x7fe3881084c0 2024-08-20T01:16:47.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.062+0000 7fe37f400640 1 -- 172.21.15.138:0/2126304270 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11932 (secure 0 0 0) 0x7fe3740ab0d0 con 0x7fe3881084c0 2024-08-20T01:16:47.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:16:47.061 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":39,"btime":"2024-08-20T01:16:04:725509+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:16:04.226894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:16:47.061 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 39 2024-08-20T01:16:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 msgr2=0x7fe36c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 0x7fe36c0b6200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe37802a930 tx=0x7fe378030340 comp rx=0 tx=0).stop 2024-08-20T01:16:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 msgr2=0x7fe38819b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe37400e9c0 tx=0x7fe37400ee80 comp rx=0 tx=0).stop 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 shutdown_connections 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe36c0b3d40 0x7fe36c0b6200 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe388108e40 0x7fe3881a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe3881084c0 0x7fe38819b6d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 --2- 172.21.15.138:0/2126304270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe388102ab0 0x7fe38819b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.065+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 >> 172.21.15.138:0/2126304270 conn(0x7fe3880fe500 msgr2=0x7fe3880ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.066+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 shutdown_connections 2024-08-20T01:16:47.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:47.066+0000 7fe38d400640 1 -- 172.21.15.138:0/2126304270 wait complete. 2024-08-20T01:16:47.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 363483 4.09 MB/sec execute 299 sec latency 2396.389 ms 2024-08-20T01:16:47.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:46 smithi167 ceph-mon[21965]: pgmap v627: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:16:47.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:46 smithi142 ceph-mon[21799]: pgmap v627: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 -- 172.21.15.138:0/1806258481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 msgr2=0x7f64d0075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 --2- 172.21.15.138:0/1806258481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d0075650 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f64c0009900 tx=0x7f64c002a410 comp rx=0 tx=0).stop 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 -- 172.21.15.138:0/1806258481 shutdown_connections 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 --2- 172.21.15.138:0/1806258481 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0113550 0x7f64d0115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 --2- 172.21.15.138:0/1806258481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0075b90 0x7f64d0113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 --2- 172.21.15.138:0/1806258481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d0075650 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.425+0000 7f64cfe00640 1 -- 172.21.15.138:0/1806258481 >> 172.21.15.138:0/1806258481 conn(0x7f64d00fe280 msgr2=0x7f64d01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.426+0000 7f64cfe00640 1 -- 172.21.15.138:0/1806258481 shutdown_connections 2024-08-20T01:16:47.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.426+0000 7f64cfe00640 1 -- 172.21.15.138:0/1806258481 wait complete. 2024-08-20T01:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.426+0000 7f64cfe00640 1 Processor -- start 2024-08-20T01:16:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 -- start start 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0075b90 0x7f64d019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0113550 0x7f64d01a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0118e00 con 0x7f64d0075270 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0118c80 con 0x7f64d0113550 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64d0118f80 con 0x7f64d0075b90 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37216/0 (socket says 172.21.15.138:37216) 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cea00640 1 -- 172.21.15.138:0/4133995775 learned_addr learned my addr 172.21.15.138:0/4133995775 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64ce000640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0075b90 0x7f64d019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.427+0000 7f64cf400640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0113550 0x7f64d01a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 -- 172.21.15.138:0/4133995775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0075b90 msgr2=0x7f64d019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0075b90 0x7f64d019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 -- 172.21.15.138:0/4133995775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0113550 msgr2=0x7f64d01a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0113550 0x7f64d01a4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 -- 172.21.15.138:0/4133995775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d01a5400 con 0x7f64d0075270 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cea00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005940 tx=0x7f64c00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64d4a00640 1 -- 172.21.15.138:0/4133995775 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c003f030 con 0x7f64d0075270 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d01a5690 con 0x7f64d0075270 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.428+0000 7f64d4a00640 1 -- 172.21.15.138:0/4133995775 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c00044c0 con 0x7f64d0075270 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.429+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d01abdb0 con 0x7f64d0075270 2024-08-20T01:16:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.429+0000 7f64d4a00640 1 -- 172.21.15.138:0/4133995775 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0049060 con 0x7f64d0075270 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.431+0000 7f64d8c62740 1 Processor -- start 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 -- start start 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c987893b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987896470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c9877fd210 con 0x55c9878880f0 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c9877ec710 con 0x55c9878877d0 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d8c62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c9877f0750 con 0x55c987894080 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64cf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64cf400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35224/0 (socket says 172.21.15.138:35224) 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64cf400640 1 -- 172.21.15.138:0/1559591361 learned_addr learned my addr 172.21.15.138:0/1559591361 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64ce000640 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987896470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64d4a00640 1 -- 172.21.15.138:0/4133995775 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f64c0042030 con 0x7f64d0075270 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.432+0000 7f64cf400640 1 -- 172.21.15.138:0/1559591361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 msgr2=0x55c987896470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64cf400640 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987896470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64cf400640 1 -- 172.21.15.138:0/1559591361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 msgr2=0x55c987893b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64cf400640 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c987893b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64cf400640 1 -- 172.21.15.138:0/1559591361 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9877b8de0 con 0x55c9878877d0 2024-08-20T01:16:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64cf400640 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009f90 tx=0x7f64c80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b592bf199438114f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64bbe00640 1 -- 172.21.15.138:0/1559591361 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004d20 con 0x55c9878877d0 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64bbe00640 1 -- 172.21.15.138:0/1559591361 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64c8004050 con 0x55c9878877d0 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.433+0000 7f64bbe00640 1 -- 172.21.15.138:0/1559591361 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c8005280 con 0x55c9878877d0 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 -- 172.21.15.138:0/1559591361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 msgr2=0x55c987887bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009f90 tx=0x7f64c80118d0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 -- 172.21.15.138:0/1559591361 shutdown_connections 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987896470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c987893b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 --2- 172.21.15.138:0/1559591361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c987887bb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 -- 172.21.15.138:0/1559591361 >> 172.21.15.138:0/1559591361 conn(0x55c987885600 msgr2=0x55c9878867a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 -- 172.21.15.138:0/1559591361 shutdown_connections 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.434+0000 7f64d8c62740 1 -- 172.21.15.138:0/1559591361 wait complete. 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 -- start start 2024-08-20T01:16:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 Processor -- start 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d4a00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 0x7f64b40b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64cf400640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 0x7f64b40b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d4a00640 1 -- 172.21.15.138:0/4133995775 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f64c0104ab0 con 0x7f64d0075270 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 0x55c9878915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c987894080 0x55c987891b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c987725ab0 con 0x55c9878877d0 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c987793140 con 0x55c987894080 2024-08-20T01:16:47.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.435+0000 7f64d8c62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c987896d60 con 0x55c9878880f0 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64d8c62740 10 client.0 _ll_register_callbacks cb 0x55c9877a65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:37232/0 (socket says 172.21.15.138:37232) 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cea00640 1 -- 172.21.15.138:0/2668293254 learned_addr learned my addr 172.21.15.138:0/2668293254 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cf400640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c987894080 0x55c987891b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 0x55c9878915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 -- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c987894080 msgr2=0x55c987891b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c987894080 0x55c987891b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 -- 172.21.15.138:0/2668293254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 msgr2=0x55c987891090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64ce000640 1 -- 172.21.15.138:0/2668293254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c987794170 con 0x55c9878880f0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cea00640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.436+0000 7f64cf400640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 0x7f64b40b6110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64c8011dd0 tx=0x7f64c8000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64ce000640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 0x55c9878915d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0126d0 tx=0x7f64bc012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5872f883666881d6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00f8a0 con 0x55c9878880f0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c9877c3a30 con 0x55c9878880f0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f64bc00fa40 con 0x55c9878880f0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f64bc00fc40 con 0x55c9878880f0 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64d8c62740 10 client.26503 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:47.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c98754f790 con 0x55c9878880f0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c9877c3a30 con 0x55c9878880f0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.437+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f64bc01c420 con 0x55c9878880f0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 10 client.26503 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 10 client.26503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 20 client.26503 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c9878a4080 0x55c9878a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 4 client.26503 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9877ff160 con 0x55c9878a4080 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64cea00640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c9878a4080 0x55c9878a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64cea00640 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c9878a4080 0x55c9878a64a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bccca99ecdf833d0 server_cookie=2f639537a8253c73 in_seq=0 out_seq=0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.438+0000 7f64b8c00640 10 client.26503 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64b8c00640 1 -- 172.21.15.138:0/2668293254 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9877ff160 con 0x55c9878a4080 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64b8c00640 10 client.26503 handle_command_reply: tid=0 2024-08-20T01:16:47.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 client.26503 shutdown 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 msgr2=0x55c9878915d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 0x55c9878915d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0126d0 tx=0x7f64bc012b90 comp rx=0 tx=0).stop 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 shutdown_connections 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c9878a4080 0x55c9878a64a0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c987894080 0x55c987891b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878880f0 0x55c9878915d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2668293254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878877d0 0x55c987891090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64b8c00640 0 client.26503 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.439+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 >> 172.21.15.138:0/2668293254 conn(0x55c987885600 msgr2=0x55c98788b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.440+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 shutdown_connections 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.440+0000 7f64d8c62740 1 -- 172.21.15.138:0/2668293254 wait complete. 2024-08-20T01:16:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.440+0000 7f64d8c62740 20 client.26503 trim_cache size 0 max 16384 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 Processor -- start 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 -- start start 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 0x55c9878abd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c987894080 0x55c9878ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b8820 con 0x55c987894080 2024-08-20T01:16:47.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b60f0 con 0x55c9878880f0 2024-08-20T01:16:47.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64d8c62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b3f60 con 0x55c9878877d0 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64cea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64ce000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 0x55c9878abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64cea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33470/0 (socket says 172.21.15.138:33470) 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64cea00640 1 -- 172.21.15.138:0/2885037790 learned_addr learned my addr 172.21.15.138:0/2885037790 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.455+0000 7f64cf400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c987894080 0x55c9878ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 -- 172.21.15.138:0/2885037790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 msgr2=0x55c987886e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 -- 172.21.15.138:0/2885037790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c987894080 msgr2=0x55c9878ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c987894080 0x55c9878ac2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 -- 172.21.15.138:0/2885037790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9875c8620 con 0x55c9878880f0 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64cea00640 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64ce000640 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 0x55c9878abd70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f64bc001eb0 tx=0x7f64bc005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f146498035f1889f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64bb400640 1 -- 172.21.15.138:0/2885037790 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00fde0 con 0x55c9878880f0 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64bb400640 1 -- 172.21.15.138:0/2885037790 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc00f8a0 con 0x55c9878880f0 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.456+0000 7f64bb400640 1 -- 172.21.15.138:0/2885037790 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0054c0 con 0x55c9878880f0 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 -- 172.21.15.138:0/2885037790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 msgr2=0x55c9878abd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 0x55c9878abd70 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f64bc001eb0 tx=0x7f64bc005db0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 -- 172.21.15.138:0/2885037790 shutdown_connections 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c987894080 0x55c9878ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878880f0 0x55c9878abd70 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 --2- 172.21.15.138:0/2885037790 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c9878877d0 0x55c987886e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 -- 172.21.15.138:0/2885037790 >> 172.21.15.138:0/2885037790 conn(0x55c987885600 msgr2=0x55c987886710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 -- 172.21.15.138:0/2885037790 shutdown_connections 2024-08-20T01:16:47.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.457+0000 7f64d8c62740 1 -- 172.21.15.138:0/2885037790 wait complete. 2024-08-20T01:16:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 -- start start 2024-08-20T01:16:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 Processor -- start 2024-08-20T01:16:47.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c98789c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987906d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b45b0 con 0x55c9878880f0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b5a20 con 0x55c9878877d0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f64d01b4c80 con 0x55c987894080 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.458+0000 7f64d8c62740 10 client.0 _ll_register_callbacks cb 0x7f64bc009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c98789c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35246/0 (socket says 172.21.15.138:35246) 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 -- 172.21.15.138:0/3456822256 learned_addr learned my addr 172.21.15.138:0/3456822256 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cf400640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987906d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 -- 172.21.15.138:0/3456822256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 msgr2=0x55c987906d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987906d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 -- 172.21.15.138:0/3456822256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 msgr2=0x55c98789c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c98789c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 -- 172.21.15.138:0/3456822256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c987907530 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64cea00640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64c00c85e0 tx=0x7f64c0004230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae86e329796d12 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c0035040 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64bc00f8a0 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.459+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f64c00c7960 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f64c00d4370 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 10 client.26537 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c98790e8f0 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64bc00f8a0 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f64c00d3ee0 con 0x55c9878877d0 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 10 client.26537 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:47.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 20 client.26537 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c98790efc0 0x55c9879113b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 4 client.26537 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9877fac00 con 0x55c98790efc0 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.460+0000 7f64ce000640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c98790efc0 0x55c9879113b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.461+0000 7f64ce000640 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c98790efc0 0x55c9879113b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8917ea30e15d04 server_cookie=6b4cf98367f020d2 in_seq=0 out_seq=0 2024-08-20T01:16:47.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.461+0000 7f64b8c00640 10 client.26537 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.505+0000 7f64b8c00640 1 -- 172.21.15.138:0/3456822256 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9877fac00 con 0x55c98790efc0 2024-08-20T01:16:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.505+0000 7f64b8c00640 10 client.26537 handle_command_reply: tid=0 2024-08-20T01:16:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.505+0000 7f64d8c62740 1 client.26537 shutdown 2024-08-20T01:16:47.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.505+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 msgr2=0x55c98789bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.505+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f64c00c85e0 tx=0x7f64c0004230 comp rx=0 tx=0).stop 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 shutdown_connections 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c98790efc0 0x55c9879113b0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c987894080 0x55c987906d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c9878880f0 0x55c98789c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 --2- 172.21.15.138:0/3456822256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c9878877d0 0x55c98789bf80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64b8c00640 0 client.26537 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 >> 172.21.15.138:0/3456822256 conn(0x55c987885600 msgr2=0x55c98788a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 shutdown_connections 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 1 -- 172.21.15.138:0/3456822256 wait complete. 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.506+0000 7f64d8c62740 20 client.26537 trim_cache size 0 max 16384 2024-08-20T01:16:47.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:16:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:16:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:16:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:16:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 msgr2=0x7f64b40b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 0x7f64b40b6110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f64c8011dd0 tx=0x7f64c8000f40 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 msgr2=0x7f64d019f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f64c0005940 tx=0x7f64c00040c0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 shutdown_connections 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f64b40b3c50 0x7f64b40b6110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64d0113550 0x7f64d01a4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f64d0075b90 0x7f64d019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 --2- 172.21.15.138:0/4133995775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f64d0075270 0x7f64d019f360 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 >> 172.21.15.138:0/4133995775 conn(0x7f64d00fe280 msgr2=0x7f64d00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 shutdown_connections 2024-08-20T01:16:47.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:16:47.507+0000 7f64cfe00640 1 -- 172.21.15.138:0/4133995775 wait complete. 2024-08-20T01:16:48.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2126304270' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:16:48.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 365161 4.11 MB/sec execute 300 sec latency 2718.694 ms 2024-08-20T01:16:48.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2126304270' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:16:48.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2126304270' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:16:48.452 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 147.85 --await-for 45.0' 2024-08-20T01:16:48.452 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 147.85 --await-for 45.0 2024-08-20T01:16:48.899 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:16:49.070 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:48 smithi138 ceph-mon[18076]: pgmap v628: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-08-20T01:16:49.173 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366242 4.11 MB/sec execute 301 sec latency 20.421 ms 2024-08-20T01:16:49.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:48 smithi167 ceph-mon[21965]: pgmap v628: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-08-20T01:16:49.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:48 smithi142 ceph-mon[21799]: pgmap v628: 129 pgs: 129 active+clean; 255 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 107 op/s 2024-08-20T01:16:49.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 -- 172.21.15.138:0/641733414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c104cc0 msgr2=0x7f360c1050a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 --2- 172.21.15.138:0/641733414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c104cc0 0x7f360c1050a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3600009900 tx=0x7f360002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 -- 172.21.15.138:0/641733414 shutdown_connections 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 --2- 172.21.15.138:0/641733414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c10b050 0x7f360c111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 --2- 172.21.15.138:0/641733414 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10a6d0 0x7f360c10ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 --2- 172.21.15.138:0/641733414 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c104cc0 0x7f360c1050a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 -- 172.21.15.138:0/641733414 >> 172.21.15.138:0/641733414 conn(0x7f360c100710 msgr2=0x7f360c102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 -- 172.21.15.138:0/641733414 shutdown_connections 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.562+0000 7f3613e00640 1 -- 172.21.15.138:0/641733414 wait complete. 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.563+0000 7f3613e00640 1 Processor -- start 2024-08-20T01:16:49.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.563+0000 7f3613e00640 1 -- start start 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c10a6d0 0x7f360c19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10b050 0x7f360c1a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f360c114d20 con 0x7f360c10b050 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f360c114ba0 con 0x7f360c104cc0 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3613e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f360c114ea0 con 0x7f360c10a6d0 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50748/0 (socket says 172.21.15.138:50748) 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 -- 172.21.15.138:0/4155931597 learned_addr learned my addr 172.21.15.138:0/4155931597 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10b050 0x7f360c1a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3610a00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c10a6d0 0x7f360c19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 -- 172.21.15.138:0/4155931597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c10a6d0 msgr2=0x7f360c19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c10a6d0 0x7f360c19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 -- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10b050 msgr2=0x7f360c1a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10b050 0x7f360c1a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.564+0000 7f3611400640 1 -- 172.21.15.138:0/4155931597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f360c1a1240 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f3611400640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3600005ab0 tx=0x7f3600003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f360a000640 1 -- 172.21.15.138:0/4155931597 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360003f070 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f360c1a14d0 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f360a000640 1 -- 172.21.15.138:0/4155931597 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3600031b60 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f360a000640 1 -- 172.21.15.138:0/4155931597 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600035030 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.565+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f360c1a7bf0 con 0x7f360c104cc0 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.568+0000 7f361687c740 1 Processor -- start 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 -- start start 2024-08-20T01:16:49.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f360a000640 1 -- 172.21.15.138:0/4155931597 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3600042070 con 0x7f360c104cc0 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4074760 0x5572b40801b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4082ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572b3fe9600 con 0x5572b40806f0 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5572b3fd8b00 con 0x5572b4073e40 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f361687c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5572b3fdcb40 con 0x5572b4074760 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f3611e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f3611e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50756/0 (socket says 172.21.15.138:50756) 2024-08-20T01:16:49.586 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f3611e00640 1 -- 172.21.15.138:0/2405254794 learned_addr learned my addr 172.21.15.138:0/2405254794 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f3611400640 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4074760 0x5572b40801b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.569+0000 7f3610a00640 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4082ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 -- 172.21.15.138:0/2405254794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4074760 msgr2=0x5572b40801b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4074760 0x5572b40801b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 -- 172.21.15.138:0/2405254794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 msgr2=0x5572b4082ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4082ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 -- 172.21.15.138:0/2405254794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b3f7eee0 con 0x5572b4073e40 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f3611e00640 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3604009fd0 tx=0x7f36040118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa124aab25832ab9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f35fe600640 1 -- 172.21.15.138:0/2405254794 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604004d20 con 0x5572b4073e40 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f35fe600640 1 -- 172.21.15.138:0/2405254794 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3604004050 con 0x5572b4073e40 2024-08-20T01:16:49.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.570+0000 7f35fe600640 1 -- 172.21.15.138:0/2405254794 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3604005280 con 0x5572b4073e40 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f360a000640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 0x7f35f40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 -- 172.21.15.138:0/2405254794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 msgr2=0x5572b4074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3604009fd0 tx=0x7f36040118d0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 -- 172.21.15.138:0/2405254794 shutdown_connections 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f360a000640 1 -- 172.21.15.138:0/4155931597 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f36001049b0 con 0x7f360c104cc0 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4082ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4074760 0x5572b40801b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 --2- 172.21.15.138:0/2405254794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4073e40 0x5572b4074220 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 -- 172.21.15.138:0/2405254794 >> 172.21.15.138:0/2405254794 conn(0x5572b4071c70 msgr2=0x5572b4072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f3611400640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 0x7f35f40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.571+0000 7f361687c740 1 -- 172.21.15.138:0/2405254794 shutdown_connections 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.572+0000 7f361687c740 1 -- 172.21.15.138:0/2405254794 wait complete. 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.572+0000 7f361687c740 1 -- start start 2024-08-20T01:16:49.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f3611400640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 0x7f35f40b6280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3600101e10 tx=0x7f3600009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 Processor -- start 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b4073e40 0x5572b407d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 0x5572b407e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5572b3e3b7b0 con 0x5572b4073e40 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5572b3f73dd0 con 0x5572b4074760 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.573+0000 7f361687c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5572b4083370 con 0x5572b40806f0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f361687c740 10 client.0 _ll_register_callbacks cb 0x5572b3ce50a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50762/0 (socket says 172.21.15.138:50762) 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 -- 172.21.15.138:0/150984496 learned_addr learned my addr 172.21.15.138:0/150984496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3611e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b4073e40 0x5572b407d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3611e00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 0x5572b407e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 -- 172.21.15.138:0/150984496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 msgr2=0x5572b407e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 0x5572b407e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 -- 172.21.15.138:0/150984496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b4073e40 msgr2=0x5572b407d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b4073e40 0x5572b407d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 -- 172.21.15.138:0/150984496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b3d47ba0 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3611e00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 0x5572b407e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f3610a00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35f80126d0 tx=0x7f35f8012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bae78ccf03ff6dbe server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.574+0000 7f35f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8004050 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3604004050 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f35f80041f0 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35f8025050 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f361687c740 10 client.26549 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572b408ef80 con 0x5572b4074760 2024-08-20T01:16:49.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3604004050 con 0x5572b4074760 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.575+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f35f8003a30 con 0x5572b4074760 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 10 client.26549 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 10 client.26549 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 20 client.26549 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40903e0 0x5572b4092830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 4 client.26549 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b3feb550 con 0x5572b40903e0 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.576+0000 7f3611e00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40903e0 0x5572b4092830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.577+0000 7f3611e00640 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40903e0 0x5572b4092830 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cc78e404f9f1ab server_cookie=d74611e1956afcfd in_seq=0 out_seq=0 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.577+0000 7f35f2a00640 10 client.26549 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f35f2a00640 1 -- 172.21.15.138:0/150984496 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572b3feb550 con 0x5572b40903e0 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f35f2a00640 10 client.26549 handle_command_reply: tid=0 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 client.26549 shutdown 2024-08-20T01:16:49.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 msgr2=0x5572b407dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f35f80126d0 tx=0x7f35f8012b90 comp rx=0 tx=0).stop 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 shutdown_connections 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40903e0 0x5572b4092830 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b40806f0 0x5572b407e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b407dbf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 --2- 172.21.15.138:0/150984496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b4073e40 0x5572b407d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f35f2a00640 0 client.26549 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 >> 172.21.15.138:0/150984496 conn(0x5572b4071c70 msgr2=0x5572b40777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 shutdown_connections 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.578+0000 7f361687c740 1 -- 172.21.15.138:0/150984496 wait complete. 2024-08-20T01:16:49.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.579+0000 7f361687c740 20 client.26549 trim_cache size 0 max 16384 2024-08-20T01:16:49.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.599+0000 7f361687c740 1 Processor -- start 2024-08-20T01:16:49.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.599+0000 7f361687c740 1 -- start start 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b407f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4098680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1b45e0 con 0x5572b40806f0 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1b1eb0 con 0x5572b4074760 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f361687c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1afd20 con 0x5572b4073e40 2024-08-20T01:16:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50772/0 (socket says 172.21.15.138:50772) 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 -- 172.21.15.138:0/2936141423 learned_addr learned my addr 172.21.15.138:0/2936141423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3611e00640 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b407f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3611e00640 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 -- 172.21.15.138:0/2936141423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 msgr2=0x5572b407f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b407f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 -- 172.21.15.138:0/2936141423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 msgr2=0x5572b4098680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4098680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.600+0000 7f3610a00640 1 -- 172.21.15.138:0/2936141423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b4074550 con 0x5572b4074760 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.601+0000 7f3610a00640 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35f800a010 tx=0x7f35f8012320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cb05bf3ca0a38ad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.601+0000 7f35fdc00640 1 -- 172.21.15.138:0/2936141423 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8003fc0 con 0x5572b4074760 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.601+0000 7f35fdc00640 1 -- 172.21.15.138:0/2936141423 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f800f920 con 0x5572b4074760 2024-08-20T01:16:49.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.601+0000 7f35fdc00640 1 -- 172.21.15.138:0/2936141423 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f80052f0 con 0x5572b4074760 2024-08-20T01:16:49.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 -- 172.21.15.138:0/2936141423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 msgr2=0x5572b4098140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f35f800a010 tx=0x7f35f8012320 comp rx=0 tx=0).stop 2024-08-20T01:16:49.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 -- 172.21.15.138:0/2936141423 shutdown_connections 2024-08-20T01:16:49.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b4098680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b4098140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 --2- 172.21.15.138:0/2936141423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b407f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 -- 172.21.15.138:0/2936141423 >> 172.21.15.138:0/2936141423 conn(0x5572b4071c70 msgr2=0x5572b407ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 -- 172.21.15.138:0/2936141423 shutdown_connections 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.602+0000 7f361687c740 1 -- 172.21.15.138:0/2936141423 wait complete. 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.603+0000 7f361687c740 1 -- start start 2024-08-20T01:16:49.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.603+0000 7f361687c740 1 Processor -- start 2024-08-20T01:16:49.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b4084d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b40852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b40f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1b0370 con 0x5572b40806f0 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1b17e0 con 0x5572b4074760 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f360c1b0a40 con 0x5572b4073e40 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f361687c740 10 client.0 _ll_register_callbacks cb 0x7f35f8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3611e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b4084d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b40852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3611e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b4084d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41676/0 (socket says 172.21.15.138:41676) 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3611e00640 1 -- 172.21.15.138:0/3488457911 learned_addr learned my addr 172.21.15.138:0/3488457911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3611e00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b40f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 -- 172.21.15.138:0/3488457911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 msgr2=0x5572b4084d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b4084d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 -- 172.21.15.138:0/3488457911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 msgr2=0x5572b40f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b40f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.604 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.604+0000 7f3610a00640 1 -- 172.21.15.138:0/3488457911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b40f4820 con 0x5572b4074760 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f3610a00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b40852b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35f80097e0 tx=0x7f35f800fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915d04191dfe5ae0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f35f2a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f8036020 con 0x5572b4074760 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f800f920 con 0x5572b4074760 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f35f8005540 con 0x5572b4074760 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f35f8041050 con 0x5572b4074760 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f361687c740 10 client.26555 fetch_fsmap learned FSMap version 39 2024-08-20T01:16:49.605 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.605+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5572b40fbbb0 con 0x5572b4074760 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.606+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35f800f920 con 0x5572b4074760 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f35f8003f70 con 0x5572b4074760 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 10 client.26555 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 10 client.26555 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 20 client.26555 populate_metadata read hostname 'smithi138' 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40fc1d0 0x5572b40fe5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 4 client.26555 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.85, "await_for": 45.0}] 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.85, "await_for": 45.0}) -- 0x5572b3fe6ff0 con 0x5572b40fc1d0 2024-08-20T01:16:49.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.607+0000 7f3611e00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40fc1d0 0x5572b40fe5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:16:49.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.608+0000 7f3611e00640 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40fc1d0 0x5572b40fe5c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e783f52af29d9f0b server_cookie=2cdb00b008a8f42f in_seq=0 out_seq=0 2024-08-20T01:16:49.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.608+0000 7f35f2a00640 10 client.26555 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:49.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.668+0000 7f35f2a00640 1 -- 172.21.15.138:0/3488457911 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5572b3fe6ff0 con 0x5572b40fc1d0 2024-08-20T01:16:49.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.668+0000 7f35f2a00640 10 client.26555 handle_command_reply: tid=0 2024-08-20T01:16:49.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.668+0000 7f361687c740 1 client.26555 shutdown 2024-08-20T01:16:49.667 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 msgr2=0x5572b40852b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b40852b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f35f80097e0 tx=0x7f35f800fdf0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 shutdown_connections 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5572b40fc1d0 0x5572b40fe5c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5572b40806f0 0x5572b40f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572b4074760 0x5572b40852b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 --2- 172.21.15.138:0/3488457911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5572b4073e40 0x5572b4084d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f35f2a00640 0 client.26555 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 >> 172.21.15.138:0/3488457911 conn(0x5572b4071c70 msgr2=0x5572b407ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 shutdown_connections 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 1 -- 172.21.15.138:0/3488457911 wait complete. 2024-08-20T01:16:49.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.669+0000 7f361687c740 20 client.26555 trim_cache size 0 max 16384 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 39, 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 34, 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "be3839cb": { 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 34, 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:16:49.669 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 147.8, 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:16:49.670 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 msgr2=0x7f35f40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 0x7f35f40b6280 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3600101e10 tx=0x7f3600009260 comp rx=0 tx=0).stop 2024-08-20T01:16:49.671 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 msgr2=0x7f360c19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3600005ab0 tx=0x7f3600003c10 comp rx=0 tx=0).stop 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 shutdown_connections 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f35f40b3dc0 0x7f35f40b6280 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f360c10b050 0x7f360c1a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f360c10a6d0 0x7f360c19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 --2- 172.21.15.138:0/4155931597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f360c104cc0 0x7f360c19b190 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 >> 172.21.15.138:0/4155931597 conn(0x7f360c100710 msgr2=0x7f360c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 shutdown_connections 2024-08-20T01:16:49.672 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:16:49.670+0000 7f3613e00640 1 -- 172.21.15.138:0/4155931597 wait complete. 2024-08-20T01:16:50.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:49 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:16:50.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.11 MB/sec execute 302 sec latency 496.424 ms 2024-08-20T01:16:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:49 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:16:50.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:49 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:16:50.643 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'be3839cb', quiesce took 5.4 seconds. Will release after: 22.450000000000003 2024-08-20T01:16:51.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.10 MB/sec execute 303 sec latency 1496.521 ms 2024-08-20T01:16:51.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:50 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:16:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:50 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:16:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:50 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:16:51.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:50 smithi167 ceph-mon[21965]: pgmap v629: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-08-20T01:16:51.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:50 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:16:51.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:50 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:16:51.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:50 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:16:51.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:50 smithi142 ceph-mon[21799]: pgmap v629: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-08-20T01:16:51.266 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:50 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:16:51.266 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:50 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:16:51.266 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:50 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:16:51.266 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:50 smithi138 ceph-mon[18076]: pgmap v629: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.9 MiB/s wr, 176 op/s 2024-08-20T01:16:52.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.09 MB/sec execute 304 sec latency 2496.634 ms 2024-08-20T01:16:53.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.07 MB/sec execute 305 sec latency 3496.795 ms 2024-08-20T01:16:53.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:52 smithi167 ceph-mon[21965]: pgmap v630: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-08-20T01:16:53.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:52 smithi142 ceph-mon[21799]: pgmap v630: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-08-20T01:16:53.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:52 smithi138 ceph-mon[18076]: pgmap v630: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.8 MiB/s wr, 165 op/s 2024-08-20T01:16:54.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.06 MB/sec execute 306 sec latency 4496.869 ms 2024-08-20T01:16:55.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.05 MB/sec execute 307 sec latency 5496.992 ms 2024-08-20T01:16:55.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:54 smithi167 ceph-mon[21965]: pgmap v631: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-08-20T01:16:55.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:54 smithi142 ceph-mon[21799]: pgmap v631: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-08-20T01:16:55.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:54 smithi138 ceph-mon[18076]: pgmap v631: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-08-20T01:16:56.174 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.03 MB/sec execute 308 sec latency 6497.084 ms 2024-08-20T01:16:57.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.02 MB/sec execute 309 sec latency 7497.185 ms 2024-08-20T01:16:57.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:56 smithi167 ceph-mon[21965]: pgmap v632: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:16:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:56 smithi142 ceph-mon[21799]: pgmap v632: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:16:57.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:56 smithi138 ceph-mon[18076]: pgmap v632: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-08-20T01:16:58.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 4.01 MB/sec execute 310 sec latency 8497.268 ms 2024-08-20T01:16:59.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.99 MB/sec execute 311 sec latency 9497.400 ms 2024-08-20T01:16:59.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:16:58 smithi167 ceph-mon[21965]: pgmap v633: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-08-20T01:16:59.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:16:58 smithi142 ceph-mon[21799]: pgmap v633: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-08-20T01:16:59.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:16:58 smithi138 ceph-mon[18076]: pgmap v633: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.2 MiB/s wr, 108 op/s 2024-08-20T01:17:00.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.98 MB/sec execute 312 sec latency 10497.519 ms 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 -- 172.21.15.138:0/1866153176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f791010cd50 msgr2=0x7f791010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 --2- 172.21.15.138:0/1866153176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f791010cd50 0x7f791010f1d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ca0 tx=0x7f790802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 -- 172.21.15.138:0/1866153176 shutdown_connections 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 --2- 172.21.15.138:0/1866153176 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f791010cd50 0x7f791010f1d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 --2- 172.21.15.138:0/1866153176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7910104220 0x7f791010c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 --2- 172.21.15.138:0/1866153176 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910103900 0x7f7910103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 -- 172.21.15.138:0/1866153176 >> 172.21.15.138:0/1866153176 conn(0x7f79100fd2f0 msgr2=0x7f79100ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:00.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 -- 172.21.15.138:0/1866153176 shutdown_connections 2024-08-20T01:17:00.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.627+0000 7f7917000640 1 -- 172.21.15.138:0/1866153176 wait complete. 2024-08-20T01:17:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.628+0000 7f7917000640 1 Processor -- start 2024-08-20T01:17:00.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.628+0000 7f7917000640 1 -- start start 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910104220 0x7f7910199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791010cd50 0x7f791019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112a30 con 0x7f7910103900 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79101128b0 con 0x7f791010cd50 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7917000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7910112bb0 con 0x7f7910104220 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60002/0 (socket says 172.21.15.138:60002) 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 -- 172.21.15.138:0/559873320 learned_addr learned my addr 172.21.15.138:0/559873320 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790f400640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910104220 0x7f7910199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f7915000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791010cd50 0x7f791019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 -- 172.21.15.138:0/559873320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910104220 msgr2=0x7f7910199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910104220 0x7f7910199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 -- 172.21.15.138:0/559873320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791010cd50 msgr2=0x7f791019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791010cd50 0x7f791019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.629+0000 7f790fe00640 1 -- 172.21.15.138:0/559873320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f791019efb0 con 0x7f7910103900 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f790fe00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f790000e9f0 tx=0x7f790000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f790cc00640 1 -- 172.21.15.138:0/559873320 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790000cd80 con 0x7f7910103900 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f790cc00640 1 -- 172.21.15.138:0/559873320 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79000054d0 con 0x7f7910103900 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f791019f2a0 con 0x7f7910103900 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f790cc00640 1 -- 172.21.15.138:0/559873320 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7900010690 con 0x7f7910103900 2024-08-20T01:17:00.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.630+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a5960 con 0x7f7910103900 2024-08-20T01:17:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 Processor -- start 2024-08-20T01:17:00.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 -- start start 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f790cc00640 1 -- 172.21.15.138:0/559873320 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7900004e90 con 0x7f7910103900 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f9250 0x55b9069090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b90690b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b90686e210 con 0x55b9069095f0 2024-08-20T01:17:00.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b90685d710 con 0x55b9068f9250 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.633+0000 7f7919e4a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b906861750 con 0x55b9068f8930 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37948/0 (socket says 172.21.15.138:37948) 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f790f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b90690b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 -- 172.21.15.138:0/2002053376 learned_addr learned my addr 172.21.15.138:0/2002053376 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f790fe00640 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f9250 0x55b9069090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 -- 172.21.15.138:0/2002053376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f9250 msgr2=0x55b9069090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f9250 0x55b9069090b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 -- 172.21.15.138:0/2002053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 msgr2=0x55b90690b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b90690b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 -- 172.21.15.138:0/2002053376 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b906829de0 con 0x55b9068f8930 2024-08-20T01:17:00.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7915000640 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ca0 tx=0x7f79080093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15538ef6ec500fed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.634+0000 7f7905600640 1 -- 172.21.15.138:0/2002053376 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908031cf0 con 0x55b9068f8930 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7905600640 1 -- 172.21.15.138:0/2002053376 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908036030 con 0x55b9068f8930 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7905600640 1 -- 172.21.15.138:0/2002053376 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79080042a0 con 0x55b9068f8930 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f790cc00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 0x7f78f40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f790cc00640 1 -- 172.21.15.138:0/559873320 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f79000e4790 con 0x7f7910103900 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f790fe00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 0x7f78f40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 -- 172.21.15.138:0/2002053376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 msgr2=0x55b9068f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7908009ca0 tx=0x7f79080093e0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 -- 172.21.15.138:0/2002053376 shutdown_connections 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b90690b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f9250 0x55b9069090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2002053376 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f8930 0x55b9068f8d10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.635+0000 7f7919e4a740 1 -- 172.21.15.138:0/2002053376 >> 172.21.15.138:0/2002053376 conn(0x55b9068f6760 msgr2=0x55b9068f7900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f7919e4a740 1 -- 172.21.15.138:0/2002053376 shutdown_connections 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f7919e4a740 1 -- 172.21.15.138:0/2002053376 wait complete. 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f790fe00640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 0x7f78f40b6280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f790000c6a0 tx=0x7f7900092040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f7919e4a740 1 -- start start 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f7919e4a740 1 Processor -- start 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.636+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b906908d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906906190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b9069066d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b906796ab0 con 0x55b9069095f0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b906804140 con 0x55b9068f8930 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b90690c0f0 con 0x55b9068f9250 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7919e4a740 10 client.0 _ll_register_callbacks cb 0x55b9068175d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7915000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b906908d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7915000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b906908d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:49702/0 (socket says 172.21.15.138:49702) 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7915000640 1 -- 172.21.15.138:0/2055824838 learned_addr learned my addr 172.21.15.138:0/2055824838 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906906190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f7915000640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b9069066d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 -- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 msgr2=0x55b906908d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b906908d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 -- 172.21.15.138:0/2055824838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 msgr2=0x55b9069066d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b9069066d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.637+0000 7f790f400640 1 -- 172.21.15.138:0/2055824838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b906805170 con 0x55b9068f9250 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f790f400640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906906190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f791006bef0 tx=0x7f78fc012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2749b020906e367e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc004050 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7908036030 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f78fc0041f0 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78fc025050 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f7919e4a740 10 client.26536 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b9065c0790 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7908036030 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f78fc003950 con 0x55b9068f9250 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f7919e4a740 10 client.26536 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.638+0000 7f7919e4a740 10 client.26536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7919e4a740 20 client.26536 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b906919080 0x55b90691b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7919e4a740 4 client.26536 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b906870160 con 0x55b906919080 2024-08-20T01:17:00.641 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7915000640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b906919080 0x55b90691b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f7915000640 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b906919080 0x55b90691b470 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77297702aa227fa server_cookie=2a862039ed43adaf in_seq=0 out_seq=0 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.639+0000 7f78fa000640 10 client.26536 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.640+0000 7f78fa000640 1 -- 172.21.15.138:0/2055824838 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b906870160 con 0x55b906919080 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.640+0000 7f78fa000640 10 client.26536 handle_command_reply: tid=0 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.640+0000 7f7919e4a740 1 client.26536 shutdown 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.640+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 msgr2=0x55b906906190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.640+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906906190 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f791006bef0 tx=0x7f78fc012b60 comp rx=0 tx=0).stop 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 shutdown_connections 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b906919080 0x55b90691b470 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9069095f0 0x55b9069066d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906906190 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f78fa000640 0 client.26536 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 --2- 172.21.15.138:0/2055824838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b906908d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 >> 172.21.15.138:0/2055824838 conn(0x55b9068f6760 msgr2=0x55b9068fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 shutdown_connections 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 1 -- 172.21.15.138:0/2055824838 wait complete. 2024-08-20T01:17:00.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.641+0000 7f7919e4a740 20 client.26536 trim_cache size 0 max 16384 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.655+0000 7f7919e4a740 1 Processor -- start 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 -- start start 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b9068f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b9068fba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79101b0150 con 0x55b9068f6760 2024-08-20T01:17:00.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79101ada20 con 0x55b9068f8930 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7919e4a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79101ab890 con 0x55b9068f9250 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7915000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7915000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60030/0 (socket says 172.21.15.138:60030) 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7915000640 1 -- 172.21.15.138:0/3301731988 learned_addr learned my addr 172.21.15.138:0/3301731988 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b9068f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7915000640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b9068fba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 -- 172.21.15.138:0/3301731988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 msgr2=0x55b9068fba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b9068fba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 -- 172.21.15.138:0/3301731988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 msgr2=0x55b90690b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f790f400640 1 -- 172.21.15.138:0/3301731988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b90691b7e0 con 0x55b9068f8930 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.656+0000 7f7915000640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.657+0000 7f790f400640 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b9068f5f50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78fc003f10 tx=0x7f78fc010640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f15a4beefa4b49 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.657+0000 7f7904c00640 1 -- 172.21.15.138:0/3301731988 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02e070 con 0x55b9068f8930 2024-08-20T01:17:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.657+0000 7f7904c00640 1 -- 172.21.15.138:0/3301731988 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc003aa0 con 0x55b9068f8930 2024-08-20T01:17:00.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.657+0000 7f7904c00640 1 -- 172.21.15.138:0/3301731988 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc0107d0 con 0x55b9068f8930 2024-08-20T01:17:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- 172.21.15.138:0/3301731988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 msgr2=0x55b9068f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b9068f5f50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f78fc003f10 tx=0x7f78fc010640 comp rx=0 tx=0).stop 2024-08-20T01:17:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- 172.21.15.138:0/3301731988 shutdown_connections 2024-08-20T01:17:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b9068fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.659 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f8930 0x55b9068f5f50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3301731988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f6760 0x55b90690b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- 172.21.15.138:0/3301731988 >> 172.21.15.138:0/3301731988 conn(0x55b906905d60 msgr2=0x55b90690b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- 172.21.15.138:0/3301731988 shutdown_connections 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- 172.21.15.138:0/3301731988 wait complete. 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 -- start start 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 Processor -- start 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.658+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f6760 0x55b9069212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906911920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f79101abee0 con 0x55b9068f8930 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f79101ad350 con 0x55b9068f6760 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f79101ac5b0 con 0x55b9068f9250 2024-08-20T01:17:00.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7919e4a740 10 client.0 _ll_register_callbacks cb 0x7f78fc009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60032/0 (socket says 172.21.15.138:60032) 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 -- 172.21.15.138:0/3375771921 learned_addr learned my addr 172.21.15.138:0/3375771921 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7915000640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f6760 0x55b9069212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f7915000640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906911920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 -- 172.21.15.138:0/3375771921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 msgr2=0x55b906911920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906911920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 -- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f6760 msgr2=0x55b9069212d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f6760 0x55b9069212d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 -- 172.21.15.138:0/3375771921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b906639620 con 0x55b9068f8930 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f790f400640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009f50 tx=0x7f78fc005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce37dd37b25cbc8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.659+0000 7f78fa000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78fc02e070 con 0x55b9068f8930 2024-08-20T01:17:00.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9453+0+0 (secure 0 0 0) 0x7f78fc018040 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78fc003aa0 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f78fc0052e0 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f7919e4a740 10 client.16518 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b906983d10 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78fc003aa0 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f78fc00f790 con 0x55b9068f8930 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f7919e4a740 10 client.16518 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.660+0000 7f7919e4a740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7919e4a740 20 client.16518 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9069095f0 0x55b906986400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7919e4a740 4 client.16518 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b90686bc00 con 0x55b9069095f0 2024-08-20T01:17:00.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7915000640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9069095f0 0x55b906986400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f78fa000640 10 client.16518 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:00.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:00.661+0000 7f7915000640 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9069095f0 0x55b906986400 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5de56a8c89d85aac server_cookie=73a7935aaed9a689 in_seq=0 out_seq=0 2024-08-20T01:17:01.058 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:00 smithi138 ceph-mon[18076]: pgmap v634: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f78fa000640 1 -- 172.21.15.138:0/3375771921 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b90686bc00 con 0x55b9069095f0 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f78fa000640 10 client.16518 handle_command_reply: tid=0 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 client.16518 shutdown 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 msgr2=0x55b9069113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f78fc009f50 tx=0x7f78fc005da0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 shutdown_connections 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b9069095f0 0x55b906986400 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b9068f9250 0x55b906911920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f78fa000640 0 client.16518 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b9068f8930 0x55b9069113e0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 --2- 172.21.15.138:0/3375771921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b9068f6760 0x55b9069212d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.172+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 >> 172.21.15.138:0/3375771921 conn(0x55b906905d60 msgr2=0x55b906907fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 shutdown_connections 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7919e4a740 1 -- 172.21.15.138:0/3375771921 wait complete. 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7919e4a740 20 client.16518 trim_cache size 0 max 16384 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:17:01.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 msgr2=0x7f78f40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 0x7f78f40b6280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f790000c6a0 tx=0x7f7900092040 comp rx=0 tx=0).stop 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 msgr2=0x7f7910198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.173+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f790000e9f0 tx=0x7f790000edf0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 shutdown_connections 2024-08-20T01:17:01.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f78f40b3dc0 0x7f78f40b6280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f791010cd50 0x7f791019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7910104220 0x7f7910199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 --2- 172.21.15.138:0/559873320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7910103900 0x7f7910198f10 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 >> 172.21.15.138:0/559873320 conn(0x7f79100fd2f0 msgr2=0x7f79100fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 shutdown_connections 2024-08-20T01:17:01.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:01.174+0000 7f7917000640 1 -- 172.21.15.138:0/559873320 wait complete. 2024-08-20T01:17:01.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.97 MB/sec execute 313 sec latency 11498.053 ms 2024-08-20T01:17:01.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:00 smithi167 ceph-mon[21965]: pgmap v634: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-08-20T01:17:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:00 smithi142 ceph-mon[21799]: pgmap v634: 129 pgs: 129 active+clean; 258 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-08-20T01:17:02.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.96 MB/sec execute 314 sec latency 12498.138 ms 2024-08-20T01:17:03.175 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.94 MB/sec execute 315 sec latency 13498.213 ms 2024-08-20T01:17:03.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:02 smithi167 ceph-mon[21965]: pgmap v635: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 694 KiB/s wr, 48 op/s 2024-08-20T01:17:03.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:02 smithi142 ceph-mon[21799]: pgmap v635: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 694 KiB/s wr, 48 op/s 2024-08-20T01:17:03.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:02 smithi138 ceph-mon[18076]: pgmap v635: 129 pgs: 129 active+clean; 257 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 694 KiB/s wr, 48 op/s 2024-08-20T01:17:04.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.93 MB/sec execute 316 sec latency 14498.316 ms 2024-08-20T01:17:05.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.92 MB/sec execute 317 sec latency 15498.440 ms 2024-08-20T01:17:05.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:05 smithi138 ceph-mon[18076]: osdmap e94: 12 total, 12 up, 12 in 2024-08-20T01:17:05.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:05 smithi138 ceph-mon[18076]: pgmap v637: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 256 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-08-20T01:17:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:05 smithi167 ceph-mon[21965]: osdmap e94: 12 total, 12 up, 12 in 2024-08-20T01:17:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:05 smithi167 ceph-mon[21965]: pgmap v637: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 256 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-08-20T01:17:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:05 smithi142 ceph-mon[21799]: osdmap e94: 12 total, 12 up, 12 in 2024-08-20T01:17:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:05 smithi142 ceph-mon[21799]: pgmap v637: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 256 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 885 KiB/s wr, 68 op/s 2024-08-20T01:17:06.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.91 MB/sec execute 318 sec latency 16498.493 ms 2024-08-20T01:17:07.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:06 smithi138 ceph-mon[18076]: pgmap v638: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 254 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 857 KiB/s wr, 71 op/s 2024-08-20T01:17:07.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.89 MB/sec execute 319 sec latency 17498.575 ms 2024-08-20T01:17:07.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:06 smithi167 ceph-mon[21965]: pgmap v638: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 254 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 857 KiB/s wr, 71 op/s 2024-08-20T01:17:07.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:06 smithi142 ceph-mon[21799]: pgmap v638: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 254 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 857 KiB/s wr, 71 op/s 2024-08-20T01:17:08.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.88 MB/sec execute 320 sec latency 18498.653 ms 2024-08-20T01:17:09.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.87 MB/sec execute 321 sec latency 19498.794 ms 2024-08-20T01:17:09.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:08 smithi167 ceph-mon[21965]: pgmap v639: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 836 KiB/s wr, 71 op/s 2024-08-20T01:17:09.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:08 smithi142 ceph-mon[21799]: pgmap v639: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 836 KiB/s wr, 71 op/s 2024-08-20T01:17:09.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:08 smithi138 ceph-mon[18076]: pgmap v639: 129 pgs: 6 active+clean+snaptrim, 11 active+clean+snaptrim_wait, 112 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 836 KiB/s wr, 71 op/s 2024-08-20T01:17:10.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.86 MB/sec execute 322 sec latency 20498.938 ms 2024-08-20T01:17:11.093 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:10 smithi138 ceph-mon[18076]: pgmap v640: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:17:11.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.85 MB/sec execute 323 sec latency 21499.019 ms 2024-08-20T01:17:11.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:10 smithi167 ceph-mon[21965]: pgmap v640: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:17:11.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:10 smithi142 ceph-mon[21799]: pgmap v640: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:17:12.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.83 MB/sec execute 324 sec latency 22499.088 ms 2024-08-20T01:17:13.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:12 smithi138 ceph-mon[18076]: pgmap v641: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 987 KiB/s wr, 71 op/s 2024-08-20T01:17:13.094 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:17:13.176 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.82 MB/sec execute 325 sec latency 23499.177 ms 2024-08-20T01:17:13.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:12 smithi167 ceph-mon[21965]: pgmap v641: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 987 KiB/s wr, 71 op/s 2024-08-20T01:17:13.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:12 smithi142 ceph-mon[21799]: pgmap v641: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 987 KiB/s wr, 71 op/s 2024-08-20T01:17:13.477 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:14.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 -- 172.21.15.138:0/1757259346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 msgr2=0x7f4ea810a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 --2- 172.21.15.138:0/1757259346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea810a960 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009900 tx=0x7f4e9402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.106 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 -- 172.21.15.138:0/1757259346 shutdown_connections 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 --2- 172.21.15.138:0/1757259346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 0x7f4ea81116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 --2- 172.21.15.138:0/1757259346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea810a960 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 --2- 172.21.15.138:0/1757259346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea8104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 -- 172.21.15.138:0/1757259346 >> 172.21.15.138:0/1757259346 conn(0x7f4ea8100540 msgr2=0x7f4ea8102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 -- 172.21.15.138:0/1757259346 shutdown_connections 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.106+0000 7f4eae600640 1 -- 172.21.15.138:0/1757259346 wait complete. 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.107+0000 7f4eae600640 1 Processor -- start 2024-08-20T01:17:14.107 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.107+0000 7f4eae600640 1 -- start start 2024-08-20T01:17:14.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 0x7f4ea81a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8114b70 con 0x7f4ea810a520 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea81149f0 con 0x7f4ea8104b10 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eae600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea8114cf0 con 0x7f4ea810aea0 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ea7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ea7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59588/0 (socket says 172.21.15.138:59588) 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ea7e00640 1 -- 172.21.15.138:0/102536784 learned_addr learned my addr 172.21.15.138:0/102536784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4eadc00640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 0x7f4ea81a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 -- 172.21.15.138:0/102536784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 msgr2=0x7f4ea81a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 0x7f4ea81a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 -- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 msgr2=0x7f4ea819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.109 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.108+0000 7f4ead200640 1 -- 172.21.15.138:0/102536784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea81a0ff0 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4ead200640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea819af40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e9f0 tx=0x7f4e9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00ccc0 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e9c0108e0 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c018880 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea81a12e0 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.109+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea81a7b70 con 0x7f4ea8104b10 2024-08-20T01:17:14.110 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.110+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea8104ef0 con 0x7f4ea8104b10 2024-08-20T01:17:14.112 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.113+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4e9c010430 con 0x7f4ea8104b10 2024-08-20T01:17:14.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.114+0000 7f4ea5600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 0x7f4e900b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.115+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9541+0+0 (secure 0 0 0) 0x7f4e9c0e4b90 con 0x7f4ea8104b10 2024-08-20T01:17:14.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.115+0000 7f4ea7e00640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 0x7f4e900b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.114 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.115+0000 7f4ea7e00640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 0x7f4e900b6320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8109070 tx=0x7f4e94005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.115 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.116+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4e9c0ab3f0 con 0x7f4ea8104b10 2024-08-20T01:17:14.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.81 MB/sec execute 326 sec latency 24499.230 ms 2024-08-20T01:17:14.293 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 -- 172.21.15.138:0/1770812516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8108200 msgr2=0x7f5ed8108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 --2- 172.21.15.138:0/1770812516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8108200 0x7f5ed8108640 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009900 tx=0x7f5ec402a410 comp rx=0 tx=0).stop 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 -- 172.21.15.138:0/1770812516 shutdown_connections 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 --2- 172.21.15.138:0/1770812516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108b80 0x7f5ed810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 --2- 172.21.15.138:0/1770812516 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8108200 0x7f5ed8108640 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 --2- 172.21.15.138:0/1770812516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8102800 0x7f5ed8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 -- 172.21.15.138:0/1770812516 >> 172.21.15.138:0/1770812516 conn(0x7f5ed80fe280 msgr2=0x7f5ed81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 -- 172.21.15.138:0/1770812516 shutdown_connections 2024-08-20T01:17:14.294 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.294+0000 7f5ed7200640 1 -- 172.21.15.138:0/1770812516 wait complete. 2024-08-20T01:17:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.295+0000 7f5ed7200640 1 Processor -- start 2024-08-20T01:17:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.295+0000 7f5ed7200640 1 -- start start 2024-08-20T01:17:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.295+0000 7f5ed7200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 0x7f5ed819aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.295+0000 7f5ed7200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.296 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed7200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8108b80 0x7f5ed81a08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed7200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8112840 con 0x7f5ed8108200 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed7200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81126c0 con 0x7f5ed8108b80 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed7200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81129c0 con 0x7f5ed8102800 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59604/0 (socket says 172.21.15.138:59604) 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 -- 172.21.15.138:0/259207562 learned_addr learned my addr 172.21.15.138:0/259207562 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 0x7f5ed819aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed6800640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8108b80 0x7f5ed81a08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 -- 172.21.15.138:0/259207562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 msgr2=0x7f5ed819aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 0x7f5ed819aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 -- 172.21.15.138:0/259207562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8108b80 msgr2=0x7f5ed81a08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8108b80 0x7f5ed81a08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5400640 1 -- 172.21.15.138:0/259207562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed81a1090 con 0x7f5ed8108200 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.296+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 0x7f5ed819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ed5400640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005940 tx=0x7f5ec40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ecea00640 1 -- 172.21.15.138:0/259207562 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec403f030 con 0x7f5ed8108200 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed81a1320 con 0x7f5ed8108200 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ecea00640 1 -- 172.21.15.138:0/259207562 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40044c0 con 0x7f5ed8108200 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ecea00640 1 -- 172.21.15.138:0/259207562 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4035070 con 0x7f5ed8108200 2024-08-20T01:17:14.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.297+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed81a7a40 con 0x7f5ed8108200 2024-08-20T01:17:14.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.300+0000 7f5ecea00640 1 -- 172.21.15.138:0/259207562 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5ec4042030 con 0x7f5ed8108200 2024-08-20T01:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 Processor -- start 2024-08-20T01:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 -- start start 2024-08-20T01:17:14.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5ecea00640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 0x7f5ebc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5ecea00640 1 -- 172.21.15.138:0/259207562 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9541+0+0 (secure 0 0 0) 0x7f5ec4104370 con 0x7f5ed8108200 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b442ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5ed6800640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 0x7f5ebc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 0x55566b44ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 0x55566b4517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55566b3b8210 con 0x55566b44f3c0 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55566b3a7710 con 0x55566b442b10 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.302+0000 7f5edf9be740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55566b3ab750 con 0x55566b443430 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b442ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 0x55566b4517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 0x55566b44ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 0x55566b4517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59606/0 (socket says 172.21.15.138:59606) 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed6800640 1 -- 172.21.15.138:0/758409011 learned_addr learned my addr 172.21.15.138:0/758409011 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 -- 172.21.15.138:0/758409011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 msgr2=0x55566b442ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b442ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 -- 172.21.15.138:0/758409011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 msgr2=0x55566b4517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 0x55566b4517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed5400640 1 -- 172.21.15.138:0/758409011 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566b373de0 con 0x55566b443430 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.303+0000 7f5ed6800640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 0x7f5ebc0b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009fd0 tx=0x7f5ed0013b80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.304+0000 7f5ed5400640 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 0x55566b44ee80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4101660 tx=0x7f5ec40c6360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f94e13b884aceeb1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.304+0000 7f5ec3000640 1 -- 172.21.15.138:0/758409011 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4049040 con 0x55566b443430 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.304+0000 7f5ec3000640 1 -- 172.21.15.138:0/758409011 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40cbe50 con 0x55566b443430 2024-08-20T01:17:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.304+0000 7f5ec3000640 1 -- 172.21.15.138:0/758409011 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40b20e0 con 0x55566b443430 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 -- 172.21.15.138:0/758409011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 msgr2=0x55566b44ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 0x55566b44ee80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4101660 tx=0x7f5ec40c6360 comp rx=0 tx=0).stop 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 -- 172.21.15.138:0/758409011 shutdown_connections 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b44f3c0 0x55566b4517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b443430 0x55566b44ee80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 --2- 172.21.15.138:0/758409011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b442ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 -- 172.21.15.138:0/758409011 >> 172.21.15.138:0/758409011 conn(0x55566b440940 msgr2=0x55566b441ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 -- 172.21.15.138:0/758409011 shutdown_connections 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.305+0000 7f5edf9be740 1 -- 172.21.15.138:0/758409011 wait complete. 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.306+0000 7f5edf9be740 1 -- start start 2024-08-20T01:17:14.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.306+0000 7f5edf9be740 1 Processor -- start 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.306+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b44ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b44f3c0 0x55566b44c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55566b2e0ab0 con 0x55566b443430 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55566b34e140 con 0x55566b442b10 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55566b451e60 con 0x55566b44f3c0 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5edf9be740 10 client.0 _ll_register_callbacks cb 0x55566b3615d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59612/0 (socket says 172.21.15.138:59612) 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 -- 172.21.15.138:0/3335967648 learned_addr learned my addr 172.21.15.138:0/3335967648 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b44ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed6800640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b44f3c0 0x55566b44c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 -- 172.21.15.138:0/3335967648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b44f3c0 msgr2=0x55566b44c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b44f3c0 0x55566b44c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 -- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 msgr2=0x55566b44ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b44ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 -- 172.21.15.138:0/3335967648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566b34f170 con 0x55566b443430 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40d3d90 tx=0x7f5ec4009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d9a681bb2276ec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.307+0000 7f5eb7400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4049070 con 0x55566b443430 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9541+0+0 (secure 0 0 0) 0x7f5ec40b22c0 con 0x55566b443430 2024-08-20T01:17:14.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec40cbe50 con 0x55566b443430 2024-08-20T01:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ec40cf120 con 0x55566b443430 2024-08-20T01:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 10 client.16536 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55566b10a790 con 0x55566b443430 2024-08-20T01:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40cbe50 con 0x55566b443430 2024-08-20T01:17:14.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ec40c6040 con 0x55566b443430 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 10 client.16536 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.308+0000 7f5edf9be740 20 client.16536 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b45ebc0 0x55566b460fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5edf9be740 4 client.16536 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55566b3ba160 con 0x55566b45ebc0 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b45ebc0 0x55566b460fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b45ebc0 0x55566b460fb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36560535b988dea5 server_cookie=eeed0a15dcb1f393 in_seq=0 out_seq=0 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.309+0000 7f5eb7400640 10 client.16536 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5eb7400640 1 -- 172.21.15.138:0/3335967648 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55566b3ba160 con 0x55566b45ebc0 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5eb7400640 10 client.16536 handle_command_reply: tid=0 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5edf9be740 1 client.16536 shutdown 2024-08-20T01:17:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 msgr2=0x55566b44c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40d3d90 tx=0x7f5ec4009680 comp rx=0 tx=0).stop 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 shutdown_connections 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.310+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b45ebc0 0x55566b460fb0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b44f3c0 0x55566b44c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44c0d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3335967648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b442b10 0x55566b44ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5eb7400640 0 client.16536 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 >> 172.21.15.138:0/3335967648 conn(0x55566b440940 msgr2=0x55566b4464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 shutdown_connections 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 1 -- 172.21.15.138:0/3335967648 wait complete. 2024-08-20T01:17:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.311+0000 7f5edf9be740 20 client.16536 trim_cache size 0 max 16384 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 Processor -- start 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 -- start start 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4421b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b466740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b4630 con 0x55566b443430 2024-08-20T01:17:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b1f00 con 0x55566b44f3c0 2024-08-20T01:17:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5edf9be740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81afd70 con 0x55566b442b10 2024-08-20T01:17:14.326 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.327 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5ed5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59618/0 (socket says 172.21.15.138:59618) 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5ed5400640 1 -- 172.21.15.138:0/4144263767 learned_addr learned my addr 172.21.15.138:0/4144263767 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.326+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed6800640 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b466740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 -- 172.21.15.138:0/4144263767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 msgr2=0x55566b4421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4421b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 -- 172.21.15.138:0/4144263767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 msgr2=0x55566b466740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b466740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 -- 172.21.15.138:0/4144263767 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55566b183620 con 0x55566b443430 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ed5400640 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cab60 tx=0x7f5ec40c6490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61135167723ee897 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ec2600640 1 -- 172.21.15.138:0/4144263767 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4049070 con 0x55566b443430 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ec2600640 1 -- 172.21.15.138:0/4144263767 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40cbe50 con 0x55566b443430 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.327+0000 7f5ec2600640 1 -- 172.21.15.138:0/4144263767 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40cae30 con 0x55566b443430 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- 172.21.15.138:0/4144263767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 msgr2=0x55566b44de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.328 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cab60 tx=0x7f5ec40c6490 comp rx=0 tx=0).stop 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- 172.21.15.138:0/4144263767 shutdown_connections 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b466740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b44de10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 --2- 172.21.15.138:0/4144263767 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- 172.21.15.138:0/4144263767 >> 172.21.15.138:0/4144263767 conn(0x55566b440940 msgr2=0x55566b44d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- 172.21.15.138:0/4144263767 shutdown_connections 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- 172.21.15.138:0/4144263767 wait complete. 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.328+0000 7f5edf9be740 1 -- start start 2024-08-20T01:17:14.329 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 Processor -- start 2024-08-20T01:17:14.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b4549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b454f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b03c0 con 0x55566b443430 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b1830 con 0x55566b44f3c0 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81b0a90 con 0x55566b442b10 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5edf9be740 10 client.0 _ll_register_callbacks cb 0x7f5ed0009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5ed5e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42622/0 (socket says 172.21.15.138:42622) 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5ed5e00640 1 -- 172.21.15.138:0/3561432290 learned_addr learned my addr 172.21.15.138:0/3561432290 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b4549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.329+0000 7f5ed6800640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b454f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 -- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 msgr2=0x55566b454f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b454f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 -- 172.21.15.138:0/3561432290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 msgr2=0x55566b4549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b4549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 -- 172.21.15.138:0/3561432290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55566b455500 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5ed5e00640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55566b4616b0 tx=0x7f5ec8012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b8a76c450cd4034 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5eb7400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8004230 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec40cbe50 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9541+0+0 (secure 0 0 0) 0x7f5ec80043d0 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5ec8031050 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5edf9be740 10 client.26566 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.330+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55566b4c95e0 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec40cbe50 con 0x55566b442b10 2024-08-20T01:17:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f5ec8003ae0 con 0x55566b442b10 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 10 client.26566 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 20 client.26566 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b4c9d00 0x55566b4cc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 4 client.26566 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55566b3b5c00 con 0x55566b4c9d00 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.331+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b4c9d00 0x55566b4cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.332+0000 7f5eb7400640 10 client.26566 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.332+0000 7f5ed5400640 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b4c9d00 0x55566b4cc120 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce77c692f971405 server_cookie=c7e92c671b22e852 in_seq=0 out_seq=0 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5eb7400640 1 -- 172.21.15.138:0/3561432290 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55566b3b5c00 con 0x55566b4c9d00 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5eb7400640 10 client.26566 handle_command_reply: tid=0 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 client.26566 shutdown 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 msgr2=0x55566b4574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x55566b4616b0 tx=0x7f5ec8012b80 comp rx=0 tx=0).stop 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 shutdown_connections 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55566b4c9d00 0x55566b4cc120 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55566b44f3c0 0x55566b454f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55566b443430 0x55566b4549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5eb7400640 0 client.26566 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:14.348 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 --2- 172.21.15.138:0/3561432290 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55566b442b10 0x55566b4574f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.348+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 >> 172.21.15.138:0/3561432290 conn(0x55566b440940 msgr2=0x55566b44df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 shutdown_connections 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5edf9be740 1 -- 172.21.15.138:0/3561432290 wait complete. 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5edf9be740 20 client.26566 trim_cache size 0 max 16384 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:17:14.349 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 msgr2=0x7f5ebc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 0x7f5ebc0b6280 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009fd0 tx=0x7f5ed0013b80 comp rx=0 tx=0).stop 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 msgr2=0x7f5ed819b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4005940 tx=0x7f5ec40040c0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 shutdown_connections 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5ebc0b3dc0 0x7f5ebc0b6280 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.350 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5ed8108b80 0x7f5ed81a08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.349+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ed8108200 0x7f5ed819b530 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.350+0000 7f5ed7200640 1 --2- 172.21.15.138:0/259207562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5ed8102800 0x7f5ed819aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.350+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 >> 172.21.15.138:0/259207562 conn(0x7f5ed80fe280 msgr2=0x7f5ed81047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.350+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 shutdown_connections 2024-08-20T01:17:14.351 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:14.350+0000 7f5ed7200640 1 -- 172.21.15.138:0/259207562 wait complete. 2024-08-20T01:17:14.356 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.357+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea819c070 con 0x7f4ea8104b10 2024-08-20T01:17:14.358 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.358+0000 7f4ea5600640 1 -- 172.21.15.138:0/102536784 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11932 (secure 0 0 0) 0x7f4ea819c070 con 0x7f4ea8104b10 2024-08-20T01:17:14.358 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:17:14.358 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":39,"btime":"2024-08-20T01:16:04:725509+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:16:04.226894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:17:14.358 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 39 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 msgr2=0x7f4e900b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 0x7f4e900b6320 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8109070 tx=0x7f4e94005a20 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 msgr2=0x7f4ea819af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea819af40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00e9f0 tx=0x7f4e9c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 shutdown_connections 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e900b3e60 0x7f4e900b6320 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ea810aea0 0x7f4ea81a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ea810a520 0x7f4ea819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 --2- 172.21.15.138:0/102536784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ea8104b10 0x7f4ea819af40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.361+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 >> 172.21.15.138:0/102536784 conn(0x7f4ea8100540 msgr2=0x7f4ea8102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.362+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 shutdown_connections 2024-08-20T01:17:14.361 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:14.362+0000 7f4eae600640 1 -- 172.21.15.138:0/102536784 wait complete. 2024-08-20T01:17:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:14 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/102536784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:14 smithi167 ceph-mon[21965]: osdmap e95: 12 total, 12 up, 12 in 2024-08-20T01:17:14.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:14 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/102536784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:14.716 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:14 smithi138 ceph-mon[18076]: osdmap e95: 12 total, 12 up, 12 in 2024-08-20T01:17:14.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:14 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/102536784' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:14.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:14 smithi142 ceph-mon[21799]: osdmap e95: 12 total, 12 up, 12 in 2024-08-20T01:17:15.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.80 MB/sec execute 327 sec latency 25499.375 ms 2024-08-20T01:17:15.291 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id be3839cb --release --await' 2024-08-20T01:17:15.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id be3839cb --release --await 2024-08-20T01:17:15.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:15 smithi138 ceph-mon[18076]: pgmap v643: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 778 KiB/s wr, 44 op/s 2024-08-20T01:17:15.666 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:15 smithi167 ceph-mon[21965]: pgmap v643: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 778 KiB/s wr, 44 op/s 2024-08-20T01:17:15.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:15 smithi142 ceph-mon[21799]: pgmap v643: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 778 KiB/s wr, 44 op/s 2024-08-20T01:17:16.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366934 3.79 MB/sec execute 328 sec latency 26499.541 ms 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 -- 172.21.15.138:0/2358372785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8102910 msgr2=0x7f75b8102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 --2- 172.21.15.138:0/2358372785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8102910 0x7f75b8102cf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f75a8009900 tx=0x7f75a802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 -- 172.21.15.138:0/2358372785 shutdown_connections 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 --2- 172.21.15.138:0/2358372785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 0x7f75b810f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 --2- 172.21.15.138:0/2358372785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8108320 0x7f75b8108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 --2- 172.21.15.138:0/2358372785 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8102910 0x7f75b8102cf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 -- 172.21.15.138:0/2358372785 >> 172.21.15.138:0/2358372785 conn(0x7f75b80fe3c0 msgr2=0x7f75b81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 -- 172.21.15.138:0/2358372785 shutdown_connections 2024-08-20T01:17:16.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.418+0000 7f75b7e00640 1 -- 172.21.15.138:0/2358372785 wait complete. 2024-08-20T01:17:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 Processor -- start 2024-08-20T01:17:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 -- start start 2024-08-20T01:17:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8108320 0x7f75b8199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.420 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 0x7f75b819e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112920 con 0x7f75b8102910 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81127a0 con 0x7f75b8108ca0 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112aa0 con 0x7f75b8108320 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.420+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59652/0 (socket says 172.21.15.138:59652) 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 -- 172.21.15.138:0/4216007830 learned_addr learned my addr 172.21.15.138:0/4216007830 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:16.421 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b7400640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 0x7f75b819e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 -- 172.21.15.138:0/4216007830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8108320 msgr2=0x7f75b8199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8108320 0x7f75b8199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 -- 172.21.15.138:0/4216007830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 msgr2=0x7f75b819e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 0x7f75b819e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 -- 172.21.15.138:0/4216007830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b819ede0 con 0x7f75b8102910 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.421+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f75a8005940 tx=0x7f75a80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.422+0000 7f75af400640 1 -- 172.21.15.138:0/4216007830 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a803f030 con 0x7f75b8102910 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.422+0000 7f75af400640 1 -- 172.21.15.138:0/4216007830 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a80044c0 con 0x7f75b8102910 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.422+0000 7f75af400640 1 -- 172.21.15.138:0/4216007830 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8035070 con 0x7f75b8102910 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.422+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b819f070 con 0x7f75b8102910 2024-08-20T01:17:16.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.422+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a5790 con 0x7f75b8102910 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.425+0000 7f75af400640 1 -- 172.21.15.138:0/4216007830 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f75a8042030 con 0x7f75b8102910 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 Processor -- start 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 -- start start 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 0x55b2628aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2628aab40 0x55b2628acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b26283b670 con 0x55b26289e290 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b26280fd70 con 0x55b26289ebb0 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.426+0000 7f75c07f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b2627ff270 con 0x55b2628aab40 2024-08-20T01:17:16.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59666/0 (socket says 172.21.15.138:59666) 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 -- 172.21.15.138:0/2594426058 learned_addr learned my addr 172.21.15.138:0/2594426058 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b6a00640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 0x55b2628aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b6000640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2628aab40 0x55b2628acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75af400640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 0x7f759c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 -- 172.21.15.138:0/2594426058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2628aab40 msgr2=0x55b2628acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2628aab40 0x55b2628acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 -- 172.21.15.138:0/2594426058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 msgr2=0x55b2628aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 0x55b2628aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75b7400640 1 -- 172.21.15.138:0/2594426058 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26279b870 con 0x55b26289e290 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.427+0000 7f75af400640 1 -- 172.21.15.138:0/4216007830 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f75a8104520 con 0x7f75b8102910 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75b6a00640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 0x55b2628aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:16.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75b7400640 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009fc0 tx=0x7f75b0010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7602cef18cb7279a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75b7400640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 0x7f759c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75a3e00640 1 -- 172.21.15.138:0/2594426058 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0004040 con 0x55b26289e290 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75a3e00640 1 -- 172.21.15.138:0/2594426058 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b00046f0 con 0x55b26289e290 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.428+0000 7f75a3e00640 1 -- 172.21.15.138:0/2594426058 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b001ea90 con 0x55b26289e290 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.429+0000 7f75c07f4740 1 -- 172.21.15.138:0/2594426058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 msgr2=0x55b26289e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.429+0000 7f75c07f4740 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009fc0 tx=0x7f75b0010860 comp rx=0 tx=0).stop 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.429+0000 7f75b7400640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 0x7f759c0b62a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75b001f000 tx=0x7f75b0020000 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 -- 172.21.15.138:0/2594426058 shutdown_connections 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b2628aab40 0x55b2628acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b26289ebb0 0x55b2628aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 --2- 172.21.15.138:0/2594426058 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b26289e670 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 -- 172.21.15.138:0/2594426058 >> 172.21.15.138:0/2594426058 conn(0x55b26289c0c0 msgr2=0x55b26289d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 -- 172.21.15.138:0/2594426058 shutdown_connections 2024-08-20T01:17:16.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 -- 172.21.15.138:0/2594426058 wait complete. 2024-08-20T01:17:16.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.430+0000 7f75c07f4740 1 -- start start 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 Processor -- start 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289ebb0 0x55b2628a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b262785ff0 con 0x55b26289ebb0 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b2628032b0 con 0x55b2628aab40 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b262661bb0 con 0x55b26289e290 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.431+0000 7f75c07f4740 10 client.0 _ll_register_callbacks cb 0x55b26250ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42654/0 (socket says 172.21.15.138:42654) 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 -- 172.21.15.138:0/1906547696 learned_addr learned my addr 172.21.15.138:0/1906547696 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b7400640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6000640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289ebb0 0x55b2628a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 -- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 msgr2=0x55b2628a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 -- 172.21.15.138:0/1906547696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289ebb0 msgr2=0x55b2628a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289ebb0 0x55b2628a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 -- 172.21.15.138:0/1906547696 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26255eb00 con 0x55b26289e290 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75b6a00640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a8107f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91d9251446f1d32 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.432+0000 7f75a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8049070 con 0x55b26289e290 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b00046f0 con 0x55b26289e290 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f75a8104130 con 0x55b26289e290 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f75a80ca840 con 0x55b26289e290 2024-08-20T01:17:16.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 10 client.26572 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b2625d5960 con 0x55b26289e290 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75b00046f0 con 0x55b26289e290 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f75a804c060 con 0x55b26289e290 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 10 client.26572 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 10 client.26572 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 20 client.26572 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b2628ba0a0 0x55b2628bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 4 client.26572 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.433+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b262811cc0 con 0x55b2628ba0a0 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.434+0000 7f75b6000640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b2628ba0a0 0x55b2628bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.434+0000 7f75b6000640 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b2628ba0a0 0x55b2628bc4b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8133d2575d9fa238 server_cookie=36c0f62e91c1b067 in_seq=0 out_seq=0 2024-08-20T01:17:16.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.434+0000 7f75a0c00640 10 client.26572 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75a0c00640 1 -- 172.21.15.138:0/1906547696 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b262811cc0 con 0x55b2628ba0a0 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75a0c00640 10 client.26572 handle_command_reply: tid=0 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 client.26572 shutdown 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 msgr2=0x55b2628aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f75a8006010 tx=0x7f75a8107f90 comp rx=0 tx=0).stop 2024-08-20T01:17:16.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 shutdown_connections 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b2628ba0a0 0x55b2628bc4b0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289ebb0 0x55b2628a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 --2- 172.21.15.138:0/1906547696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289e290 0x55b2628aa360 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 >> 172.21.15.138:0/1906547696 conn(0x55b26289c0c0 msgr2=0x55b2628a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.435+0000 7f75a0c00640 0 client.26572 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.436+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 shutdown_connections 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.436+0000 7f75c07f4740 1 -- 172.21.15.138:0/1906547696 wait complete. 2024-08-20T01:17:16.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.436+0000 7f75c07f4740 20 client.26572 trim_cache size 0 max 16384 2024-08-20T01:17:16.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.453+0000 7f75c07f4740 1 Processor -- start 2024-08-20T01:17:16.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.453+0000 7f75c07f4740 1 -- start start 2024-08-20T01:17:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81afff0 con 0x55b26289e290 2024-08-20T01:17:16.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ad8c0 con 0x55b2628aab40 2024-08-20T01:17:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75c07f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ab730 con 0x55b26289ebb0 2024-08-20T01:17:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59696/0 (socket says 172.21.15.138:59696) 2024-08-20T01:17:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 -- 172.21.15.138:0/3398707121 learned_addr learned my addr 172.21.15.138:0/3398707121 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:16.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b7400640 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 -- 172.21.15.138:0/3398707121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 msgr2=0x55b2628c1dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628c1dd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 -- 172.21.15.138:0/3398707121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 msgr2=0x55b2628c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.454+0000 7f75b6a00640 1 -- 172.21.15.138:0/3398707121 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2628aa6d0 con 0x55b26289e290 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.455+0000 7f75b6a00640 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f75a8032af0 tx=0x7f75a81047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4cc55c37364b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.455+0000 7f75a3400640 1 -- 172.21.15.138:0/3398707121 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a80e5070 con 0x55b26289e290 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.455+0000 7f75a3400640 1 -- 172.21.15.138:0/3398707121 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75a8101ce0 con 0x55b26289e290 2024-08-20T01:17:16.455 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.455+0000 7f75a3400640 1 -- 172.21.15.138:0/3398707121 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a8049040 con 0x55b26289e290 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- 172.21.15.138:0/3398707121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 msgr2=0x55b2628b6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f75a8032af0 tx=0x7f75a81047a0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- 172.21.15.138:0/3398707121 shutdown_connections 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b2628c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 --2- 172.21.15.138:0/3398707121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628b6760 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- 172.21.15.138:0/3398707121 >> 172.21.15.138:0/3398707121 conn(0x55b26289c0c0 msgr2=0x55b2628a9600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.456 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- 172.21.15.138:0/3398707121 shutdown_connections 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- 172.21.15.138:0/3398707121 wait complete. 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.456+0000 7f75c07f4740 1 -- start start 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 Processor -- start 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b262923da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81abd80 con 0x55b26289e290 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ad1f0 con 0x55b2628aab40 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f75b81ac450 con 0x55b26289ebb0 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75c07f4740 10 client.0 _ll_register_callbacks cb 0x7f75a803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75b6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59700/0 (socket says 172.21.15.138:59700) 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75b6a00640 1 -- 172.21.15.138:0/4076139229 learned_addr learned my addr 172.21.15.138:0/4076139229 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:16.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.457+0000 7f75b6000640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.459 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 -- 172.21.15.138:0/4076139229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 msgr2=0x55b2628aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 -- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 msgr2=0x55b262923da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b262923da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 -- 172.21.15.138:0/4076139229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2628af870 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75b6a00640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f75a803d040 tx=0x7f75a80d3be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3c31714cb4d3e7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75a0c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a808d030 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f75a80c7320 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.458+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b26280d760 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f75a8049030 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 10 client.16560 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b262925450 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b26280d760 con 0x55b26289e290 2024-08-20T01:17:16.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7f75a80c9a00 con 0x55b26289e290 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 10 client.16560 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 10 client.16560 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 20 client.16560 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b262925900 0x55b262927d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 4 client.16560 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "be3839cb", "release": true, "await": true}] 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.459+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "be3839cb", "release": true, "await": true}) -- 0x55b26280aea0 con 0x55b262925900 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.460+0000 7f75b6000640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b262925900 0x55b262927d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.460+0000 7f75b6000640 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b262925900 0x55b262927d20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7a666bc7cb4f85 server_cookie=444c62b5549873f9 in_seq=0 out_seq=0 2024-08-20T01:17:16.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.460+0000 7f75a0c00640 10 client.16560 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:16.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.464+0000 7f75a0c00640 1 -- 172.21.15.138:0/4076139229 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b26280aea0 con 0x55b262925900 2024-08-20T01:17:16.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.464+0000 7f75a0c00640 10 client.16560 handle_command_reply: tid=0 2024-08-20T01:17:16.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.464+0000 7f75c07f4740 1 client.16560 shutdown 2024-08-20T01:17:16.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 msgr2=0x55b2628aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f75a803d040 tx=0x7f75a80d3be0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 shutdown_connections 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b262925900 0x55b262927d20 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b2628aab40 0x55b262923da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b26289ebb0 0x55b2628aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 --2- 172.21.15.138:0/4076139229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b26289e290 0x55b2628aea40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 >> 172.21.15.138:0/4076139229 conn(0x55b26289c0c0 msgr2=0x55b2628a6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75a0c00640 0 client.16560 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 shutdown_connections 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 1 -- 172.21.15.138:0/4076139229 wait complete. 2024-08-20T01:17:16.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.465+0000 7f75c07f4740 20 client.16560 trim_cache size 0 max 16384 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 39, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 36, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "be3839cb": { 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 36, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:17:16.466 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 147.8, 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:16.467 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:17:16.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 msgr2=0x7f759c0b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 0x7f759c0b62a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f75b001f000 tx=0x7f75b0020000 comp rx=0 tx=0).stop 2024-08-20T01:17:16.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 msgr2=0x7f75b8198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:16.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f75a8005940 tx=0x7f75a80040c0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 shutdown_connections 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f759c0b3de0 0x7f759c0b62a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75b8108ca0 0x7f75b819e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f75b8108320 0x7f75b8199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 --2- 172.21.15.138:0/4216007830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f75b8102910 0x7f75b8198d30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 >> 172.21.15.138:0/4216007830 conn(0x7f75b80fe3c0 msgr2=0x7f75b81007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 shutdown_connections 2024-08-20T01:17:16.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:16.466+0000 7f75b7e00640 1 -- 172.21.15.138:0/4216007830 wait complete. 2024-08-20T01:17:17.054 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:16 smithi138 ceph-mon[18076]: pgmap v644: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 741 KiB/s wr, 38 op/s 2024-08-20T01:17:17.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 366999 3.78 MB/sec execute 329 sec latency 27425.265 ms 2024-08-20T01:17:17.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:16 smithi167 ceph-mon[21965]: pgmap v644: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 741 KiB/s wr, 38 op/s 2024-08-20T01:17:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:16 smithi142 ceph-mon[21799]: pgmap v644: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 741 KiB/s wr, 38 op/s 2024-08-20T01:17:17.740 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'be3839cb', total seconds elapsed: 32.5 2024-08-20T01:17:17.740 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.85 2024-08-20T01:17:17.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:17:18.142 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:18.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 367032 3.77 MB/sec execute 330 sec latency 982.660 ms 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 -- 172.21.15.138:0/3013624314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc077400 msgr2=0x7f2fbc0777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 --2- 172.21.15.138:0/3013624314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc077400 0x7f2fbc0777e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009900 tx=0x7f2fb002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 -- 172.21.15.138:0/3013624314 shutdown_connections 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 --2- 172.21.15.138:0/3013624314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc113690 0x7f2fbc115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 --2- 172.21.15.138:0/3013624314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077d20 0x7f2fbc113150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 --2- 172.21.15.138:0/3013624314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc077400 0x7f2fbc0777e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 -- 172.21.15.138:0/3013624314 >> 172.21.15.138:0/3013624314 conn(0x7f2fbc100730 msgr2=0x7f2fbc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 -- 172.21.15.138:0/3013624314 shutdown_connections 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.777+0000 7f2fc4600640 1 -- 172.21.15.138:0/3013624314 wait complete. 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.778+0000 7f2fc4600640 1 Processor -- start 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.778+0000 7f2fc4600640 1 -- start start 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc077d20 0x7f2fbc19fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc113690 0x7f2fbc1a4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc1190f0 con 0x7f2fbc113690 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc118f70 con 0x7f2fbc077d20 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc4600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc119270 con 0x7f2fbc077400 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc2600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc113690 0x7f2fbc1a4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:18.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:33208/0 (socket says 172.21.15.138:33208) 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 -- 172.21.15.138:0/2526045238 learned_addr learned my addr 172.21.15.138:0/2526045238 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1200640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc077d20 0x7f2fbc19fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 -- 172.21.15.138:0/2526045238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc077d20 msgr2=0x7f2fbc19fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc077d20 0x7f2fbc19fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 -- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc113690 msgr2=0x7f2fbc1a4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc113690 0x7f2fbc1a4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.779+0000 7f2fc1c00640 1 -- 172.21.15.138:0/2526045238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc1a5680 con 0x7f2fbc077400 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fc1c00640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0005ab0 tx=0x7f2fb0003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb003f070 con 0x7f2fbc077400 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb0031b60 con 0x7f2fbc077400 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fbc1a5910 con 0x7f2fbc077400 2024-08-20T01:17:18.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0035030 con 0x7f2fbc077400 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.780+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fbc1ac030 con 0x7f2fbc077400 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.781+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fbc0777e0 con 0x7f2fbc077400 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.783+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2fb0042070 con 0x7f2fbc077400 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.784+0000 7f2fba000640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 0x7f2fa40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.785+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f2fb0105160 con 0x7f2fbc077400 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.785+0000 7f2fc1200640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 0x7f2fa40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.785+0000 7f2fc1200640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 0x7f2fa40b6200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc115d20 tx=0x7f2fa8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:18.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:18.786+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2fb00cc0d0 con 0x7f2fbc077400 2024-08-20T01:17:19.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.007+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2fbc1ac260 con 0x7f2fbc077400 2024-08-20T01:17:19.007 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.008+0000 7f2fba000640 1 -- 172.21.15.138:0/2526045238 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11932 (secure 0 0 0) 0x7f2fb00cb9d0 con 0x7f2fbc077400 2024-08-20T01:17:19.008 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:17:19.008 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":39,"btime":"2024-08-20T01:16:04:725509+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:16:04.226894+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:17:19.008 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 39 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 msgr2=0x7f2fa40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 0x7f2fa40b6200 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc115d20 tx=0x7f2fa8005f90 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 msgr2=0x7f2fbc19f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0005ab0 tx=0x7f2fb0003c10 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 shutdown_connections 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2fa40b3d40 0x7f2fa40b6200 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2fbc113690 0x7f2fbc1a4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2fbc077d20 0x7f2fbc19fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 --2- 172.21.15.138:0/2526045238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2fbc077400 0x7f2fbc19f5d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 >> 172.21.15.138:0/2526045238 conn(0x7f2fbc100730 msgr2=0x7f2fbc102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 shutdown_connections 2024-08-20T01:17:19.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:19.011+0000 7f2fc4600640 1 -- 172.21.15.138:0/2526045238 wait complete. 2024-08-20T01:17:19.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 367032 3.75 MB/sec execute 331 sec latency 1982.775 ms 2024-08-20T01:17:19.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:18 smithi167 ceph-mon[21965]: pgmap v645: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 750 KiB/s wr, 39 op/s 2024-08-20T01:17:19.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:18 smithi142 ceph-mon[21799]: pgmap v645: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 750 KiB/s wr, 39 op/s 2024-08-20T01:17:19.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:18 smithi138 ceph-mon[18076]: pgmap v645: 129 pgs: 129 active+clean; 253 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 750 KiB/s wr, 39 op/s 2024-08-20T01:17:19.836 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 147.85 --await-for 45.0' 2024-08-20T01:17:19.836 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 147.85 --await-for 45.0 2024-08-20T01:17:20.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:19 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2526045238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:20.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 369170 3.78 MB/sec execute 332 sec latency 2040.357 ms 2024-08-20T01:17:20.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:19 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2526045238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:20.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:19 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2526045238' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:20.275 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:21.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:20 smithi138 ceph-mon[18076]: pgmap v646: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 732 KiB/s wr, 51 op/s 2024-08-20T01:17:21.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370403 3.79 MB/sec execute 333 sec latency 25.829 ms 2024-08-20T01:17:21.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:20 smithi167 ceph-mon[21965]: pgmap v646: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 732 KiB/s wr, 51 op/s 2024-08-20T01:17:21.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:20 smithi142 ceph-mon[21799]: pgmap v646: 129 pgs: 129 active+clean; 253 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 732 KiB/s wr, 51 op/s 2024-08-20T01:17:21.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 -- 172.21.15.138:0/2519970559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 msgr2=0x7ff494103bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 --2- 172.21.15.138:0/2519970559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff494103bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff484009900 tx=0x7ff48402a410 comp rx=0 tx=0).stop 2024-08-20T01:17:21.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 -- 172.21.15.138:0/2519970559 shutdown_connections 2024-08-20T01:17:21.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 --2- 172.21.15.138:0/2519970559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff49410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 --2- 172.21.15.138:0/2519970559 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 0x7ff49410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 --2- 172.21.15.138:0/2519970559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff494103bf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 -- 172.21.15.138:0/2519970559 >> 172.21.15.138:0/2519970559 conn(0x7ff4940fd2d0 msgr2=0x7ff4940ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.302+0000 7ff499600640 1 -- 172.21.15.138:0/2519970559 shutdown_connections 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.303+0000 7ff499600640 1 -- 172.21.15.138:0/2519970559 wait complete. 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 Processor -- start 2024-08-20T01:17:21.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 -- start start 2024-08-20T01:17:21.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 0x7ff49419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.304 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff4941a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff494112930 con 0x7ff49410cc70 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941127b0 con 0x7ff494103810 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.304+0000 7ff499600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff494112ab0 con 0x7ff494104130 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37420/0 (socket says 172.21.15.138:37420) 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 -- 172.21.15.138:0/3482275864 learned_addr learned my addr 172.21.15.138:0/3482275864 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff498c00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff4941a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493400640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 0x7ff49419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 -- 172.21.15.138:0/3482275864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 msgr2=0x7ff49419b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 0x7ff49419b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 -- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 msgr2=0x7ff4941a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff4941a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff493e00640 1 -- 172.21.15.138:0/3482275864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4941a1010 con 0x7ff494103810 2024-08-20T01:17:21.305 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.305+0000 7ff498c00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff4941a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:21.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff493e00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff490c00640 1 -- 172.21.15.138:0/3482275864 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48403f070 con 0x7ff494103810 2024-08-20T01:17:21.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff490c00640 1 -- 172.21.15.138:0/3482275864 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff484031b60 con 0x7ff494103810 2024-08-20T01:17:21.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4941a12a0 con 0x7ff494103810 2024-08-20T01:17:21.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff490c00640 1 -- 172.21.15.138:0/3482275864 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484035030 con 0x7ff494103810 2024-08-20T01:17:21.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.306+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4941a79c0 con 0x7ff494103810 2024-08-20T01:17:21.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.309+0000 7ff49d7cc740 1 Processor -- start 2024-08-20T01:17:21.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.309+0000 7ff49d7cc740 1 -- start start 2024-08-20T01:17:21.308 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.309+0000 7ff490c00640 1 -- 172.21.15.138:0/3482275864 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff484042070 con 0x7ff494103810 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df257410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df2633a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557df1cc600 con 0x5557df257030 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5557df1bbb00 con 0x5557df2638e0 2024-08-20T01:17:21.309 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557df1bfb40 con 0x5557df257950 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff493400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff498c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df257410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff493400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37424/0 (socket says 172.21.15.138:37424) 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.310+0000 7ff493400640 1 -- 172.21.15.138:0/1923218243 learned_addr learned my addr 172.21.15.138:0/1923218243 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 -- 172.21.15.138:0/1923218243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 msgr2=0x5557df2633a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df2633a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 -- 172.21.15.138:0/1923218243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 msgr2=0x5557df257410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.310 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df257410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.311 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 -- 172.21.15.138:0/1923218243 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557df161ee0 con 0x5557df2638e0 2024-08-20T01:17:21.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff490c00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 0x7ff4780b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.314 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.311+0000 7ff493400640 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff480009c50 tx=0x7ff480011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eff281abf47847fb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.312+0000 7ff490c00640 1 -- 172.21.15.138:0/3482275864 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7ff484104b10 con 0x7ff494103810 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.312+0000 7ff489200640 1 -- 172.21.15.138:0/1923218243 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004220 con 0x5557df2638e0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.312+0000 7ff498c00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 0x7ff4780b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.312+0000 7ff489200640 1 -- 172.21.15.138:0/1923218243 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x5557df2638e0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.312+0000 7ff489200640 1 -- 172.21.15.138:0/1923218243 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480004c70 con 0x5557df2638e0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1923218243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 msgr2=0x5557df265cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff480009c50 tx=0x7ff480011810 comp rx=0 tx=0).stop 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff498c00640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 0x7ff4780b6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1923218243 shutdown_connections 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df265cd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df2633a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1923218243 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df257410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.315 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.313+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1923218243 >> 172.21.15.138:0/1923218243 conn(0x5557df254e60 msgr2=0x5557df256000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.314+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1923218243 shutdown_connections 2024-08-20T01:17:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.314+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1923218243 wait complete. 2024-08-20T01:17:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.315+0000 7ff49d7cc740 1 -- start start 2024-08-20T01:17:21.316 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.315+0000 7ff49d7cc740 1 Processor -- start 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df260890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df261310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5557df01e7b0 con 0x5557df257030 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5557df156dd0 con 0x5557df257950 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5557df266560 con 0x5557df2638e0 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff49d7cc740 10 client.0 _ll_register_callbacks cb 0x5557deec80a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37440/0 (socket says 172.21.15.138:37440) 2024-08-20T01:17:21.317 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 -- 172.21.15.138:0/3876325307 learned_addr learned my addr 172.21.15.138:0/3876325307 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff498c00640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df261310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 -- 172.21.15.138:0/3876325307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 msgr2=0x5557df261310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df261310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.316+0000 7ff493400640 1 -- 172.21.15.138:0/3876325307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 msgr2=0x5557df260890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff493400640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df260890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff493400640 1 -- 172.21.15.138:0/3876325307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557def2aba0 con 0x5557df257950 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff498c00640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df261310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff493400640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff480009380 tx=0x7ff480019bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee74b3b725b65ea1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff47d600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.317+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480027030 con 0x5557df257950 2024-08-20T01:17:21.318 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4800043c0 con 0x5557df257950 2024-08-20T01:17:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7ff480019db0 con 0x5557df257950 2024-08-20T01:17:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff480034050 con 0x5557df257950 2024-08-20T01:17:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff49d7cc740 10 client.26630 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5557df272130 con 0x5557df257950 2024-08-20T01:17:21.320 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.318+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800043c0 con 0x5557df257950 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff480023c40 con 0x5557df257950 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 10 client.26630 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 10 client.26630 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 20 client.26630 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2734e0 0x5557df2758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 4 client.26630 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557df1ce550 con 0x5557df2734e0 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.319+0000 7ff493e00640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2734e0 0x5557df2758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.320+0000 7ff493e00640 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2734e0 0x5557df2758d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f133c3d9363e7ec server_cookie=5b7eec9565c2482a in_seq=0 out_seq=0 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.320+0000 7ff47d600640 10 client.26630 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.320+0000 7ff47d600640 1 -- 172.21.15.138:0/3876325307 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5557df1ce550 con 0x5557df2734e0 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.320+0000 7ff47d600640 10 client.26630 handle_command_reply: tid=0 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.320+0000 7ff49d7cc740 1 client.26630 shutdown 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 msgr2=0x5557df260dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.321 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff480009380 tx=0x7ff480019bc0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 shutdown_connections 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2734e0 0x5557df2758d0 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df261310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257950 0x5557df260dd0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/3876325307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df260890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 >> 172.21.15.138:0/3876325307 conn(0x5557df254e60 msgr2=0x5557df25a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff47d600640 0 client.26630 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 shutdown_connections 2024-08-20T01:17:21.322 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 1 -- 172.21.15.138:0/3876325307 wait complete. 2024-08-20T01:17:21.323 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.321+0000 7ff49d7cc740 20 client.26630 trim_cache size 0 max 16384 2024-08-20T01:17:21.341 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.342+0000 7ff49d7cc740 1 Processor -- start 2024-08-20T01:17:21.342 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.342+0000 7ff49d7cc740 1 -- start start 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df275720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df253e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b43f0 con 0x5557df257030 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b1cc0 con 0x5557df2638e0 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941afb30 con 0x5557df257950 2024-08-20T01:17:21.343 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff498c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff493e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df275720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff498c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37456/0 (socket says 172.21.15.138:37456) 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff493e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df275720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58120/0 (socket says 172.21.15.138:58120) 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff498c00640 1 -- 172.21.15.138:0/1086787369 learned_addr learned my addr 172.21.15.138:0/1086787369 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.343+0000 7ff493400640 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df253e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 -- 172.21.15.138:0/1086787369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 msgr2=0x5557df253e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df253e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 -- 172.21.15.138:0/1086787369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 msgr2=0x5557df275720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df275720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 -- 172.21.15.138:0/1086787369 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557df2558e0 con 0x5557df2638e0 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff493400640 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df253e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:21.344 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff498c00640 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0109a0 tx=0x7ff48c011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94c57f5e117be7e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff47fe00640 1 -- 172.21.15.138:0/1086787369 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c02c070 con 0x5557df2638e0 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff47fe00640 1 -- 172.21.15.138:0/1086787369 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c01edd0 con 0x5557df2638e0 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.344+0000 7ff47fe00640 1 -- 172.21.15.138:0/1086787369 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c027910 con 0x5557df2638e0 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1086787369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 msgr2=0x5557df27b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0109a0 tx=0x7ff48c011f90 comp rx=0 tx=0).stop 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1086787369 shutdown_connections 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df2638e0 0x5557df27b0c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df257950 0x5557df253e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/1086787369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257030 0x5557df275720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1086787369 >> 172.21.15.138:0/1086787369 conn(0x5557df254e60 msgr2=0x5557df253760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.346 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1086787369 shutdown_connections 2024-08-20T01:17:21.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.345+0000 7ff49d7cc740 1 -- 172.21.15.138:0/1086787369 wait complete. 2024-08-20T01:17:21.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.346+0000 7ff49d7cc740 1 -- start start 2024-08-20T01:17:21.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.346+0000 7ff49d7cc740 1 Processor -- start 2024-08-20T01:17:21.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.346+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.347 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 0x5557df269540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df269c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b0180 con 0x5557df257950 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b15f0 con 0x5557df257030 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff4941b0850 con 0x5557df2638e0 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff49d7cc740 10 client.0 _ll_register_callbacks cb 0x7ff48c0099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.348 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37466/0 (socket says 172.21.15.138:37466) 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 -- 172.21.15.138:0/2297326703 learned_addr learned my addr 172.21.15.138:0/2297326703 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493400640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 0x5557df269540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff498c00640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df269c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 -- 172.21.15.138:0/2297326703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 msgr2=0x5557df269c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df269c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 -- 172.21.15.138:0/2297326703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 msgr2=0x5557df269540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 0x5557df269540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493e00640 1 -- 172.21.15.138:0/2297326703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557df2dd0a0 con 0x5557df257030 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff498c00640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df269c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.347+0000 7ff493400640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 0x5557df269540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff493e00640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff484101ec0 tx=0x7ff4840caef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3cbacf2b2631cdc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff47d600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:17:21.349 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4840e5e60 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7ff48403b460 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48c01edd0 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ff484049040 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff49d7cc740 10 client.26636 fetch_fsmap learned FSMap version 39 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5557df2de2e0 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.348+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff48c01edd0 con 0x5557df257030 2024-08-20T01:17:21.350 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 39) ==== 8354+0+0 (secure 0 0 0) 0x7ff4840cb580 con 0x5557df257030 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 10 client.26636 fetch_fsmap finished waiting for FSMap version 39 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 10 client.26636 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 20 client.26636 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2dea50 0x5557df2e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 4 client.26636 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.85, "await_for": 45.0}] 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.85, "await_for": 45.0}) -- 0x5557df1c9ff0 con 0x5557df2dea50 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.349+0000 7ff493400640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2dea50 0x5557df2e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.350+0000 7ff493400640 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2dea50 0x5557df2e0e70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee9b8daa9278f9ac server_cookie=cafb52db480f6500 in_seq=0 out_seq=0 2024-08-20T01:17:21.351 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.350+0000 7ff47d600640 10 client.26636 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:21.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff47d600640 1 -- 172.21.15.138:0/2297326703 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5557df1c9ff0 con 0x5557df2dea50 2024-08-20T01:17:21.452 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff47d600640 10 client.26636 handle_command_reply: tid=0 2024-08-20T01:17:21.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 client.26636 shutdown 2024-08-20T01:17:21.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 msgr2=0x5557df26c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff484101ec0 tx=0x7ff4840caef0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 shutdown_connections 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5557df2dea50 0x5557df2e0e70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5557df2638e0 0x5557df269c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5557df257950 0x5557df269540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 --2- 172.21.15.138:0/2297326703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5557df257030 0x5557df26c050 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 >> 172.21.15.138:0/2297326703 conn(0x5557df254e60 msgr2=0x5557df253f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff47d600640 0 client.26636 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 shutdown_connections 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 1 -- 172.21.15.138:0/2297326703 wait complete. 2024-08-20T01:17:21.454 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.450+0000 7ff49d7cc740 20 client.26636 trim_cache size 0 max 16384 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 39, 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 38, 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "a94abfe1": { 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 38, 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:21.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 147.8, 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:21.456 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 msgr2=0x7ff4780b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 0x7ff4780b6320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff48c005fd0 tx=0x7ff48c009500 comp rx=0 tx=0).stop 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 msgr2=0x7ff49419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff484005ab0 tx=0x7ff484003c10 comp rx=0 tx=0).stop 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 shutdown_connections 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff4780b3e60 0x7ff4780b6320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff49410cc70 0x7ff4941a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.457 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff494104130 0x7ff49419b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 --2- 172.21.15.138:0/3482275864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff494103810 0x7ff49419af60 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 >> 172.21.15.138:0/3482275864 conn(0x7ff4940fd2d0 msgr2=0x7ff49410ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 shutdown_connections 2024-08-20T01:17:21.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:21.455+0000 7ff499600640 1 -- 172.21.15.138:0/3482275864 wait complete. 2024-08-20T01:17:22.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.78 MB/sec execute 334 sec latency 795.559 ms 2024-08-20T01:17:22.330 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a94abfe1', quiesce took 4.6 seconds. Will release after: 23.25 2024-08-20T01:17:23.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.77 MB/sec execute 335 sec latency 1795.653 ms 2024-08-20T01:17:23.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:22 smithi167 ceph-mon[21965]: pgmap v647: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 800 KiB/s wr, 54 op/s 2024-08-20T01:17:23.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:22 smithi142 ceph-mon[21799]: pgmap v647: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 800 KiB/s wr, 54 op/s 2024-08-20T01:17:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:22 smithi138 ceph-mon[18076]: pgmap v647: 129 pgs: 129 active+clean; 254 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 800 KiB/s wr, 54 op/s 2024-08-20T01:17:24.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.75 MB/sec execute 336 sec latency 2795.706 ms 2024-08-20T01:17:25.177 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.74 MB/sec execute 337 sec latency 3795.863 ms 2024-08-20T01:17:25.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:25 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:17:25.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:25 smithi138 ceph-mon[18076]: pgmap v648: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-08-20T01:17:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:25 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:17:25.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:25 smithi167 ceph-mon[21965]: pgmap v648: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-08-20T01:17:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:25 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:17:25.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:25 smithi142 ceph-mon[21799]: pgmap v648: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.1 MiB/s wr, 90 op/s 2024-08-20T01:17:26.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.73 MB/sec execute 338 sec latency 4795.988 ms 2024-08-20T01:17:27.067 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:26 smithi138 ceph-mon[18076]: pgmap v649: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-08-20T01:17:27.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.72 MB/sec execute 339 sec latency 5796.147 ms 2024-08-20T01:17:27.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:26 smithi167 ceph-mon[21965]: pgmap v649: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-08-20T01:17:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:26 smithi142 ceph-mon[21799]: pgmap v649: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 79 op/s 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 -- 172.21.15.138:0/850931039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077400 msgr2=0x7f5f480777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 --2- 172.21.15.138:0/850931039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077400 0x7f5f480777e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009900 tx=0x7f5f3802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 -- 172.21.15.138:0/850931039 shutdown_connections 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 --2- 172.21.15.138:0/850931039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 0x7f5f481157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 --2- 172.21.15.138:0/850931039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077d20 0x7f5f48112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 --2- 172.21.15.138:0/850931039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077400 0x7f5f480777e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 -- 172.21.15.138:0/850931039 >> 172.21.15.138:0/850931039 conn(0x7f5f48100460 msgr2=0x7f5f48102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 -- 172.21.15.138:0/850931039 shutdown_connections 2024-08-20T01:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.469+0000 7f5f47e00640 1 -- 172.21.15.138:0/850931039 wait complete. 2024-08-20T01:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.470+0000 7f5f47e00640 1 Processor -- start 2024-08-20T01:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.470+0000 7f5f47e00640 1 -- start start 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 0x7f5f4819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 0x7f5f481a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48118e10 con 0x7f5f48077d20 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48118c90 con 0x7f5f48113400 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48118f90 con 0x7f5f48077400 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58160/0 (socket says 172.21.15.138:58160) 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 -- 172.21.15.138:0/190914499 learned_addr learned my addr 172.21.15.138:0/190914499 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f47400640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 0x7f5f481a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46a00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 0x7f5f4819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 -- 172.21.15.138:0/190914499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 msgr2=0x7f5f4819f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 0x7f5f4819f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 -- 172.21.15.138:0/190914499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 msgr2=0x7f5f481a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 0x7f5f481a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46000640 1 -- 172.21.15.138:0/190914499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f481a5430 con 0x7f5f48077d20 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.471+0000 7f5f46a00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 0x7f5f4819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f46000640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400e9f0 tx=0x7f5f3400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f3f400640 1 -- 172.21.15.138:0/190914499 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400cd80 con 0x7f5f48077d20 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f481a5720 con 0x7f5f48077d20 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f3f400640 1 -- 172.21.15.138:0/190914499 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f340054d0 con 0x7f5f48077d20 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f3f400640 1 -- 172.21.15.138:0/190914499 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34010690 con 0x7f5f48077d20 2024-08-20T01:17:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.472+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481abde0 con 0x7f5f48077d20 2024-08-20T01:17:27.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.475+0000 7f5f50c51740 1 Processor -- start 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.475+0000 7f5f50c51740 1 -- start start 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.475+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.475+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.475+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff7b080 0x55832ff7d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f50c51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55832fee4210 con 0x55832ff6e7d0 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f50c51740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55832fed3710 con 0x55832ff6f0f0 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f50c51740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55832fed7750 con 0x55832ff7b080 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f3f400640 1 -- 172.21.15.138:0/190914499 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5f34004e90 con 0x7f5f48077d20 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58170/0 (socket says 172.21.15.138:58170) 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 -- 172.21.15.138:0/341209337 learned_addr learned my addr 172.21.15.138:0/341209337 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:27.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f46a00640 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 -- 172.21.15.138:0/341209337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff7b080 msgr2=0x55832ff7d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff7b080 0x55832ff7d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 -- 172.21.15.138:0/341209337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 msgr2=0x55832ff7ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff7ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 -- 172.21.15.138:0/341209337 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832fe9fde0 con 0x55832ff6e7d0 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.476+0000 7f5f47400640 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009f90 tx=0x7f5f400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b93250d647f4e66 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f33e00640 1 -- 172.21.15.138:0/341209337 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40004d20 con 0x55832ff6e7d0 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f33e00640 1 -- 172.21.15.138:0/341209337 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f40004050 con 0x55832ff6e7d0 2024-08-20T01:17:27.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f33e00640 1 -- 172.21.15.138:0/341209337 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40005280 con 0x55832ff6e7d0 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 -- 172.21.15.138:0/341209337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 msgr2=0x55832ff6ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009f90 tx=0x7f5f400118d0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f3f400640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 0x7f5f2c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 -- 172.21.15.138:0/341209337 shutdown_connections 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff7b080 0x55832ff7d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 --2- 172.21.15.138:0/341209337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff6e7d0 0x55832ff6ebb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.477+0000 7f5f50c51740 1 -- 172.21.15.138:0/341209337 >> 172.21.15.138:0/341209337 conn(0x55832ff6c600 msgr2=0x55832ff6d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f3f400640 1 -- 172.21.15.138:0/190914499 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f5f340e4840 con 0x7f5f48077d20 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f47400640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 0x7f5f2c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f50c51740 1 -- 172.21.15.138:0/341209337 shutdown_connections 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f50c51740 1 -- 172.21.15.138:0/341209337 wait complete. 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f50c51740 1 -- start start 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f47400640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 0x7f5f2c0b6280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f40004910 tx=0x7f5f40000f40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.478+0000 7f5f50c51740 1 Processor -- start 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff77ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff78a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55832fe0cab0 con 0x55832ff7b080 2024-08-20T01:17:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55832fe7a140 con 0x55832ff6f0f0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55832ff7dd60 con 0x55832ff6e7d0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f50c51740 10 client.0 _ll_register_callbacks cb 0x55832fe8d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f46000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f46000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37490/0 (socket says 172.21.15.138:37490) 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f46000640 1 -- 172.21.15.138:0/995971684 learned_addr learned my addr 172.21.15.138:0/995971684 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f46a00640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff77ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.479+0000 7f5f47400640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff78a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 -- 172.21.15.138:0/995971684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 msgr2=0x55832ff77ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff77ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 -- 172.21.15.138:0/995971684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 msgr2=0x55832ff78a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff78a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 -- 172.21.15.138:0/995971684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832fe7b170 con 0x55832ff6f0f0 2024-08-20T01:17:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46a00640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f47400640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f46000640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400e9c0 tx=0x7f5f340048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29fe9316e8dbf806 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f340e2020 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55832feaaa30 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f5f340e43e0 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5f340ab750 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f50c51740 10 client.26648 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55832fc36790 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.480+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55832feaaa30 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f5f340b3c70 con 0x55832ff6f0f0 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 10 client.26648 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 10 client.26648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:27.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 20 client.26648 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832ff8b020 0x55832ff8d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 4 client.26648 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55832fee6160 con 0x55832ff8b020 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f46a00640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832ff8b020 0x55832ff8d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f46a00640 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832ff8b020 0x55832ff8d410 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1072d0bc0200dcde server_cookie=c237836d0c501c37 in_seq=0 out_seq=0 2024-08-20T01:17:27.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.481+0000 7f5f30c00640 10 client.26648 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.482+0000 7f5f30c00640 1 -- 172.21.15.138:0/995971684 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55832fee6160 con 0x55832ff8b020 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.482+0000 7f5f30c00640 10 client.26648 handle_command_reply: tid=0 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 client.26648 shutdown 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 msgr2=0x55832ff78530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400e9c0 tx=0x7f5f340048f0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 shutdown_connections 2024-08-20T01:17:27.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832ff8b020 0x55832ff8d410 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff78a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff78530 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f30c00640 0 client.26648 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 --2- 172.21.15.138:0/995971684 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff77ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 >> 172.21.15.138:0/995971684 conn(0x55832ff6c600 msgr2=0x55832ff721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 shutdown_connections 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 1 -- 172.21.15.138:0/995971684 wait complete. 2024-08-20T01:17:27.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.483+0000 7f5f50c51740 20 client.26648 trim_cache size 0 max 16384 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 Processor -- start 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 -- start start 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff92c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b88b0 con 0x55832ff7b080 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b6150 con 0x55832ff6f0f0 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f50c51740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b3fc0 con 0x55832ff6e7d0 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58182/0 (socket says 172.21.15.138:58182) 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f47400640 1 -- 172.21.15.138:0/1304543303 learned_addr learned my addr 172.21.15.138:0/1304543303 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f46000640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.498+0000 7f5f46a00640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff92c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 -- 172.21.15.138:0/1304543303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 msgr2=0x55832ff92c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff92c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 -- 172.21.15.138:0/1304543303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 msgr2=0x55832ff93160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff93160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 -- 172.21.15.138:0/1304543303 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55832ff6ef20 con 0x55832ff7b080 2024-08-20T01:17:27.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f46000640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f47400640 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009f90 tx=0x7f5f40021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5664d1cd1081eb79 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f33400640 1 -- 172.21.15.138:0/1304543303 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40033030 con 0x55832ff7b080 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f33400640 1 -- 172.21.15.138:0/1304543303 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f40028040 con 0x55832ff7b080 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.499+0000 7f5f33400640 1 -- 172.21.15.138:0/1304543303 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4000f4e0 con 0x55832ff7b080 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 -- 172.21.15.138:0/1304543303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 msgr2=0x55832ff79620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009f90 tx=0x7f5f40021000 comp rx=0 tx=0).stop 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 -- 172.21.15.138:0/1304543303 shutdown_connections 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff79620 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff93160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 --2- 172.21.15.138:0/1304543303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff92c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 -- 172.21.15.138:0/1304543303 >> 172.21.15.138:0/1304543303 conn(0x55832ff6c600 msgr2=0x55832ff73220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 -- 172.21.15.138:0/1304543303 shutdown_connections 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.500+0000 7f5f50c51740 1 -- 172.21.15.138:0/1304543303 wait complete. 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 -- start start 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 Processor -- start 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff83210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff83750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b4610 con 0x55832ff7b080 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b5a80 con 0x55832ff6f0f0 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481b4ce0 con 0x55832ff6e7d0 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f50c51740 10 client.0 _ll_register_callbacks cb 0x7f5f34009720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f47400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58184/0 (socket says 172.21.15.138:58184) 2024-08-20T01:17:27.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.501+0000 7f5f47400640 1 -- 172.21.15.138:0/3786908739 learned_addr learned my addr 172.21.15.138:0/3786908739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f46a00640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff83210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f46000640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff83750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 -- 172.21.15.138:0/3786908739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 msgr2=0x55832ff83210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff83210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 -- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 msgr2=0x55832ff83750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff83750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 -- 172.21.15.138:0/3786908739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55832fcaf620 con 0x55832ff7b080 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f47400640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5f40005ad0 tx=0x7f5f40005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b32de00244f2dab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f30c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f40033040 con 0x55832ff7b080 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f5f40005040 con 0x55832ff7b080 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.502+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f40028040 con 0x55832ff7b080 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f5f40003bb0 con 0x55832ff7b080 2024-08-20T01:17:27.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 10 client.16599 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55832fff5560 con 0x55832ff7b080 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f40028040 con 0x55832ff7b080 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f5f40024440 con 0x55832ff7b080 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 10 client.16599 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 10 client.16599 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 20 client.16599 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832fff5ab0 0x55832fff7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 4 client.16599 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.503+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55832fee1c00 con 0x55832fff5ab0 2024-08-20T01:17:27.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.504+0000 7f5f46a00640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832fff5ab0 0x55832fff7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.504+0000 7f5f46a00640 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832fff5ab0 0x55832fff7ed0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb1db7264b5271c1 server_cookie=369d57d8cd702b0c in_seq=0 out_seq=0 2024-08-20T01:17:27.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.504+0000 7f5f30c00640 10 client.16599 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f30c00640 1 -- 172.21.15.138:0/3786908739 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55832fee1c00 con 0x55832fff5ab0 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f30c00640 10 client.16599 handle_command_reply: tid=0 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 client.16599 shutdown 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 msgr2=0x55832ff74820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5f40005ad0 tx=0x7f5f40005b00 comp rx=0 tx=0).stop 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 shutdown_connections 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55832fff5ab0 0x55832fff7ed0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55832ff7b080 0x55832ff74820 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f30c00640 0 client.16599 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55832ff6f0f0 0x55832ff83750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.529 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 --2- 172.21.15.138:0/3786908739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55832ff6e7d0 0x55832ff83210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 >> 172.21.15.138:0/3786908739 conn(0x55832ff6c600 msgr2=0x55832ff79620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 shutdown_connections 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 1 -- 172.21.15.138:0/3786908739 wait complete. 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.529+0000 7f5f50c51740 20 client.16599 trim_cache size 0 max 16384 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:17:27.530 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 msgr2=0x7f5f2c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 0x7f5f2c0b6280 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5f40004910 tx=0x7f5f40000f40 comp rx=0 tx=0).stop 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 msgr2=0x7f5f4819f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400e9f0 tx=0x7f5f3400edf0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 shutdown_connections 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5f2c0b3dc0 0x7f5f2c0b6280 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f48113400 0x7f5f481a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.531 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f48077d20 0x7f5f4819f8d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 --2- 172.21.15.138:0/190914499 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5f48077400 0x7f5f4819f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 >> 172.21.15.138:0/190914499 conn(0x7f5f48100460 msgr2=0x7f5f48102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 shutdown_connections 2024-08-20T01:17:27.532 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:27.530+0000 7f5f47e00640 1 -- 172.21.15.138:0/190914499 wait complete. 2024-08-20T01:17:28.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.71 MB/sec execute 340 sec latency 6796.244 ms 2024-08-20T01:17:29.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.70 MB/sec execute 341 sec latency 7796.361 ms 2024-08-20T01:17:29.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:28 smithi167 ceph-mon[21965]: pgmap v650: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 947 KiB/s wr, 72 op/s 2024-08-20T01:17:29.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:28 smithi142 ceph-mon[21799]: pgmap v650: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 947 KiB/s wr, 72 op/s 2024-08-20T01:17:29.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:28 smithi138 ceph-mon[18076]: pgmap v650: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 947 KiB/s wr, 72 op/s 2024-08-20T01:17:30.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.69 MB/sec execute 342 sec latency 8796.504 ms 2024-08-20T01:17:31.178 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.68 MB/sec execute 343 sec latency 9796.671 ms 2024-08-20T01:17:31.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:30 smithi167 ceph-mon[21965]: pgmap v651: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 949 KiB/s wr, 80 op/s 2024-08-20T01:17:31.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:30 smithi142 ceph-mon[21799]: pgmap v651: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 949 KiB/s wr, 80 op/s 2024-08-20T01:17:31.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:30 smithi138 ceph-mon[18076]: pgmap v651: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 949 KiB/s wr, 80 op/s 2024-08-20T01:17:32.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.67 MB/sec execute 344 sec latency 10796.821 ms 2024-08-20T01:17:33.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.66 MB/sec execute 345 sec latency 11796.963 ms 2024-08-20T01:17:33.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:32 smithi167 ceph-mon[21965]: pgmap v652: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 635 KiB/s wr, 59 op/s 2024-08-20T01:17:33.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:32 smithi142 ceph-mon[21799]: pgmap v652: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 635 KiB/s wr, 59 op/s 2024-08-20T01:17:33.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:32 smithi138 ceph-mon[18076]: pgmap v652: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 635 KiB/s wr, 59 op/s 2024-08-20T01:17:34.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.65 MB/sec execute 346 sec latency 12797.073 ms 2024-08-20T01:17:35.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.64 MB/sec execute 347 sec latency 13797.198 ms 2024-08-20T01:17:35.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:34 smithi167 ceph-mon[21965]: pgmap v653: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 610 KiB/s wr, 58 op/s 2024-08-20T01:17:35.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:34 smithi142 ceph-mon[21799]: pgmap v653: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 610 KiB/s wr, 58 op/s 2024-08-20T01:17:35.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:34 smithi138 ceph-mon[18076]: pgmap v653: 129 pgs: 129 active+clean; 257 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 610 KiB/s wr, 58 op/s 2024-08-20T01:17:36.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.63 MB/sec execute 348 sec latency 14797.349 ms 2024-08-20T01:17:37.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.61 MB/sec execute 349 sec latency 15797.486 ms 2024-08-20T01:17:37.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:36 smithi167 ceph-mon[21965]: pgmap v654: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 347 KiB/s wr, 26 op/s 2024-08-20T01:17:37.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:36 smithi142 ceph-mon[21799]: pgmap v654: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 347 KiB/s wr, 26 op/s 2024-08-20T01:17:37.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:36 smithi138 ceph-mon[18076]: pgmap v654: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 347 KiB/s wr, 26 op/s 2024-08-20T01:17:38.179 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.60 MB/sec execute 350 sec latency 16797.613 ms 2024-08-20T01:17:39.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.59 MB/sec execute 351 sec latency 17797.730 ms 2024-08-20T01:17:39.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:38 smithi167 ceph-mon[21965]: pgmap v655: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 122 KiB/s wr, 17 op/s 2024-08-20T01:17:39.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:38 smithi142 ceph-mon[21799]: pgmap v655: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 122 KiB/s wr, 17 op/s 2024-08-20T01:17:39.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:38 smithi138 ceph-mon[18076]: pgmap v655: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 122 KiB/s wr, 17 op/s 2024-08-20T01:17:40.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.58 MB/sec execute 352 sec latency 18797.871 ms 2024-08-20T01:17:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 -- 172.21.15.138:0/1121154983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810a610 msgr2=0x7fbbe810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1121154983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810a610 0x7fbbe810aa50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009900 tx=0x7fbbd402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 -- 172.21.15.138:0/1121154983 shutdown_connections 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1121154983 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe810af90 0x7fbbe8111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1121154983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810a610 0x7fbbe810aa50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1121154983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe8104c10 0x7fbbe8104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 -- 172.21.15.138:0/1121154983 >> 172.21.15.138:0/1121154983 conn(0x7fbbe8100630 msgr2=0x7fbbe8102a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 -- 172.21.15.138:0/1121154983 shutdown_connections 2024-08-20T01:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.655+0000 7fbbedc00640 1 -- 172.21.15.138:0/1121154983 wait complete. 2024-08-20T01:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 Processor -- start 2024-08-20T01:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 -- start start 2024-08-20T01:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8104c10 0x7fbbe819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe810a610 0x7fbbe819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114ca0 con 0x7fbbe810af90 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114b20 con 0x7fbbe810a610 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.656+0000 7fbbedc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe8114e20 con 0x7fbbe8104c10 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47218/0 (socket says 172.21.15.138:47218) 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 -- 172.21.15.138:0/1249422080 learned_addr learned my addr 172.21.15.138:0/1249422080 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe810a610 0x7fbbe819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8104c10 0x7fbbe819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 -- 172.21.15.138:0/1249422080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8104c10 msgr2=0x7fbbe819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8104c10 0x7fbbe819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 -- 172.21.15.138:0/1249422080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe810a610 msgr2=0x7fbbe819b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe810a610 0x7fbbe819b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.657+0000 7fbbe7400640 1 -- 172.21.15.138:0/1249422080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe81a1200 con 0x7fbbe810af90 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000d750 tx=0x7fbbe000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbdf400640 1 -- 172.21.15.138:0/1249422080 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0017070 con 0x7fbbe810af90 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbdf400640 1 -- 172.21.15.138:0/1249422080 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbe00054d0 con 0x7fbbe810af90 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbdf400640 1 -- 172.21.15.138:0/1249422080 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004c60 con 0x7fbbe810af90 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe81a14f0 con 0x7fbbe810af90 2024-08-20T01:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.658+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe81a7bb0 con 0x7fbbe810af90 2024-08-20T01:17:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.662+0000 7fbbf0aac740 1 Processor -- start 2024-08-20T01:17:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.662+0000 7fbbdf400640 1 -- 172.21.15.138:0/1249422080 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fbbe0009bd0 con 0x7fbbe810af90 2024-08-20T01:17:40.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 -- start start 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be46ef0 0x56537be52940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be52e80 0x56537be55270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56537bdbc210 con 0x56537be465d0 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56537bdab710 con 0x56537be52e80 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56537bdaf750 con 0x56537be46ef0 2024-08-20T01:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbe6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbe6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47224/0 (socket says 172.21.15.138:47224) 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbe6000640 1 -- 172.21.15.138:0/1052990928 learned_addr learned my addr 172.21.15.138:0/1052990928 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be46ef0 0x56537be52940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.663+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be52e80 0x56537be55270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 -- 172.21.15.138:0/1052990928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be46ef0 msgr2=0x56537be52940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be46ef0 0x56537be52940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 -- 172.21.15.138:0/1052990928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be52e80 msgr2=0x56537be55270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be52e80 0x56537be55270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 -- 172.21.15.138:0/1052990928 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537bd77de0 con 0x56537be465d0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4000c00 tx=0x7fbbd4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f87982b40120a22 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbd3e00640 1 -- 172.21.15.138:0/1052990928 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4044020 con 0x56537be465d0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbd3e00640 1 -- 172.21.15.138:0/1052990928 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd4004770 con 0x56537be465d0 2024-08-20T01:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbd3e00640 1 -- 172.21.15.138:0/1052990928 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4035040 con 0x56537be465d0 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.664+0000 7fbbdf400640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 0x7fbbcc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbdf400640 1 -- 172.21.15.138:0/1249422080 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fbbe00e3cf0 con 0x7fbbe810af90 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 0x7fbbcc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1052990928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 msgr2=0x56537be469b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4000c00 tx=0x7fbbd4009560 comp rx=0 tx=0).stop 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1052990928 shutdown_connections 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be52e80 0x56537be55270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be46ef0 0x56537be52940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1052990928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be469b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1052990928 >> 172.21.15.138:0/1052990928 conn(0x56537be443b0 msgr2=0x56537be455a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1052990928 shutdown_connections 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.665+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1052990928 wait complete. 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 0x7fbbcc0b6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56537be49750 tx=0x7fbbd8009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 -- start start 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 Processor -- start 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be465d0 0x56537be4fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be502f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56537bce4ab0 con 0x56537be52e80 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56537bd52140 con 0x56537be46ef0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56537be55b00 con 0x56537be465d0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.666+0000 7fbbf0aac740 10 client.0 _ll_register_callbacks cb 0x56537bd655d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47230/0 (socket says 172.21.15.138:47230) 2024-08-20T01:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 -- 172.21.15.138:0/4050701992 learned_addr learned my addr 172.21.15.138:0/4050701992 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be465d0 0x56537be4fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 -- 172.21.15.138:0/4050701992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be465d0 msgr2=0x56537be4fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be465d0 0x56537be4fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 -- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 msgr2=0x56537be502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 -- 172.21.15.138:0/4050701992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537bd53170 con 0x56537be52e80 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbe7400640 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00a7820 tx=0x7fbbe00aa0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd2291589c94453 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0020050 con 0x56537be52e80 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fbbe00a9a20 con 0x56537be52e80 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.667+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56537bd82a30 con 0x56537be52e80 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbbe00aa5b0 con 0x56537be52e80 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 10 client.16611 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56537bb0e790 con 0x56537be52e80 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56537bd82a30 con 0x56537be52e80 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fbbe00afc90 con 0x56537be52e80 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 10 client.16611 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 20 client.16611 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537be62d70 0x56537be65160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 4 client.16611 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.668+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56537bdbe160 con 0x56537be62d70 2024-08-20T01:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.669+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537be62d70 0x56537be65160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.669+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537be62d70 0x56537be65160 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671428fb9cce3d2d server_cookie=f8805059fe39190d in_seq=0 out_seq=0 2024-08-20T01:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.669+0000 7fbbd0c00640 10 client.16611 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbd0c00640 1 -- 172.21.15.138:0/4050701992 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56537bdbe160 con 0x56537be62d70 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbd0c00640 10 client.16611 handle_command_reply: tid=0 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 client.16611 shutdown 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 msgr2=0x56537be50830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00a7820 tx=0x7fbbe00aa0c0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 shutdown_connections 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537be62d70 0x56537be65160 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be52e80 0x56537be50830 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/4050701992 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be465d0 0x56537be4fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbd0c00640 0 client.16611 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.670+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 >> 172.21.15.138:0/4050701992 conn(0x56537be443b0 msgr2=0x56537be49fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.671+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 shutdown_connections 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.671+0000 7fbbf0aac740 1 -- 172.21.15.138:0/4050701992 wait complete. 2024-08-20T01:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.671+0000 7fbbf0aac740 20 client.16611 trim_cache size 0 max 16384 2024-08-20T01:17:40.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 Processor -- start 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 -- start start 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b4530 con 0x56537be465d0 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b1dd0 con 0x56537be46ef0 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81afc40 con 0x56537be52e80 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47234/0 (socket says 172.21.15.138:47234) 2024-08-20T01:17:40.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.686+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1056531141 learned_addr learned my addr 172.21.15.138:0/1056531141 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1056531141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 msgr2=0x56537be513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1056531141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 msgr2=0x56537be6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be6afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1056531141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56537be654d0 con 0x56537be465d0 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:40.687 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x56537be65a40 tx=0x7fbbd8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e4088be64c9f9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbd3400640 1 -- 172.21.15.138:0/1056531141 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd801c670 con 0x56537be465d0 2024-08-20T01:17:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbd3400640 1 -- 172.21.15.138:0/1056531141 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd801ccc0 con 0x56537be465d0 2024-08-20T01:17:40.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.687+0000 7fbbd3400640 1 -- 172.21.15.138:0/1056531141 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8029b20 con 0x56537be465d0 2024-08-20T01:17:40.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1056531141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 msgr2=0x56537be6aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x56537be65a40 tx=0x7fbbd8018000 comp rx=0 tx=0).stop 2024-08-20T01:17:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1056531141 shutdown_connections 2024-08-20T01:17:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be46ef0 0x56537be6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1056531141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be465d0 0x56537be6aaa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1056531141 >> 172.21.15.138:0/1056531141 conn(0x56537be443b0 msgr2=0x56537be4b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1056531141 shutdown_connections 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1056531141 wait complete. 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.688+0000 7fbbf0aac740 1 -- start start 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 Processor -- start 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be4cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 0x56537be4d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b0290 con 0x56537be46ef0 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b1700 con 0x56537be4d240 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe81b0960 con 0x56537be52e80 2024-08-20T01:17:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbf0aac740 10 client.0 _ll_register_callbacks cb 0x7fbbe000de60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbe6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47242/0 (socket says 172.21.15.138:47242) 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1488593652 learned_addr learned my addr 172.21.15.138:0/1488593652 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 0x56537be4d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.689+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be4cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1488593652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 msgr2=0x56537be4cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be4cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 msgr2=0x56537be4d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 0x56537be4d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 -- 172.21.15.138:0/1488593652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56537bb87620 con 0x56537be46ef0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe7400640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 0x56537be4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbe6a00640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800f8d0 tx=0x7fbbd800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c595c592c1686f02 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8036030 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fbbd802e050 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.690+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd801ccc0 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbbd8043050 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 10 client.16617 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56537becde50 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd801ccc0 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fbbd801c700 con 0x56537be46ef0 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 10 client.16617 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 20 client.16617 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537bece4f0 0x56537bed0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 4 client.16617 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:17:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.691+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56537bdb9c00 con 0x56537bece4f0 2024-08-20T01:17:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.692+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537bece4f0 0x56537bed0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.692+0000 7fbbe6000640 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537bece4f0 0x56537bed0910 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b464d8abc0368f server_cookie=1acd9e40776b1d4a in_seq=0 out_seq=0 2024-08-20T01:17:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.692+0000 7fbbd0c00640 10 client.16617 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.706+0000 7fbbd0c00640 1 -- 172.21.15.138:0/1488593652 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56537bdb9c00 con 0x56537bece4f0 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbd0c00640 10 client.16617 handle_command_reply: tid=0 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 client.16617 shutdown 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 msgr2=0x56537be5b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800f8d0 tx=0x7fbbd800f610 comp rx=0 tx=0).stop 2024-08-20T01:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 shutdown_connections 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56537bece4f0 0x56537bed0910 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56537be4d240 0x56537be4d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56537be52e80 0x56537be4cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbd0c00640 0 client.16617 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 --2- 172.21.15.138:0/1488593652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56537be46ef0 0x56537be5b5b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 >> 172.21.15.138:0/1488593652 conn(0x56537be443b0 msgr2=0x56537be42cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 shutdown_connections 2024-08-20T01:17:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 1 -- 172.21.15.138:0/1488593652 wait complete. 2024-08-20T01:17:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.707+0000 7fbbf0aac740 20 client.16617 trim_cache size 0 max 16384 2024-08-20T01:17:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:17:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:17:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:17:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:17:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 msgr2=0x7fbbcc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 0x7fbbcc0b6280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56537be49750 tx=0x7fbbd8009500 comp rx=0 tx=0).stop 2024-08-20T01:17:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 msgr2=0x7fbbe81a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000d750 tx=0x7fbbe000dc10 comp rx=0 tx=0).stop 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 shutdown_connections 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbbcc0b3dc0 0x7fbbcc0b6280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbe810af90 0x7fbbe81a0a30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbe810a610 0x7fbbe819b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 --2- 172.21.15.138:0/1249422080 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbbe8104c10 0x7fbbe819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 >> 172.21.15.138:0/1249422080 conn(0x7fbbe8100630 msgr2=0x7fbbe8102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 shutdown_connections 2024-08-20T01:17:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:40.708+0000 7fbbedc00640 1 -- 172.21.15.138:0/1249422080 wait complete. 2024-08-20T01:17:41.093 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:40 smithi138 ceph-mon[18076]: pgmap v656: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 133 KiB/s wr, 20 op/s 2024-08-20T01:17:41.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.57 MB/sec execute 353 sec latency 19798.005 ms 2024-08-20T01:17:41.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:40 smithi167 ceph-mon[21965]: pgmap v656: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 133 KiB/s wr, 20 op/s 2024-08-20T01:17:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:40 smithi142 ceph-mon[21799]: pgmap v656: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 133 KiB/s wr, 20 op/s 2024-08-20T01:17:42.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.56 MB/sec execute 354 sec latency 20798.105 ms 2024-08-20T01:17:43.019 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:42 smithi138 ceph-mon[18076]: pgmap v657: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 114 KiB/s wr, 11 op/s 2024-08-20T01:17:43.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.55 MB/sec execute 355 sec latency 21798.173 ms 2024-08-20T01:17:43.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:42 smithi167 ceph-mon[21965]: pgmap v657: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 114 KiB/s wr, 11 op/s 2024-08-20T01:17:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:42 smithi142 ceph-mon[21799]: pgmap v657: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 114 KiB/s wr, 11 op/s 2024-08-20T01:17:44.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.54 MB/sec execute 356 sec latency 22798.285 ms 2024-08-20T01:17:45.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.53 MB/sec execute 357 sec latency 23798.424 ms 2024-08-20T01:17:45.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:44 smithi167 ceph-mon[21965]: pgmap v658: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 247 KiB/s wr, 16 op/s 2024-08-20T01:17:45.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:44 smithi142 ceph-mon[21799]: pgmap v658: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 247 KiB/s wr, 16 op/s 2024-08-20T01:17:45.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:44 smithi138 ceph-mon[18076]: pgmap v658: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 247 KiB/s wr, 16 op/s 2024-08-20T01:17:45.581 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:17:45.963 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:46.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.52 MB/sec execute 358 sec latency 24798.514 ms 2024-08-20T01:17:46.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.576+0000 7f0899e00640 1 -- 172.21.15.138:0/2929042571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f089410b030 msgr2=0x7f0894111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:46.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.576+0000 7f0899e00640 1 --2- 172.21.15.138:0/2929042571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f089410b030 0x7f0894111830 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f088c009ca0 tx=0x7f088c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.576+0000 7f0899e00640 1 -- 172.21.15.138:0/2929042571 shutdown_connections 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.576+0000 7f0899e00640 1 --2- 172.21.15.138:0/2929042571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f089410b030 0x7f0894111830 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 --2- 172.21.15.138:0/2929042571 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 0x7f089410aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 --2- 172.21.15.138:0/2929042571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0894104ca0 0x7f0894105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 -- 172.21.15.138:0/2929042571 >> 172.21.15.138:0/2929042571 conn(0x7f0894100730 msgr2=0x7f0894102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 -- 172.21.15.138:0/2929042571 shutdown_connections 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 -- 172.21.15.138:0/2929042571 wait complete. 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.577+0000 7f0899e00640 1 Processor -- start 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 -- start start 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 0x7f089419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:46.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f089410b030 0x7f08941a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0894114d00 con 0x7f0894104ca0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0894114b80 con 0x7f089410b030 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0899e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0894114e80 con 0x7f089410a6b0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0892a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0892a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:47254/0 (socket says 172.21.15.138:47254) 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0892a00640 1 -- 172.21.15.138:0/2302559552 learned_addr learned my addr 172.21.15.138:0/2302559552 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f0893400640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f089410b030 0x7f08941a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.578+0000 7f088be00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 0x7f089419b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 -- 172.21.15.138:0/2302559552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 msgr2=0x7f089419b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 0x7f089419b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 -- 172.21.15.138:0/2302559552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f089410b030 msgr2=0x7f08941a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f089410b030 0x7f08941a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 -- 172.21.15.138:0/2302559552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08941a1220 con 0x7f0894104ca0 2024-08-20T01:17:46.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0892a00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f088400d950 tx=0x7f088400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0884017030 con 0x7f0894104ca0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0884004020 con 0x7f0894104ca0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08941a1510 con 0x7f0894104ca0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08840047a0 con 0x7f0894104ca0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.579+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08941a7bd0 con 0x7f0894104ca0 2024-08-20T01:17:46.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.581+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0894105080 con 0x7f0894104ca0 2024-08-20T01:17:46.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.583+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0884009a30 con 0x7f0894104ca0 2024-08-20T01:17:46.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.584+0000 7f0898a00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 0x7f08780b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:46.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.584+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7f08840e44a0 con 0x7f0894104ca0 2024-08-20T01:17:46.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.585+0000 7f088be00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 0x7f08780b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:46.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.585+0000 7f088be00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 0x7f08780b6200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0894103f90 tx=0x7f087c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:46.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.586+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f08840ab3b0 con 0x7f0894104ca0 2024-08-20T01:17:46.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.810+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f089402e340 con 0x7f0894104ca0 2024-08-20T01:17:46.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.811+0000 7f0898a00640 1 -- 172.21.15.138:0/2302559552 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11933 (secure 0 0 0) 0x7f08840cd020 con 0x7f0894104ca0 2024-08-20T01:17:46.809 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:17:46.810 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":40,"btime":"2024-08-20T01:17:24:278890+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:17:24.228659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:17:46.810 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 40 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 msgr2=0x7f08780b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 0x7f08780b6200 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0894103f90 tx=0x7f087c005f90 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 msgr2=0x7f089419b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f088400d950 tx=0x7f088400de10 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 shutdown_connections 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f08780b3d40 0x7f08780b6200 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f089410b030 0x7f08941a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f089410a6b0 0x7f089419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 --2- 172.21.15.138:0/2302559552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0894104ca0 0x7f089419b170 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.813+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 >> 172.21.15.138:0/2302559552 conn(0x7f0894100730 msgr2=0x7f0894102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:46.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.814+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 shutdown_connections 2024-08-20T01:17:46.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:46.814+0000 7f0899e00640 1 -- 172.21.15.138:0/2302559552 wait complete. 2024-08-20T01:17:47.179 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:46 smithi138 ceph-mon[18076]: pgmap v659: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 342 KiB/s wr, 21 op/s 2024-08-20T01:17:47.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.51 MB/sec execute 359 sec latency 25798.572 ms 2024-08-20T01:17:47.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:46 smithi167 ceph-mon[21965]: pgmap v659: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 342 KiB/s wr, 21 op/s 2024-08-20T01:17:47.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:46 smithi142 ceph-mon[21799]: pgmap v659: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 342 KiB/s wr, 21 op/s 2024-08-20T01:17:47.723 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id a94abfe1 --release --await' 2024-08-20T01:17:47.723 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id a94abfe1 --release --await 2024-08-20T01:17:48.093 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:17:48.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370552 3.50 MB/sec execute 360 sec latency 26798.612 ms 2024-08-20T01:17:48.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2302559552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:48.232 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2302559552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:48.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2302559552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:17:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 -- 172.21.15.138:0/1384045563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa410aea0 msgr2=0x7fbaa41116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 --2- 172.21.15.138:0/1384045563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa410aea0 0x7fbaa41116a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fba9c009ca0 tx=0x7fba9c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.700 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 -- 172.21.15.138:0/1384045563 shutdown_connections 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 --2- 172.21.15.138:0/1384045563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa410aea0 0x7fbaa41116a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 --2- 172.21.15.138:0/1384045563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 0x7fbaa410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 --2- 172.21.15.138:0/1384045563 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa4104b10 0x7fbaa4104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 -- 172.21.15.138:0/1384045563 >> 172.21.15.138:0/1384045563 conn(0x7fbaa4100540 msgr2=0x7fbaa4102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 -- 172.21.15.138:0/1384045563 shutdown_connections 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.701+0000 7fbaaae00640 1 -- 172.21.15.138:0/1384045563 wait complete. 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.702+0000 7fbaaae00640 1 Processor -- start 2024-08-20T01:17:48.701 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.702+0000 7fbaaae00640 1 -- start start 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.702+0000 7fbaaae00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaae00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 0x7fbaa419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaae00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa410aea0 0x7fbaa41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaae00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4114b60 con 0x7fbaa4104b10 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaae00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41149e0 con 0x7fbaa410a520 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaae00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa4114ce0 con 0x7fbaa410aea0 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 0x7fbaa419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40982/0 (socket says 172.21.15.138:40982) 2024-08-20T01:17:48.703 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 -- 172.21.15.138:0/111691731 learned_addr learned my addr 172.21.15.138:0/111691731 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaaa400640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa410aea0 0x7fbaa41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 -- 172.21.15.138:0/111691731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa410aea0 msgr2=0x7fbaa41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa410aea0 0x7fbaa41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 -- 172.21.15.138:0/111691731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 msgr2=0x7fbaa419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 0x7fbaa419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.703+0000 7fbaa9a00640 1 -- 172.21.15.138:0/111691731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaa41a0f80 con 0x7fbaa4104b10 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fba9800e9f0 tx=0x7fba9800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaa2000640 1 -- 172.21.15.138:0/111691731 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9800cd80 con 0x7fbaa4104b10 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaa2000640 1 -- 172.21.15.138:0/111691731 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba980108e0 con 0x7fbaa4104b10 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaa41a1270 con 0x7fbaa4104b10 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaa2000640 1 -- 172.21.15.138:0/111691731 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba98018980 con 0x7fbaa4104b10 2024-08-20T01:17:48.704 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.704+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaa41a7b00 con 0x7fbaa4104b10 2024-08-20T01:17:48.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.707+0000 7fbaaee79740 1 Processor -- start 2024-08-20T01:17:48.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.707+0000 7fbaaee79740 1 -- start start 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.707+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.707+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 0x557c9b1ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 0x557c9b1d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaa2000640 1 -- 172.21.15.138:0/111691731 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fba98010430 con 0x7fbaa4104b10 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaee79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c9b163670 con 0x557c9b1c1e80 2024-08-20T01:17:48.706 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaee79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c9b137d70 con 0x557c9b1ce730 2024-08-20T01:17:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaee79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c9b127270 con 0x557c9b1c27a0 2024-08-20T01:17:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40984/0 (socket says 172.21.15.138:40984) 2024-08-20T01:17:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 -- 172.21.15.138:0/915648774 learned_addr learned my addr 172.21.15.138:0/915648774 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:48.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 0x557c9b1ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaa9000640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 0x557c9b1d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 -- 172.21.15.138:0/915648774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 msgr2=0x557c9b1ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 0x557c9b1ce1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 -- 172.21.15.138:0/915648774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 msgr2=0x557c9b1d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 0x557c9b1d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaaa400640 1 -- 172.21.15.138:0/915648774 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c9b0c3870 con 0x557c9b1c1e80 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 0x557c9b1ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.708+0000 7fbaa9000640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 0x557c9b1d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:17:48.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.709+0000 7fbaaa400640 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005fa0 tx=0x7fba9c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca0e9879e047855 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.709+0000 7fba96a00640 1 -- 172.21.15.138:0/915648774 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c031cf0 con 0x557c9b1c1e80 2024-08-20T01:17:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.709+0000 7fba96a00640 1 -- 172.21.15.138:0/915648774 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba9c036030 con 0x557c9b1c1e80 2024-08-20T01:17:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.709+0000 7fba96a00640 1 -- 172.21.15.138:0/915648774 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c0042a0 con 0x557c9b1c1e80 2024-08-20T01:17:48.709 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaa2000640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 0x7fba8c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 -- 172.21.15.138:0/915648774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 msgr2=0x557c9b1c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fba9c005fa0 tx=0x7fba9c009800 comp rx=0 tx=0).stop 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaa2000640 1 -- 172.21.15.138:0/111691731 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fba980e4690 con 0x7fbaa4104b10 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 -- 172.21.15.138:0/915648774 shutdown_connections 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1ce730 0x557c9b1d0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c27a0 0x557c9b1ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 --2- 172.21.15.138:0/915648774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c1e80 0x557c9b1c2260 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 -- 172.21.15.138:0/915648774 >> 172.21.15.138:0/915648774 conn(0x557c9b1bfcb0 msgr2=0x557c9b1c0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 0x7fba8c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.711 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 -- 172.21.15.138:0/915648774 shutdown_connections 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.710+0000 7fbaaee79740 1 -- 172.21.15.138:0/915648774 wait complete. 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaa9a00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 0x7fba8c0b6320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557c9b1c0b90 tx=0x7fba9800cf20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 -- start start 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 Processor -- start 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 0x557c9b1cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1ce730 0x557c9b1cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557c9b0adff0 con 0x557c9b1ce730 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x557c9b12b2b0 con 0x557c9b1c27a0 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.711+0000 7fbaaee79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557c9af89bb0 con 0x557c9b1c1e80 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaaee79740 10 client.0 _ll_register_callbacks cb 0x557c9ae32d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaaa400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 0x557c9b1cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48256/0 (socket says 172.21.15.138:48256) 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 -- 172.21.15.138:0/2973291023 learned_addr learned my addr 172.21.15.138:0/2973291023 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1ce730 0x557c9b1cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 -- 172.21.15.138:0/2973291023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 msgr2=0x557c9b1cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 0x557c9b1cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 -- 172.21.15.138:0/2973291023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1ce730 msgr2=0x557c9b1cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1ce730 0x557c9b1cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 -- 172.21.15.138:0/2973291023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c9ae86b00 con 0x557c9b1c27a0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 0x557c9b1cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fbaa9000640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba9000bf80 tx=0x7fba90011870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a47ece09f01962d0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.712+0000 7fba8b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba90012920 con 0x557c9b1c27a0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c036030 con 0x557c9b1c27a0 2024-08-20T01:17:48.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fba90012ac0 con 0x557c9b1c27a0 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fba90026050 con 0x557c9b1c27a0 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fbaaee79740 10 client.26687 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557c9aefd960 con 0x557c9b1c27a0 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.713+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba9c036030 con 0x557c9b1c27a0 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fba9000f3d0 con 0x557c9b1c27a0 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 10 client.26687 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:48.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 10 client.26687 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 20 client.26687 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b1e63f0 0x557c9b1e8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 4 client.26687 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c9b139cc0 con 0x557c9b1e63f0 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.714+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b1e63f0 0x557c9b1e8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.715+0000 7fba8b400640 10 client.26687 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.715+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b1e63f0 0x557c9b1e8810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520c6b31127269ec server_cookie=f3567dec9974379a in_seq=0 out_seq=0 2024-08-20T01:17:48.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.715+0000 7fba8b400640 1 -- 172.21.15.138:0/2973291023 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c9b139cc0 con 0x557c9b1e63f0 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.715+0000 7fba8b400640 10 client.26687 handle_command_reply: tid=0 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.715+0000 7fbaaee79740 1 client.26687 shutdown 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 msgr2=0x557c9b1cb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fba9000bf80 tx=0x7fba90011870 comp rx=0 tx=0).stop 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 shutdown_connections 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b1e63f0 0x557c9b1e8810 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1ce730 0x557c9b1cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c27a0 0x557c9b1cb360 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2973291023 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1c1e80 0x557c9b1cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fba8b400640 0 client.26687 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:48.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 >> 172.21.15.138:0/2973291023 conn(0x557c9b1bfcb0 msgr2=0x557c9b1c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 shutdown_connections 2024-08-20T01:17:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 1 -- 172.21.15.138:0/2973291023 wait complete. 2024-08-20T01:17:48.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.716+0000 7fbaaee79740 20 client.26687 trim_cache size 0 max 16384 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 Processor -- start 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 -- start start 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1ee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41b0040 con 0x557c9b1c27a0 2024-08-20T01:17:48.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ad8e0 con 0x557c9b1c1e80 2024-08-20T01:17:48.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaee79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ab790 con 0x557c9b1ce730 2024-08-20T01:17:48.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaa400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaa400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48266/0 (socket says 172.21.15.138:48266) 2024-08-20T01:17:48.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaa400640 1 -- 172.21.15.138:0/2318069275 learned_addr learned my addr 172.21.15.138:0/2318069275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:48.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaa9000640 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.734+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1ee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 -- 172.21.15.138:0/2318069275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 msgr2=0x557c9b1ee600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1ee600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 -- 172.21.15.138:0/2318069275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 msgr2=0x557c9b1ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 -- 172.21.15.138:0/2318069275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c9b1e4fa0 con 0x557c9b1c1e80 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fbaaa400640 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02f550 tx=0x7fba9c0093a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16264a1cfefe9bc1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fba96000640 1 -- 172.21.15.138:0/2318069275 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c062070 con 0x557c9b1c1e80 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fba96000640 1 -- 172.21.15.138:0/2318069275 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x557c9b1c1e80 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.735+0000 7fba96000640 1 -- 172.21.15.138:0/2318069275 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9c04a040 con 0x557c9b1c1e80 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- 172.21.15.138:0/2318069275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 msgr2=0x557c9b1e2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fba9c02f550 tx=0x7fba9c0093a0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- 172.21.15.138:0/2318069275 shutdown_connections 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1ee600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 --2- 172.21.15.138:0/2318069275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1e2a60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- 172.21.15.138:0/2318069275 >> 172.21.15.138:0/2318069275 conn(0x557c9b1bfcb0 msgr2=0x557c9b1bec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- 172.21.15.138:0/2318069275 shutdown_connections 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- 172.21.15.138:0/2318069275 wait complete. 2024-08-20T01:17:48.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.736+0000 7fbaaee79740 1 -- start start 2024-08-20T01:17:48.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 Processor -- start 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1daaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1db520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41abde0 con 0x557c9b1c27a0 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ad210 con 0x557c9b1c1e80 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbaa41ac470 con 0x557c9b1ce730 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaee79740 10 client.0 _ll_register_callbacks cb 0x7fba9000bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaa9000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41000/0 (socket says 172.21.15.138:41000) 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaa9000640 1 -- 172.21.15.138:0/672245400 learned_addr learned my addr 172.21.15.138:0/672245400 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaa400640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1daaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.737+0000 7fbaaa400640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1db520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 -- 172.21.15.138:0/672245400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 msgr2=0x557c9b1db520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1db520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 -- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 msgr2=0x557c9b1daaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1daaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 -- 172.21.15.138:0/672245400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c9b1c0730 con 0x557c9b1c27a0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaaa400640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaa9000640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fba90005d00 tx=0x7fba9000f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900c378f63f8dd48 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fba8b400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba9000fc20 con 0x557c9b1c27a0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fba9000fdc0 con 0x557c9b1c27a0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba9c036070 con 0x557c9b1c27a0 2024-08-20T01:17:48.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fba9003a050 con 0x557c9b1c27a0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.738+0000 7fbaaee79740 10 client.16641 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x557c9b251300 con 0x557c9b1c27a0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba9c036070 con 0x557c9b1c27a0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fba90033060 con 0x557c9b1c27a0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 10 client.16641 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 10 client.16641 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 20 client.16641 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b251ad0 0x557c9b253ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 4 client.16641 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a94abfe1", "release": true, "await": true}] 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a94abfe1", "release": true, "await": true}) -- 0x557c9b135760 con 0x557c9b251ad0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.739+0000 7fbaaa400640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b251ad0 0x557c9b253ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.740+0000 7fbaaa400640 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b251ad0 0x557c9b253ef0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49195e291cbf790 server_cookie=aea69c61be33215 in_seq=0 out_seq=0 2024-08-20T01:17:48.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.740+0000 7fba8b400640 10 client.16641 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:48.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.744+0000 7fba8b400640 1 -- 172.21.15.138:0/672245400 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557c9b135760 con 0x557c9b251ad0 2024-08-20T01:17:48.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.744+0000 7fba8b400640 10 client.16641 handle_command_reply: tid=0 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.744+0000 7fbaaee79740 1 client.16641 shutdown 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 msgr2=0x557c9b1dafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fba90005d00 tx=0x7fba9000f600 comp rx=0 tx=0).stop 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 shutdown_connections 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557c9b251ad0 0x557c9b253ef0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x557c9b1ce730 0x557c9b1db520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x557c9b1c27a0 0x557c9b1dafe0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 --2- 172.21.15.138:0/672245400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557c9b1c1e80 0x557c9b1daaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fba8b400640 0 client.16641 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 >> 172.21.15.138:0/672245400 conn(0x557c9b1bfcb0 msgr2=0x557c9b1cc990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 shutdown_connections 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 1 -- 172.21.15.138:0/672245400 wait complete. 2024-08-20T01:17:48.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.745+0000 7fbaaee79740 20 client.16641 trim_cache size 0 max 16384 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 40, 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 40, 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:17:48.745 INFO:teuthology.orchestra.run.smithi138.stdout: "a94abfe1": { 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 40, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 147.8, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:17:48.746 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:17:48.747 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 msgr2=0x7fba8c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 0x7fba8c0b6320 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x557c9b1c0b90 tx=0x7fba9800cf20 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 msgr2=0x7fbaa419aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fba9800e9f0 tx=0x7fba9800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 shutdown_connections 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fba8c0b3e60 0x7fba8c0b6320 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbaa410aea0 0x7fbaa41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaa410a520 0x7fbaa419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 --2- 172.21.15.138:0/111691731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbaa4104b10 0x7fbaa419aed0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 >> 172.21.15.138:0/111691731 conn(0x7fbaa4100540 msgr2=0x7fbaa4103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:48.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 shutdown_connections 2024-08-20T01:17:48.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:17:48.746+0000 7fbaaae00640 1 -- 172.21.15.138:0/111691731 wait complete. 2024-08-20T01:17:49.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:48 smithi138 ceph-mon[18076]: pgmap v660: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-08-20T01:17:49.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 370575 3.49 MB/sec execute 361 sec latency 27357.581 ms 2024-08-20T01:17:49.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:48 smithi167 ceph-mon[21965]: pgmap v660: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-08-20T01:17:49.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:48 smithi142 ceph-mon[21799]: pgmap v660: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 285 KiB/s wr, 20 op/s 2024-08-20T01:17:49.716 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a94abfe1', total seconds elapsed: 32.0 2024-08-20T01:17:50.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 372669 3.52 MB/sec execute 362 sec latency 518.064 ms 2024-08-20T01:17:51.180 INFO:tasks.workunit.client.0.smithi138.stdout: 1 373924 3.53 MB/sec execute 363 sec latency 23.785 ms 2024-08-20T01:17:51.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:50 smithi167 ceph-mon[21965]: pgmap v661: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 337 KiB/s wr, 30 op/s 2024-08-20T01:17:51.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:50 smithi142 ceph-mon[21799]: pgmap v661: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 337 KiB/s wr, 30 op/s 2024-08-20T01:17:51.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:50 smithi138 ceph-mon[18076]: pgmap v661: 129 pgs: 129 active+clean; 258 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 337 KiB/s wr, 30 op/s 2024-08-20T01:17:52.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 374123 3.52 MB/sec execute 364 sec latency 825.834 ms 2024-08-20T01:17:52.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:51 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:17:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:51 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:52.235 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:51 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:17:52.235 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:52.236 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:51 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:51 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:17:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:51 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 374123 3.51 MB/sec execute 365 sec latency 1825.907 ms 2024-08-20T01:17:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:53 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:53 smithi167 ceph-mon[21965]: pgmap v662: 129 pgs: 129 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 526 KiB/s wr, 41 op/s 2024-08-20T01:17:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:53 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:53 smithi142 ceph-mon[21799]: pgmap v662: 129 pgs: 129 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 526 KiB/s wr, 41 op/s 2024-08-20T01:17:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:53 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:53.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:53 smithi138 ceph-mon[18076]: pgmap v662: 129 pgs: 129 active+clean; 261 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 526 KiB/s wr, 41 op/s 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 -- 172.21.15.138:0/4287310456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108b80 msgr2=0x7fc0d410f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/4287310456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108b80 0x7fc0d410f380 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 -- 172.21.15.138:0/4287310456 shutdown_connections 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/4287310456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108b80 0x7fc0d410f380 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/4287310456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4108200 0x7fc0d4108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/4287310456 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4102800 0x7fc0d4102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 -- 172.21.15.138:0/4287310456 >> 172.21.15.138:0/4287310456 conn(0x7fc0d40fe280 msgr2=0x7fc0d41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 -- 172.21.15.138:0/4287310456 shutdown_connections 2024-08-20T01:17:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.828+0000 7fc0d3e00640 1 -- 172.21.15.138:0/4287310456 wait complete. 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 Processor -- start 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 -- start start 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4102800 0x7fc0d419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4108b80 0x7fc0d41a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4112890 con 0x7fc0d4108200 2024-08-20T01:17:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4112710 con 0x7fc0d4102800 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4112a10 con 0x7fc0d4108b80 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41016/0 (socket says 172.21.15.138:41016) 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.829+0000 7fc0d2000640 1 -- 172.21.15.138:0/310002764 learned_addr learned my addr 172.21.15.138:0/310002764 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d3400640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4108b80 0x7fc0d41a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4102800 0x7fc0d419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 -- 172.21.15.138:0/310002764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4108b80 msgr2=0x7fc0d41a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4108b80 0x7fc0d41a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 -- 172.21.15.138:0/310002764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4102800 msgr2=0x7fc0d419af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4102800 0x7fc0d419af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 -- 172.21.15.138:0/310002764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d41a0fc0 con 0x7fc0d4108200 2024-08-20T01:17:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d2000640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ea60 tx=0x7fc0c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0cb400640 1 -- 172.21.15.138:0/310002764 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cde0 con 0x7fc0d4108200 2024-08-20T01:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0cb400640 1 -- 172.21.15.138:0/310002764 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c00035f0 con 0x7fc0d4108200 2024-08-20T01:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d41a12b0 con 0x7fc0d4108200 2024-08-20T01:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.830+0000 7fc0cb400640 1 -- 172.21.15.138:0/310002764 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0010690 con 0x7fc0d4108200 2024-08-20T01:17:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.831+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a7970 con 0x7fc0d4108200 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.833+0000 7fc0dc42e740 1 Processor -- start 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 -- start start 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0cb400640 1 -- 172.21.15.138:0/310002764 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc0c00037a0 con 0x7fc0d4108200 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1cf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c63c136210 con 0x55c63c1c10f0 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c63c125710 con 0x55c63c1c07d0 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c63c129750 con 0x55c63c1cd080 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48300/0 (socket says 172.21.15.138:48300) 2024-08-20T01:17:53.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0d3400640 1 -- 172.21.15.138:0/2084113644 learned_addr learned my addr 172.21.15.138:0/2084113644 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.834+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1cf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 -- 172.21.15.138:0/2084113644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 msgr2=0x55c63c1cf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1cf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 -- 172.21.15.138:0/2084113644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 msgr2=0x55c63c1ccb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ccb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 -- 172.21.15.138:0/2084113644 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c63c0f1de0 con 0x55c63c1c07d0 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:17:53.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc0093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3066cfa89a37b54 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0bf400640 1 -- 172.21.15.138:0/2084113644 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc031cf0 con 0x55c63c1c07d0 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0bf400640 1 -- 172.21.15.138:0/2084113644 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc036030 con 0x55c63c1c07d0 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0bf400640 1 -- 172.21.15.138:0/2084113644 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc004240 con 0x55c63c1c07d0 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.835+0000 7fc0cb400640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 0x7fc0b80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0cb400640 1 -- 172.21.15.138:0/310002764 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fc0c00e4670 con 0x7fc0d4108200 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 0x7fc0b80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2084113644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 msgr2=0x55c63c1c0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009ca0 tx=0x7fc0cc0093e0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2084113644 shutdown_connections 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ccb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2084113644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c0bb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2084113644 >> 172.21.15.138:0/2084113644 conn(0x55c63c1be600 msgr2=0x55c63c1bf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2084113644 shutdown_connections 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2084113644 wait complete. 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.836+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 0x7fc0b80b6200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55c63c1c3950 tx=0x7fc0c4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 -- start start 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 Processor -- start 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1caa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c63c05eab0 con 0x55c63c1c10f0 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c63c0cc140 con 0x55c63c1c07d0 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c63c1cfb80 con 0x55c63c1cd080 2024-08-20T01:17:53.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.837+0000 7fc0dc42e740 10 client.0 _ll_register_callbacks cb 0x55c63c0df5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41036/0 (socket says 172.21.15.138:41036) 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 -- 172.21.15.138:0/330768820 learned_addr learned my addr 172.21.15.138:0/330768820 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d3400640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 -- 172.21.15.138:0/330768820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 msgr2=0x55c63c1caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1caa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 -- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 msgr2=0x55c63c1c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 -- 172.21.15.138:0/330768820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63c0cd170 con 0x55c63c1c10f0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0d2000640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00aabe0 tx=0x7fc0c00e1f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7cc74bac0f98f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:53.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c00e3070 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fc0c00e4260 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.838+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c63c0fca30 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0c00ab5e0 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 10 client.16653 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c63be88790 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c63c0fca30 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fc0c0004000 con 0x55c63c1c10f0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 10 client.16653 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 20 client.16653 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c1dccc0 0x55c63c1df0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 4 client.16653 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.839+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c63c138160 con 0x55c63c1dccc0 2024-08-20T01:17:53.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.840+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c1dccc0 0x55c63c1df0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.840+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c1dccc0 0x55c63c1df0b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b09f5bf09a37c40 server_cookie=5b75e1de42a98a3b in_seq=0 out_seq=0 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.840+0000 7fc0b3e00640 10 client.16653 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0b3e00640 1 -- 172.21.15.138:0/330768820 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c63c138160 con 0x55c63c1dccc0 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0b3e00640 10 client.16653 handle_command_reply: tid=0 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 client.16653 shutdown 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 msgr2=0x55c63c1ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00aabe0 tx=0x7fc0c00e1f20 comp rx=0 tx=0).stop 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 shutdown_connections 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c1dccc0 0x55c63c1df0b0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c1caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c10f0 0x55c63c1ca4d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0b3e00640 0 client.16653 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/330768820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c07d0 0x55c63c1c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 >> 172.21.15.138:0/330768820 conn(0x55c63c1be600 msgr2=0x55c63c1c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 shutdown_connections 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 1 -- 172.21.15.138:0/330768820 wait complete. 2024-08-20T01:17:53.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.841+0000 7fc0dc42e740 20 client.16653 trim_cache size 0 max 16384 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.856+0000 7fc0dc42e740 1 Processor -- start 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.856+0000 7fc0dc42e740 1 -- start start 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1c10f0 0x55c63c1cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41b4390 con 0x55c63c1c07d0 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41b1c30 con 0x55c63c1cd080 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41afaa0 con 0x55c63c1c10f0 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48312/0 (socket says 172.21.15.138:48312) 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1bdd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41042/0 (socket says 172.21.15.138:41042) 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 -- 172.21.15.138:0/2785201055 learned_addr learned my addr 172.21.15.138:0/2785201055 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1c10f0 0x55c63c1cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.857 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 -- 172.21.15.138:0/2785201055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1c10f0 msgr2=0x55c63c1cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1c10f0 0x55c63c1cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 -- 172.21.15.138:0/2785201055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 msgr2=0x55c63c1bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.857+0000 7fc0d3400640 1 -- 172.21.15.138:0/2785201055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c63bf01620 con 0x55c63c1cd080 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.858+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc006010 tx=0x7fc0cc003da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7bc14871ae271cd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.858+0000 7fc0bea00640 1 -- 172.21.15.138:0/2785201055 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc055070 con 0x55c63c1cd080 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.858+0000 7fc0bea00640 1 -- 172.21.15.138:0/2785201055 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc036040 con 0x55c63c1cd080 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.858+0000 7fc0bea00640 1 -- 172.21.15.138:0/2785201055 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc02f870 con 0x55c63c1cd080 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2785201055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 msgr2=0x55c63c1e49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc006010 tx=0x7fc0cc003da0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2785201055 shutdown_connections 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1cd080 0x55c63c1e49c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1c10f0 0x55c63c1cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2785201055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2785201055 >> 172.21.15.138:0/2785201055 conn(0x55c63c1be600 msgr2=0x55c63c1cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2785201055 shutdown_connections 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2785201055 wait complete. 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 -- start start 2024-08-20T01:17:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.859+0000 7fc0dc42e740 1 Processor -- start 2024-08-20T01:17:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 0x55c63c1d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c23fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41b00f0 con 0x55c63c1c07d0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41b1560 con 0x55c63c1c10f0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41b07c0 con 0x55c63c1cd080 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0dc42e740 10 client.0 _ll_register_callbacks cb 0x7fc0c000e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:41058/0 (socket says 172.21.15.138:41058) 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 -- 172.21.15.138:0/2688615609 learned_addr learned my addr 172.21.15.138:0/2688615609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 0x55c63c1d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d3400640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c23fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 -- 172.21.15.138:0/2688615609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 msgr2=0x55c63c23fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c23fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 -- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 msgr2=0x55c63c1d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.860+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 0x55c63c1d0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0d2a00640 1 -- 172.21.15.138:0/2688615609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c63c240170 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 0x55c63c1d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0d2a00640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c63c1df930 tx=0x7fc0c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81c65aa203f215d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0b3e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c401db90 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9453+0+0 (secure 0 0 0) 0x7fc0c4018030 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0cc036040 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fc0c401dd30 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0dc42e740 10 client.16659 fetch_fsmap learned FSMap version 40 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.861+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c63c247660 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0cc036040 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fc0c402a5b0 con 0x55c63c1c07d0 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 10 client.16659 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 10 client.16659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:17:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 20 client.16659 populate_metadata read hostname 'smithi138' 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c247e40 0x55c63c24a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 4 client.16659 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c63c133c00 con 0x55c63c247e40 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.862+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c247e40 0x55c63c24a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.863+0000 7fc0b3e00640 10 client.16659 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.863+0000 7fc0d2000640 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c247e40 0x55c63c24a260 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db5a2ee08e24984 server_cookie=506d5bff35b504f in_seq=0 out_seq=0 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0b3e00640 1 -- 172.21.15.138:0/2688615609 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c63c133c00 con 0x55c63c247e40 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0b3e00640 10 client.16659 handle_command_reply: tid=0 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 client.16659 shutdown 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 msgr2=0x55c63c1d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c63c1df930 tx=0x7fc0c4017000 comp rx=0 tx=0).stop 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 shutdown_connections 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c63c247e40 0x55c63c24a260 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.889 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c63c1cd080 0x55c63c23fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c63c1c10f0 0x55c63c1d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0dc42e740 1 --2- 172.21.15.138:0/2688615609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c63c1c07d0 0x55c63c1d0770 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.890+0000 7fc0b3e00640 0 client.16659 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.891+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 >> 172.21.15.138:0/2688615609 conn(0x55c63c1be600 msgr2=0x55c63c1cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.891+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 shutdown_connections 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.891+0000 7fc0dc42e740 1 -- 172.21.15.138:0/2688615609 wait complete. 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.891+0000 7fc0dc42e740 20 client.16659 trim_cache size 0 max 16384 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:17:53.890 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 msgr2=0x7fc0b80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 0x7fc0b80b6200 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x55c63c1c3950 tx=0x7fc0c4005f90 comp rx=0 tx=0).stop 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 msgr2=0x7fc0d419b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ea60 tx=0x7fc0c000ef20 comp rx=0 tx=0).stop 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 shutdown_connections 2024-08-20T01:17:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc0b80b3d40 0x7fc0b80b6200 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc0d4108b80 0x7fc0d41a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0d4108200 0x7fc0d419b460 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 --2- 172.21.15.138:0/310002764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc0d4102800 0x7fc0d419af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 >> 172.21.15.138:0/310002764 conn(0x7fc0d40fe280 msgr2=0x7fc0d40758d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 shutdown_connections 2024-08-20T01:17:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:17:53.892+0000 7fc0d3e00640 1 -- 172.21.15.138:0/310002764 wait complete. 2024-08-20T01:17:54.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 374929 3.51 MB/sec execute 366 sec latency 2528.063 ms 2024-08-20T01:17:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:17:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:17:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:54 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:17:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:17:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:54 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:54.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:17:54.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:17:54.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:54 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:17:55.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 376550 3.53 MB/sec execute 367 sec latency 12.810 ms 2024-08-20T01:17:55.712 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:55 smithi167 ceph-mon[21965]: pgmap v663: 129 pgs: 129 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 674 KiB/s wr, 65 op/s 2024-08-20T01:17:55.740 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:55 smithi142 ceph-mon[21799]: pgmap v663: 129 pgs: 129 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 674 KiB/s wr, 65 op/s 2024-08-20T01:17:55.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:55 smithi138 ceph-mon[18076]: pgmap v663: 129 pgs: 129 active+clean; 262 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 674 KiB/s wr, 65 op/s 2024-08-20T01:17:56.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 377666 3.53 MB/sec execute 368 sec latency 244.359 ms 2024-08-20T01:17:57.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 377666 3.52 MB/sec execute 369 sec latency 1244.457 ms 2024-08-20T01:17:57.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:56 smithi167 ceph-mon[21965]: pgmap v664: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 KiB/s wr, 85 op/s 2024-08-20T01:17:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:56 smithi142 ceph-mon[21799]: pgmap v664: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 KiB/s wr, 85 op/s 2024-08-20T01:17:57.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:56 smithi138 ceph-mon[18076]: pgmap v664: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 872 KiB/s wr, 85 op/s 2024-08-20T01:17:58.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 377666 3.51 MB/sec execute 370 sec latency 2244.582 ms 2024-08-20T01:17:59.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 377666 3.50 MB/sec execute 371 sec latency 3244.691 ms 2024-08-20T01:17:59.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:17:58 smithi167 ceph-mon[21965]: pgmap v665: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 733 KiB/s wr, 78 op/s 2024-08-20T01:17:59.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:17:58 smithi142 ceph-mon[21799]: pgmap v665: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 733 KiB/s wr, 78 op/s 2024-08-20T01:17:59.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:17:58 smithi138 ceph-mon[18076]: pgmap v665: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 733 KiB/s wr, 78 op/s 2024-08-20T01:18:00.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 379922 3.53 MB/sec execute 372 sec latency 3300.845 ms 2024-08-20T01:18:01.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 380591 3.53 MB/sec execute 373 sec latency 62.899 ms 2024-08-20T01:18:01.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:00 smithi167 ceph-mon[21965]: pgmap v666: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 815 KiB/s wr, 92 op/s 2024-08-20T01:18:01.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:00 smithi142 ceph-mon[21799]: pgmap v666: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 815 KiB/s wr, 92 op/s 2024-08-20T01:18:01.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:00 smithi138 ceph-mon[18076]: pgmap v666: 129 pgs: 129 active+clean; 266 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 815 KiB/s wr, 92 op/s 2024-08-20T01:18:02.181 INFO:tasks.workunit.client.0.smithi138.stdout: 1 381255 3.53 MB/sec execute 374 sec latency 159.122 ms 2024-08-20T01:18:03.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 381255 3.52 MB/sec execute 375 sec latency 1159.272 ms 2024-08-20T01:18:03.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:02 smithi167 ceph-mon[21965]: pgmap v667: 129 pgs: 129 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 993 KiB/s wr, 100 op/s 2024-08-20T01:18:03.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:02 smithi142 ceph-mon[21799]: pgmap v667: 129 pgs: 129 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 993 KiB/s wr, 100 op/s 2024-08-20T01:18:03.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:02 smithi138 ceph-mon[18076]: pgmap v667: 129 pgs: 129 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 993 KiB/s wr, 100 op/s 2024-08-20T01:18:04.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 381255 3.51 MB/sec execute 376 sec latency 2159.415 ms 2024-08-20T01:18:05.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 382937 3.52 MB/sec execute 377 sec latency 2212.941 ms 2024-08-20T01:18:05.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:05 smithi138 ceph-mon[18076]: osdmap e96: 12 total, 12 up, 12 in 2024-08-20T01:18:05.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:05 smithi138 ceph-mon[18076]: pgmap v669: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 115 op/s 2024-08-20T01:18:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:05 smithi167 ceph-mon[21965]: osdmap e96: 12 total, 12 up, 12 in 2024-08-20T01:18:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:05 smithi167 ceph-mon[21965]: pgmap v669: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 115 op/s 2024-08-20T01:18:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:05 smithi142 ceph-mon[21799]: osdmap e96: 12 total, 12 up, 12 in 2024-08-20T01:18:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:05 smithi142 ceph-mon[21799]: pgmap v669: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 269 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 932 KiB/s wr, 115 op/s 2024-08-20T01:18:06.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 384212 3.53 MB/sec execute 378 sec latency 34.392 ms 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.007+0000 7fae60200640 1 -- 172.21.15.138:0/449603936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae5810ef70 msgr2=0x7fae581113a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.007+0000 7fae60200640 1 --2- 172.21.15.138:0/449603936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae5810ef70 0x7fae581113a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fae50009ca0 tx=0x7fae5002a8b0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 -- 172.21.15.138:0/449603936 shutdown_connections 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 --2- 172.21.15.138:0/449603936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae5810ef70 0x7fae581113a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 --2- 172.21.15.138:0/449603936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae58075c20 0x7fae5810ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 --2- 172.21.15.138:0/449603936 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae58075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 -- 172.21.15.138:0/449603936 >> 172.21.15.138:0/449603936 conn(0x7fae580fe430 msgr2=0x7fae58100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 -- 172.21.15.138:0/449603936 shutdown_connections 2024-08-20T01:18:07.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 -- 172.21.15.138:0/449603936 wait complete. 2024-08-20T01:18:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.008+0000 7fae60200640 1 Processor -- start 2024-08-20T01:18:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 -- start start 2024-08-20T01:18:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae5819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 0x7fae581a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae58114c00 con 0x7fae58075c20 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae58114a80 con 0x7fae5810ef70 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae60200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae58114d80 con 0x7fae58075270 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae5819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55030/0 (socket says 172.21.15.138:55030) 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae5ce00640 1 -- 172.21.15.138:0/1537070685 learned_addr learned my addr 172.21.15.138:0/1537070685 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.009+0000 7fae5e200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 0x7fae581a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 -- 172.21.15.138:0/1537070685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 msgr2=0x7fae581a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 0x7fae581a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 -- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 msgr2=0x7fae5819b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 -- 172.21.15.138:0/1537070685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae581a1160 con 0x7fae58075270 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5ce00640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5e200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 0x7fae581a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae5d800640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae5819b0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9f0 tx=0x7fae4c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae56000640 1 -- 172.21.15.138:0/1537070685 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c00ccc0 con 0x7fae58075270 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae581a1450 con 0x7fae58075270 2024-08-20T01:18:07.012 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae56000640 1 -- 172.21.15.138:0/1537070685 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c005020 con 0x7fae58075270 2024-08-20T01:18:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae56000640 1 -- 172.21.15.138:0/1537070685 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c004a00 con 0x7fae58075270 2024-08-20T01:18:07.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.010+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae581a7b10 con 0x7fae58075270 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.013+0000 7fae63231740 1 Processor -- start 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.013+0000 7fae63231740 1 -- start start 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.013+0000 7fae56000640 1 -- 172.21.15.138:0/1537070685 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fae4c004ba0 con 0x7fae58075270 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd106d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd07c210 con 0x55e6dd1131e0 2024-08-20T01:18:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd06b710 con 0x55e6dd107250 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae63231740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd06f750 con 0x55e6dd106930 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55044/0 (socket says 172.21.15.138:55044) 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 -- 172.21.15.138:0/76279954 learned_addr learned my addr 172.21.15.138:0/76279954 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5e200640 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd106d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 -- 172.21.15.138:0/76279954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 msgr2=0x55e6dd106d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd106d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 -- 172.21.15.138:0/76279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 msgr2=0x55e6dd112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.014+0000 7fae5ce00640 1 -- 172.21.15.138:0/76279954 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd037de0 con 0x55e6dd1131e0 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae5ce00640 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fae440115f0 tx=0x7fae44011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe585c64bd10717 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae4aa00640 1 -- 172.21.15.138:0/76279954 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4401e070 con 0x55e6dd1131e0 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae4aa00640 1 -- 172.21.15.138:0/76279954 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4400fb70 con 0x55e6dd1131e0 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae4aa00640 1 -- 172.21.15.138:0/76279954 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae44019cb0 con 0x55e6dd1131e0 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae56000640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 0x7fae400b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae56000640 1 -- 172.21.15.138:0/1537070685 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9541+0+0 (secure 0 0 0) 0x7fae4c0e4500 con 0x7fae58075270 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.015+0000 7fae5e200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 0x7fae400b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- 172.21.15.138:0/76279954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 msgr2=0x55e6dd1155d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fae440115f0 tx=0x7fae44011ab0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- 172.21.15.138:0/76279954 shutdown_connections 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd1155d0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 --2- 172.21.15.138:0/76279954 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd106d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- 172.21.15.138:0/76279954 >> 172.21.15.138:0/76279954 conn(0x55e6dd104760 msgr2=0x55e6dd105900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- 172.21.15.138:0/76279954 shutdown_connections 2024-08-20T01:18:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- 172.21.15.138:0/76279954 wait complete. 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae5e200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 0x7fae400b6200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae50009430 tx=0x7fae50004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.016+0000 7fae63231740 1 -- start start 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 Processor -- start 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd1129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd110370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55e6dcfa4ab0 con 0x55e6dd107250 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd012140 con 0x55e6dd1131e0 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55e6dd115ec0 con 0x55e6dd106930 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae63231740 10 client.0 _ll_register_callbacks cb 0x55e6dd0255d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae5ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:55050/0 (socket says 172.21.15.138:55050) 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae5ce00640 1 -- 172.21.15.138:0/1500384905 learned_addr learned my addr 172.21.15.138:0/1500384905 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae5e200640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd110370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.017+0000 7fae5d800640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd1129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 -- 172.21.15.138:0/1500384905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 msgr2=0x55e6dd1129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd1129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 -- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 msgr2=0x55e6dd110370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd110370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 -- 172.21.15.138:0/1500384905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dd013170 con 0x55e6dd107250 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5e200640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5d800640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd1129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae5ce00640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fae44000c00 tx=0x7fae44019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68f4d6aba0d3c72 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae3f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:18:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4401e040 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4400fb70 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9541+0+0 (secure 0 0 0) 0x7fae44016070 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fae4402f050 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae63231740 10 client.16671 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.018+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6dcdce790 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4400fb70 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fae44021070 con 0x55e6dd107250 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 10 client.16671 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 10 client.16671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 20 client.16671 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd122ce0 0x55e6dd1250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 4 client.16671 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:18:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6dd07e160 con 0x55e6dd122ce0 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.019+0000 7fae5d800640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd122ce0 0x55e6dd1250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.020+0000 7fae5d800640 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd122ce0 0x55e6dd1250d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3a78a588a3bfc24 server_cookie=becb3c9c26c66ba3 in_seq=0 out_seq=0 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.020+0000 7fae3f400640 10 client.16671 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.020+0000 7fae3f400640 1 -- 172.21.15.138:0/1500384905 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6dd07e160 con 0x55e6dd122ce0 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.020+0000 7fae3f400640 10 client.16671 handle_command_reply: tid=0 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.020+0000 7fae63231740 1 client.16671 shutdown 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 msgr2=0x55e6dd10fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fae44000c00 tx=0x7fae44019640 comp rx=0 tx=0).stop 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 shutdown_connections 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd122ce0 0x55e6dd1250d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd107250 0x55e6dd10fe30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae3f400640 0 client.16671 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 --2- 172.21.15.138:0/1500384905 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd1129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 >> 172.21.15.138:0/1500384905 conn(0x55e6dd104760 msgr2=0x55e6dd10a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 shutdown_connections 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 1 -- 172.21.15.138:0/1500384905 wait complete. 2024-08-20T01:18:07.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.021+0000 7fae63231740 20 client.16671 trim_cache size 0 max 16384 2024-08-20T01:18:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 Processor -- start 2024-08-20T01:18:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 -- start start 2024-08-20T01:18:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd12a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd12add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b4760 con 0x55e6dd1131e0 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b2000 con 0x55e6dd107250 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae63231740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae581afe70 con 0x55e6dd106930 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.036+0000 7fae5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd12add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:50582/0 (socket says 172.21.15.138:50582) 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 -- 172.21.15.138:0/3077405568 learned_addr learned my addr 172.21.15.138:0/3077405568 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5ce00640 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd12a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 -- 172.21.15.138:0/3077405568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 msgr2=0x55e6dd12a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd12a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 -- 172.21.15.138:0/3077405568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 msgr2=0x55e6dd12add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd12add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 -- 172.21.15.138:0/3077405568 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6dd107080 con 0x55e6dd106930 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5e200640 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd12add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae5d800640 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0e1850 tx=0x7fae4c0ea060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7471bba5de8fd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae4a000640 1 -- 172.21.15.138:0/3077405568 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c01c040 con 0x55e6dd106930 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae4a000640 1 -- 172.21.15.138:0/3077405568 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c0aa7c0 con 0x55e6dd106930 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.037+0000 7fae4a000640 1 -- 172.21.15.138:0/3077405568 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae4c0b3d60 con 0x55e6dd106930 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- 172.21.15.138:0/3077405568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 msgr2=0x55e6dd103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fae4c0e1850 tx=0x7fae4c0ea060 comp rx=0 tx=0).stop 2024-08-20T01:18:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- 172.21.15.138:0/3077405568 shutdown_connections 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd1131e0 0x55e6dd12add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd107250 0x55e6dd12a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 --2- 172.21.15.138:0/3077405568 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd106930 0x55e6dd103650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- 172.21.15.138:0/3077405568 >> 172.21.15.138:0/3077405568 conn(0x55e6dd104760 msgr2=0x55e6dd102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- 172.21.15.138:0/3077405568 shutdown_connections 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- 172.21.15.138:0/3077405568 wait complete. 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 -- start start 2024-08-20T01:18:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.038+0000 7fae63231740 1 Processor -- start 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd106930 0x55e6dd118bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd107250 0x55e6dd119150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b04c0 con 0x55e6dd106930 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b1930 con 0x55e6dd1131e0 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fae581b0b90 con 0x55e6dd107250 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae63231740 10 client.0 _ll_register_callbacks cb 0x7fae4401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd106930 0x55e6dd118bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36032/0 (socket says 172.21.15.138:36032) 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 -- 172.21.15.138:0/3853987967 learned_addr learned my addr 172.21.15.138:0/3853987967 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5ce00640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd107250 0x55e6dd119150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 -- 172.21.15.138:0/3853987967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd107250 msgr2=0x55e6dd119150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd107250 0x55e6dd119150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 -- 172.21.15.138:0/3853987967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd106930 msgr2=0x55e6dd118bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd106930 0x55e6dd118bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.039+0000 7fae5e200640 1 -- 172.21.15.138:0/3853987967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6dce47620 con 0x55e6dd1131e0 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae5e200640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae50009ca0 tx=0x7fae500095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e258489a9abb0f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae3f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:18:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae50059060 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae4c0aa7c0 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9541+0+0 (secure 0 0 0) 0x7fae50003c40 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fae50060050 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae63231740 10 client.26729 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.040+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55e6dd18d430 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae4c0aa7c0 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fae50054640 con 0x55e6dd1131e0 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 10 client.26729 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 10 client.26729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 20 client.26729 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd18dbf0 0x55e6dd190010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 4 client.26729 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6dd079c00 con 0x55e6dd18dbf0 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae5d800640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd18dbf0 0x55e6dd190010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae5d800640 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd18dbf0 0x55e6dd190010 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3f3b1f51884852f server_cookie=8dea1887d03b0c6d in_seq=0 out_seq=0 2024-08-20T01:18:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.041+0000 7fae3f400640 10 client.26729 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae3f400640 1 -- 172.21.15.138:0/3853987967 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6dd079c00 con 0x55e6dd18dbf0 2024-08-20T01:18:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae3f400640 10 client.26729 handle_command_reply: tid=0 2024-08-20T01:18:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 client.26729 shutdown 2024-08-20T01:18:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 msgr2=0x55e6dd184890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fae50009ca0 tx=0x7fae500095c0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 shutdown_connections 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55e6dd18dbf0 0x55e6dd190010 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6dd1131e0 0x55e6dd184890 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55e6dd107250 0x55e6dd119150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae3f400640 0 client.26729 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 --2- 172.21.15.138:0/3853987967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55e6dd106930 0x55e6dd118bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 >> 172.21.15.138:0/3853987967 conn(0x55e6dd104760 msgr2=0x55e6dd103760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.080+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 shutdown_connections 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.081+0000 7fae63231740 1 -- 172.21.15.138:0/3853987967 wait complete. 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.081+0000 7fae63231740 20 client.26729 trim_cache size 0 max 16384 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:18:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 msgr2=0x7fae400b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 0x7fae400b6200 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae50009430 tx=0x7fae50004a40 comp rx=0 tx=0).stop 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 msgr2=0x7fae5819b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae5819b0c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00e9f0 tx=0x7fae4c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 shutdown_connections 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fae400b3d40 0x7fae400b6200 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fae5810ef70 0x7fae581a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58075c20 0x7fae5819b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 --2- 172.21.15.138:0/1537070685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fae58075270 0x7fae5819b0c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.082+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 >> 172.21.15.138:0/1537070685 conn(0x7fae580fe430 msgr2=0x7fae58111100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.083+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 shutdown_connections 2024-08-20T01:18:07.083 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:07.083+0000 7fae60200640 1 -- 172.21.15.138:0/1537070685 wait complete. 2024-08-20T01:18:07.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 385082 3.53 MB/sec execute 379 sec latency 392.184 ms 2024-08-20T01:18:07.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:06 smithi167 ceph-mon[21965]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 KiB/s wr, 117 op/s 2024-08-20T01:18:07.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:06 smithi142 ceph-mon[21799]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 KiB/s wr, 117 op/s 2024-08-20T01:18:07.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:06 smithi138 ceph-mon[18076]: pgmap v670: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 KiB/s wr, 117 op/s 2024-08-20T01:18:08.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 387045 3.56 MB/sec execute 380 sec latency 30.172 ms 2024-08-20T01:18:09.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:08 smithi138 ceph-mon[18076]: pgmap v671: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 788 KiB/s wr, 119 op/s 2024-08-20T01:18:09.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 388368 3.56 MB/sec execute 381 sec latency 118.326 ms 2024-08-20T01:18:09.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:08 smithi167 ceph-mon[21965]: pgmap v671: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 788 KiB/s wr, 119 op/s 2024-08-20T01:18:09.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:08 smithi142 ceph-mon[21799]: pgmap v671: 129 pgs: 5 active+clean+snaptrim, 12 active+clean+snaptrim_wait, 112 active+clean; 270 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 788 KiB/s wr, 119 op/s 2024-08-20T01:18:10.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 390522 3.59 MB/sec execute 382 sec latency 170.062 ms 2024-08-20T01:18:11.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 391753 3.59 MB/sec execute 383 sec latency 23.613 ms 2024-08-20T01:18:11.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:10 smithi167 ceph-mon[21965]: pgmap v672: 129 pgs: 129 active+clean; 272 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:18:11.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:10 smithi142 ceph-mon[21799]: pgmap v672: 129 pgs: 129 active+clean; 272 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:18:11.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:10 smithi138 ceph-mon[18076]: pgmap v672: 129 pgs: 129 active+clean; 272 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-08-20T01:18:12.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 391911 3.58 MB/sec execute 384 sec latency 885.087 ms 2024-08-20T01:18:13.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:12 smithi138 ceph-mon[18076]: pgmap v673: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-08-20T01:18:13.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 393771 3.61 MB/sec execute 385 sec latency 1267.049 ms 2024-08-20T01:18:13.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:12 smithi167 ceph-mon[21965]: pgmap v673: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-08-20T01:18:13.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:12 smithi142 ceph-mon[21799]: pgmap v673: 129 pgs: 129 active+clean; 277 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-08-20T01:18:14.182 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395051 3.61 MB/sec execute 386 sec latency 18.997 ms 2024-08-20T01:18:15.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395459 3.61 MB/sec execute 387 sec latency 751.209 ms 2024-08-20T01:18:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:15 smithi167 ceph-mon[21965]: osdmap e97: 12 total, 12 up, 12 in 2024-08-20T01:18:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:15 smithi167 ceph-mon[21965]: pgmap v675: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-20T01:18:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:15 smithi142 ceph-mon[21799]: osdmap e97: 12 total, 12 up, 12 in 2024-08-20T01:18:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:15 smithi142 ceph-mon[21799]: pgmap v675: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-20T01:18:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:15 smithi138 ceph-mon[18076]: osdmap e97: 12 total, 12 up, 12 in 2024-08-20T01:18:15.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:15 smithi138 ceph-mon[18076]: pgmap v675: 129 pgs: 129 active+clean; 281 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-08-20T01:18:16.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395459 3.60 MB/sec execute 388 sec latency 1751.332 ms 2024-08-20T01:18:17.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395459 3.59 MB/sec execute 389 sec latency 2751.517 ms 2024-08-20T01:18:17.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:16 smithi167 ceph-mon[21965]: pgmap v676: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-08-20T01:18:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:16 smithi142 ceph-mon[21799]: pgmap v676: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-08-20T01:18:17.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:16 smithi138 ceph-mon[18076]: pgmap v676: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 181 op/s 2024-08-20T01:18:18.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395459 3.58 MB/sec execute 390 sec latency 3751.633 ms 2024-08-20T01:18:19.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 395459 3.57 MB/sec execute 391 sec latency 4751.783 ms 2024-08-20T01:18:19.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:18 smithi167 ceph-mon[21965]: pgmap v677: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:18:19.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:18 smithi142 ceph-mon[21799]: pgmap v677: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:18:19.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:18 smithi138 ceph-mon[18076]: pgmap v677: 129 pgs: 129 active+clean; 282 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-08-20T01:18:20.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 397384 3.59 MB/sec execute 392 sec latency 4804.839 ms 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 -- 172.21.15.138:0/4156934403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb410a610 msgr2=0x7f7bb410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 --2- 172.21.15.138:0/4156934403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb410a610 0x7f7bb410aa50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009900 tx=0x7f7b9c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 -- 172.21.15.138:0/4156934403 shutdown_connections 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 --2- 172.21.15.138:0/4156934403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb4111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 --2- 172.21.15.138:0/4156934403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb410a610 0x7f7bb410aa50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 --2- 172.21.15.138:0/4156934403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb4104c10 0x7f7bb4104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 -- 172.21.15.138:0/4156934403 >> 172.21.15.138:0/4156934403 conn(0x7f7bb4100610 msgr2=0x7f7bb4102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 -- 172.21.15.138:0/4156934403 shutdown_connections 2024-08-20T01:18:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.226+0000 7f7bbb200640 1 -- 172.21.15.138:0/4156934403 wait complete. 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 Processor -- start 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 -- start start 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 0x7f7bb419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb41a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4114c30 con 0x7f7bb410a610 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4114ab0 con 0x7f7bb4104c10 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.227+0000 7f7bbb200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb4114db0 con 0x7f7bb410af90 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb41a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33350/0 (socket says 172.21.15.138:33350) 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 -- 172.21.15.138:0/215689453 learned_addr learned my addr 172.21.15.138:0/215689453 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7babe00640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 0x7f7bb419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 -- 172.21.15.138:0/215689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 msgr2=0x7f7bb41a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb41a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 -- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 msgr2=0x7f7bb419b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 0x7f7bb419b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 -- 172.21.15.138:0/215689453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb41a11b0 con 0x7f7bb4104c10 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7babe00640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 0x7f7bb419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb9200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb41a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400ea60 tx=0x7f7ba400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.228+0000 7f7bb1600640 1 -- 172.21.15.138:0/215689453 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400cde0 con 0x7f7bb4104c10 2024-08-20T01:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.229+0000 7f7bb1600640 1 -- 172.21.15.138:0/215689453 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba40035f0 con 0x7f7bb4104c10 2024-08-20T01:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.229+0000 7f7bb1600640 1 -- 172.21.15.138:0/215689453 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4010690 con 0x7f7bb4104c10 2024-08-20T01:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.229+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb41a14a0 con 0x7f7bb4104c10 2024-08-20T01:18:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.229+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb41a7b60 con 0x7f7bb4104c10 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.232+0000 7f7bbe0af740 1 Processor -- start 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.232+0000 7f7bb1600640 1 -- 172.21.15.138:0/215689453 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7ba40037a0 con 0x7f7bb4104c10 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.232+0000 7f7bbe0af740 1 -- start start 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.232+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a6889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.232+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a694940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a697270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b27a5fe210 con 0x55b27a694e80 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b27a5ed710 con 0x55b27a6885d0 2024-08-20T01:18:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b27a5f1750 con 0x55b27a688ef0 2024-08-20T01:18:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a694940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a6889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a6889b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33352/0 (socket says 172.21.15.138:33352) 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb9200640 1 -- 172.21.15.138:0/3130426868 learned_addr learned my addr 172.21.15.138:0/3130426868 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7babe00640 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a697270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 -- 172.21.15.138:0/3130426868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 msgr2=0x55b27a6889b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a6889b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 -- 172.21.15.138:0/3130426868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 msgr2=0x55b27a697270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a697270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7bb3e00640 1 -- 172.21.15.138:0/3130426868 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27a5b9de0 con 0x55b27a688ef0 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.233+0000 7f7babe00640 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a697270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a694940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40e1ad0 tx=0x7f7ba40eaef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd803d4c867340e5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7ba9600640 1 -- 172.21.15.138:0/3130426868 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40f3070 con 0x55b27a688ef0 2024-08-20T01:18:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7ba9600640 1 -- 172.21.15.138:0/3130426868 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba40e60d0 con 0x55b27a688ef0 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7ba9600640 1 -- 172.21.15.138:0/3130426868 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40ef620 con 0x55b27a688ef0 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7bb1600640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 0x7f7b980b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7bb1600640 1 -- 172.21.15.138:0/215689453 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f7ba40e4780 con 0x7f7bb4104c10 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.234+0000 7f7bb9200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 0x7f7b980b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- 172.21.15.138:0/3130426868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 msgr2=0x55b27a694940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a694940 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40e1ad0 tx=0x7f7ba40eaef0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- 172.21.15.138:0/3130426868 shutdown_connections 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a697270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a694940 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/3130426868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a6889b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- 172.21.15.138:0/3130426868 >> 172.21.15.138:0/3130426868 conn(0x55b27a6863b0 msgr2=0x55b27a6875a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- 172.21.15.138:0/3130426868 shutdown_connections 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- 172.21.15.138:0/3130426868 wait complete. 2024-08-20T01:18:20.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bb9200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 0x7f7b980b6200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009fd0 tx=0x7f7bac009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.235+0000 7f7bbe0af740 1 -- start start 2024-08-20T01:18:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 Processor -- start 2024-08-20T01:18:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a688ef0 0x55b27a692320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a694e80 0x55b27a692860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b27a526ab0 con 0x55b27a688ef0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b27a594140 con 0x55b27a6885d0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b27a697b60 con 0x55b27a694e80 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bbe0af740 10 client.0 _ll_register_callbacks cb 0x55b27a5a75d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.236+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33366/0 (socket says 172.21.15.138:33366) 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 -- 172.21.15.138:0/4061901349 learned_addr learned my addr 172.21.15.138:0/4061901349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7babe00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a688ef0 0x55b27a692320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb9200640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a694e80 0x55b27a692860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 -- 172.21.15.138:0/4061901349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a694e80 msgr2=0x55b27a692860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a694e80 0x55b27a692860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 -- 172.21.15.138:0/4061901349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a688ef0 msgr2=0x55b27a692320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a688ef0 0x55b27a692320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 -- 172.21.15.138:0/4061901349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27a595170 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40a8200 tx=0x7f7ba4003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edce15023255dea4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40f3020 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.237+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f7ba40e4d80 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b27a5c4a30 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7ba40a6e50 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7bbe0af740 10 client.26741 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b27a350790 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b27a5c4a30 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f7ba40e1d50 con 0x55b27a6885d0 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7bbe0af740 10 client.26741 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:20.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.238+0000 7f7bbe0af740 10 client.26741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7bbe0af740 20 client.26741 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a6a4c00 0x55b27a6a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7bbe0af740 4 client.26741 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b27a600160 con 0x55b27a6a4c00 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7babe00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a6a4c00 0x55b27a6a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7babe00640 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a6a4c00 0x55b27a6a6ff0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb34fb4599e8f5fd server_cookie=dae03ef4850cc12d in_seq=0 out_seq=0 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.239+0000 7f7ba2000640 10 client.26741 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7ba2000640 1 -- 172.21.15.138:0/4061901349 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b27a600160 con 0x55b27a6a4c00 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7ba2000640 10 client.26741 handle_command_reply: tid=0 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 client.26741 shutdown 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 msgr2=0x55b27a691de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40a8200 tx=0x7f7ba4003cf0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 shutdown_connections 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a6a4c00 0x55b27a6a6ff0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a694e80 0x55b27a692860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a688ef0 0x55b27a692320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7ba2000640 0 client.26741 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/4061901349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a691de0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.240+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 >> 172.21.15.138:0/4061901349 conn(0x55b27a6863b0 msgr2=0x55b27a68bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.241+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 shutdown_connections 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.241+0000 7f7bbe0af740 1 -- 172.21.15.138:0/4061901349 wait complete. 2024-08-20T01:18:20.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.241+0000 7f7bbe0af740 20 client.26741 trim_cache size 0 max 16384 2024-08-20T01:18:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.257+0000 7f7bbe0af740 1 Processor -- start 2024-08-20T01:18:20.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.257+0000 7f7bbe0af740 1 -- start start 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a691520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a68efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b4640 con 0x55b27a694e80 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b1ee0 con 0x55b27a6885d0 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41afd50 con 0x55b27a688ef0 2024-08-20T01:18:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33372/0 (socket says 172.21.15.138:33372) 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb3e00640 1 -- 172.21.15.138:0/1074328251 learned_addr learned my addr 172.21.15.138:0/1074328251 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb9200640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a68efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7babe00640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a691520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb3e00640 1 -- 172.21.15.138:0/1074328251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 msgr2=0x55b27a691520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.258+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a691520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7bb3e00640 1 -- 172.21.15.138:0/1074328251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 msgr2=0x55b27a68efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a68efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7bb3e00640 1 -- 172.21.15.138:0/1074328251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b27a3c9620 con 0x55b27a6885d0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7bb9200640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a68efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40e43a0 tx=0x7f7ba40eaef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=675b9c52417ad7f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7ba8c00640 1 -- 172.21.15.138:0/1074328251 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40f3070 con 0x55b27a6885d0 2024-08-20T01:18:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7ba8c00640 1 -- 172.21.15.138:0/1074328251 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ba40ea3e0 con 0x55b27a6885d0 2024-08-20T01:18:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.259+0000 7f7ba8c00640 1 -- 172.21.15.138:0/1074328251 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40aa700 con 0x55b27a6885d0 2024-08-20T01:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1074328251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 msgr2=0x55b27a68c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ba40e43a0 tx=0x7f7ba40eaef0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1074328251 shutdown_connections 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a694e80 0x55b27a68efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a691520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1074328251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a6885d0 0x55b27a68c720 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1074328251 >> 172.21.15.138:0/1074328251 conn(0x55b27a6863b0 msgr2=0x55b27a693c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1074328251 shutdown_connections 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.260+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1074328251 wait complete. 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 -- start start 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 Processor -- start 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a6a6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a694e80 0x55b27a6a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b03a0 con 0x55b27a6885d0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b1810 con 0x55b27a694e80 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.261+0000 7f7bbe0af740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb41b0a70 con 0x55b27a688ef0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bbe0af740 10 client.0 _ll_register_callbacks cb 0x7f7bac009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a6a6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bb3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:56682/0 (socket says 172.21.15.138:56682) 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bb3e00640 1 -- 172.21.15.138:0/1517088961 learned_addr learned my addr 172.21.15.138:0/1517088961 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bb9200640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a694e80 0x55b27a6a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 -- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a694e80 msgr2=0x55b27a6a7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a694e80 0x55b27a6a7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 -- 172.21.15.138:0/1517088961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 msgr2=0x55b27a6aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 -- 172.21.15.138:0/1517088961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b27a6a79c0 con 0x55b27a688ef0 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7babe00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a6a6c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c02a9e0 tx=0x7f7b9c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f7ffe303a2ce61 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.262+0000 7f7ba2000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c035040 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f7b9c005450 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b27a5fbc00 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7b9c005650 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7bbe0af740 10 client.26698 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b27a70e640 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.263+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b27a5fbc00 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f7b9c003a20 con 0x55b27a688ef0 2024-08-20T01:18:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 10 client.26698 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 10 client.26698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 20 client.26698 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a70ee20 0x55b27a711240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 4 client.26698 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b27a5f9340 con 0x55b27a70ee20 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a70ee20 0x55b27a711240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7bb3e00640 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a70ee20 0x55b27a711240 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=414eaa2921d17470 server_cookie=bb41fa45e426f781 in_seq=0 out_seq=0 2024-08-20T01:18:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.264+0000 7f7ba2000640 10 client.26698 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7ba2000640 1 -- 172.21.15.138:0/1517088961 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b27a5f9340 con 0x55b27a70ee20 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7ba2000640 10 client.26698 handle_command_reply: tid=0 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 client.26698 shutdown 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 msgr2=0x55b27a6a6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a6a6c10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c02a9e0 tx=0x7f7b9c005be0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 shutdown_connections 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b27a70ee20 0x55b27a711240 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b27a694e80 0x55b27a6a7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b27a688ef0 0x55b27a6a6c10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7ba2000640 0 client.26698 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 --2- 172.21.15.138:0/1517088961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b27a6885d0 0x55b27a6aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 >> 172.21.15.138:0/1517088961 conn(0x55b27a6863b0 msgr2=0x55b27a69fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 shutdown_connections 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 1 -- 172.21.15.138:0/1517088961 wait complete. 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.309+0000 7f7bbe0af740 20 client.26698 trim_cache size 0 max 16384 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:18:20.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:18:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 msgr2=0x7f7b980b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 0x7f7b980b6200 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009fd0 tx=0x7f7bac009500 comp rx=0 tx=0).stop 2024-08-20T01:18:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 msgr2=0x7f7bb419b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:20.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400ea60 tx=0x7f7ba400ef20 comp rx=0 tx=0).stop 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 shutdown_connections 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7b980b3d40 0x7f7b980b6200 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7bb410af90 0x7f7bb41a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7bb410a610 0x7f7bb419b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 --2- 172.21.15.138:0/215689453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7bb4104c10 0x7f7bb419b110 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 >> 172.21.15.138:0/215689453 conn(0x7f7bb4100610 msgr2=0x7f7bb4102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 shutdown_connections 2024-08-20T01:18:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:20.310+0000 7f7bbb200640 1 -- 172.21.15.138:0/215689453 wait complete. 2024-08-20T01:18:21.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 398694 3.60 MB/sec execute 393 sec latency 24.164 ms 2024-08-20T01:18:21.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:20 smithi167 ceph-mon[21965]: pgmap v678: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:18:21.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:20 smithi142 ceph-mon[21799]: pgmap v678: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:18:21.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:20 smithi138 ceph-mon[18076]: pgmap v678: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 125 op/s 2024-08-20T01:18:22.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 399001 3.59 MB/sec execute 394 sec latency 624.165 ms 2024-08-20T01:18:23.183 INFO:tasks.workunit.client.0.smithi138.stdout: 1 399001 3.58 MB/sec execute 395 sec latency 1624.306 ms 2024-08-20T01:18:23.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:22 smithi167 ceph-mon[21965]: pgmap v679: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-08-20T01:18:23.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:22 smithi142 ceph-mon[21799]: pgmap v679: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-08-20T01:18:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:22 smithi138 ceph-mon[18076]: pgmap v679: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 93 op/s 2024-08-20T01:18:24.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 399001 3.57 MB/sec execute 396 sec latency 2624.447 ms 2024-08-20T01:18:25.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 400927 3.59 MB/sec execute 397 sec latency 2677.466 ms 2024-08-20T01:18:25.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:24 smithi167 ceph-mon[21965]: pgmap v680: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-20T01:18:25.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:24 smithi142 ceph-mon[21799]: pgmap v680: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-20T01:18:25.296 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:24 smithi138 ceph-mon[18076]: pgmap v680: 129 pgs: 129 active+clean; 285 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 104 op/s 2024-08-20T01:18:26.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 401388 3.59 MB/sec execute 398 sec latency 729.846 ms 2024-08-20T01:18:27.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 402551 3.59 MB/sec execute 399 sec latency 352.140 ms 2024-08-20T01:18:27.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:26 smithi167 ceph-mon[21965]: pgmap v681: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-08-20T01:18:27.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:26 smithi142 ceph-mon[21799]: pgmap v681: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-08-20T01:18:27.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:26 smithi138 ceph-mon[18076]: pgmap v681: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 103 op/s 2024-08-20T01:18:28.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 404409 3.61 MB/sec execute 400 sec latency 725.806 ms 2024-08-20T01:18:29.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 405508 3.61 MB/sec execute 401 sec latency 27.151 ms 2024-08-20T01:18:29.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:28 smithi167 ceph-mon[21965]: pgmap v682: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-20T01:18:29.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:28 smithi142 ceph-mon[21799]: pgmap v682: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-20T01:18:29.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:28 smithi138 ceph-mon[18076]: pgmap v682: 129 pgs: 129 active+clean; 290 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-08-20T01:18:30.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 406092 3.61 MB/sec execute 402 sec latency 568.831 ms 2024-08-20T01:18:31.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 406104 3.61 MB/sec execute 403 sec latency 1550.684 ms 2024-08-20T01:18:31.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:30 smithi167 ceph-mon[21965]: pgmap v683: 129 pgs: 129 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:18:31.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:30 smithi142 ceph-mon[21799]: pgmap v683: 129 pgs: 129 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:18:31.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:30 smithi138 ceph-mon[18076]: pgmap v683: 129 pgs: 129 active+clean; 294 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:18:32.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 408324 3.63 MB/sec execute 404 sec latency 18.814 ms 2024-08-20T01:18:33.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 409641 3.63 MB/sec execute 405 sec latency 18.156 ms 2024-08-20T01:18:33.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:32 smithi167 ceph-mon[21965]: pgmap v684: 129 pgs: 129 active+clean; 298 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:18:33.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:32 smithi142 ceph-mon[21799]: pgmap v684: 129 pgs: 129 active+clean; 298 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:18:33.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:32 smithi138 ceph-mon[18076]: pgmap v684: 129 pgs: 129 active+clean; 298 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:18:33.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 -- 172.21.15.138:0/3734062542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8108b80 msgr2=0x7f4ce810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 --2- 172.21.15.138:0/3734062542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8108b80 0x7f4ce810f380 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 -- 172.21.15.138:0/3734062542 shutdown_connections 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 --2- 172.21.15.138:0/3734062542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8108b80 0x7f4ce810f380 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 --2- 172.21.15.138:0/3734062542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108200 0x7f4ce8108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 --2- 172.21.15.138:0/3734062542 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8102800 0x7f4ce8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 -- 172.21.15.138:0/3734062542 >> 172.21.15.138:0/3734062542 conn(0x7f4ce80fe280 msgr2=0x7f4ce81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 -- 172.21.15.138:0/3734062542 shutdown_connections 2024-08-20T01:18:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.423+0000 7f4cee400640 1 -- 172.21.15.138:0/3734062542 wait complete. 2024-08-20T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 Processor -- start 2024-08-20T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 -- start start 2024-08-20T01:18:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108b80 0x7f4ce81a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112890 con 0x7f4ce8102800 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112710 con 0x7f4ce8108b80 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.424+0000 7f4cee400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112a10 con 0x7f4ce8108200 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ced000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ced000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51616/0 (socket says 172.21.15.138:51616) 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ced000640 1 -- 172.21.15.138:0/1788115687 learned_addr learned my addr 172.21.15.138:0/1788115687 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37342/0 (socket says 172.21.15.138:37342) 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108b80 0x7f4ce81a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 -- 172.21.15.138:0/1788115687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108b80 msgr2=0x7f4ce81a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108b80 0x7f4ce81a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 -- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 msgr2=0x7f4ce819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 -- 172.21.15.138:0/1788115687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce81a0fc0 con 0x7f4ce8108200 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ced000640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.425+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.426+0000 7f4ce5600640 1 -- 172.21.15.138:0/1788115687 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400cd80 con 0x7f4ce8108200 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.426+0000 7f4ce5600640 1 -- 172.21.15.138:0/1788115687 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40035f0 con 0x7f4ce8108200 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.426+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce81a12b0 con 0x7f4ce8108200 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.426+0000 7f4ce5600640 1 -- 172.21.15.138:0/1788115687 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4010670 con 0x7f4ce8108200 2024-08-20T01:18:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.426+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a7970 con 0x7f4ce8108200 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.428+0000 7f4cf27f7740 1 Processor -- start 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 -- start start 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cc43c0 0x55b795cc67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b795c2d210 con 0x55b795cb7b10 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b795c1c710 con 0x55b795cc43c0 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b795c20750 con 0x55b795cb8430 2024-08-20T01:18:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ce5600640 1 -- 172.21.15.138:0/1788115687 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4cd40037a0 con 0x7f4ce8108200 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51628/0 (socket says 172.21.15.138:51628) 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 -- 172.21.15.138:0/1678347356 learned_addr learned my addr 172.21.15.138:0/1678347356 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ced000640 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 -- 172.21.15.138:0/1678347356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 msgr2=0x55b795cc3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 -- 172.21.15.138:0/1678347356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cc43c0 msgr2=0x55b795cc67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.429+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cc43c0 0x55b795cc67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.430+0000 7f4ceda00640 1 -- 172.21.15.138:0/1678347356 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b795be8de0 con 0x55b795cb7b10 2024-08-20T01:18:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.430+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e18c98b824d5768 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.430+0000 7f4cda000640 1 -- 172.21.15.138:0/1678347356 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0031cf0 con 0x55b795cb7b10 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.430+0000 7f4cda000640 1 -- 172.21.15.138:0/1678347356 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0036030 con 0x55b795cb7b10 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.430+0000 7f4cda000640 1 -- 172.21.15.138:0/1678347356 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00042a0 con 0x55b795cb7b10 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 -- 172.21.15.138:0/1678347356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 msgr2=0x55b795cb7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce00093e0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4ce5600640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 0x7f4cd00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4ce5600640 1 -- 172.21.15.138:0/1788115687 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4cd40e4660 con 0x7f4ce8108200 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 -- 172.21.15.138:0/1678347356 shutdown_connections 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cc43c0 0x55b795cc67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/1678347356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cb7ef0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 -- 172.21.15.138:0/1678347356 >> 172.21.15.138:0/1678347356 conn(0x55b795cb5940 msgr2=0x55b795cb6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 0x7f4cd00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 -- 172.21.15.138:0/1678347356 shutdown_connections 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.431+0000 7f4cf27f7740 1 -- 172.21.15.138:0/1678347356 wait complete. 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 -- start start 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4ceda00640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 0x7f4cd00b6200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4ce002ade0 tx=0x7f4ce0003da0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 Processor -- start 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb8430 0x55b795cc10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b795b55ab0 con 0x55b795cb7b10 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b795bc3140 con 0x55b795cb8430 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b795cc6ec0 con 0x55b795cc43c0 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4cf27f7740 10 client.0 _ll_register_callbacks cb 0x55b795bd65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.432+0000 7f4ced000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37372/0 (socket says 172.21.15.138:37372) 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 -- 172.21.15.138:0/2425601922 learned_addr learned my addr 172.21.15.138:0/2425601922 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb8430 0x55b795cc10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 -- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb8430 msgr2=0x55b795cc10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb8430 0x55b795cc10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 -- 172.21.15.138:0/2425601922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 msgr2=0x55b795cc3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cc3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 -- 172.21.15.138:0/2425601922 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b795bc4170 con 0x55b795cc43c0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ced000640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ceda00640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce0005d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8061feca41b7539f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ccea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce005e020 con 0x55b795cc43c0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce0036030 con 0x55b795cc43c0 2024-08-20T01:18:33.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.433+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4ce002f810 con 0x55b795cc43c0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4ce0069050 con 0x55b795cc43c0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 10 client.26710 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b79597f790 con 0x55b795cc43c0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0036030 con 0x55b795cc43c0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f4ce0059030 con 0x55b795cc43c0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 10 client.26710 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 10 client.26710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 20 client.26710 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cd3ce0 0x55b795cd60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 4 client.26710 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.434+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b795c2f160 con 0x55b795cd3ce0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.435+0000 7f4ced000640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cd3ce0 0x55b795cd60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.435+0000 7f4ccea00640 10 client.26710 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.435+0000 7f4ced000640 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cd3ce0 0x55b795cd60d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c40c643d82e3cbdd server_cookie=c77b119e239a4ec7 in_seq=0 out_seq=0 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4ccea00640 1 -- 172.21.15.138:0/2425601922 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b795c2f160 con 0x55b795cd3ce0 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4ccea00640 10 client.26710 handle_command_reply: tid=0 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 client.26710 shutdown 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 msgr2=0x55b795cc16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce0005d80 comp rx=0 tx=0).stop 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 shutdown_connections 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cd3ce0 0x55b795cd60d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cc43c0 0x55b795cc16b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb8430 0x55b795cc10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/2425601922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4ccea00640 0 client.26710 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:33.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 >> 172.21.15.138:0/2425601922 conn(0x55b795cb5940 msgr2=0x55b795cbb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 shutdown_connections 2024-08-20T01:18:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 1 -- 172.21.15.138:0/2425601922 wait complete. 2024-08-20T01:18:33.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.436+0000 7f4cf27f7740 20 client.26710 trim_cache size 0 max 16384 2024-08-20T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 Processor -- start 2024-08-20T01:18:33.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 -- start start 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb5940 0x55b795cdb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cdbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b43d0 con 0x55b795cb7b10 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b1c70 con 0x55b795cb5940 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.451+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81afae0 con 0x55b795cb8430 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ce7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cdbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ce7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cdbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51646/0 (socket says 172.21.15.138:51646) 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:37388/0 (socket says 172.21.15.138:37388) 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ce7e00640 1 -- 172.21.15.138:0/549905970 learned_addr learned my addr 172.21.15.138:0/549905970 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ced000640 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb5940 0x55b795cdb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 -- 172.21.15.138:0/549905970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb5940 msgr2=0x55b795cdb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb5940 0x55b795cdb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 -- 172.21.15.138:0/549905970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 msgr2=0x55b795cdbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cdbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 -- 172.21.15.138:0/549905970 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7959f8620 con 0x55b795cb8430 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4ceda00640 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0030a00 tx=0x7f4ce0031fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f9a768f2d531ca5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4cd9600640 1 -- 172.21.15.138:0/549905970 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce005e040 con 0x55b795cb8430 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4cd9600640 1 -- 172.21.15.138:0/549905970 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0036070 con 0x55b795cb8430 2024-08-20T01:18:33.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.452+0000 7f4cd9600640 1 -- 172.21.15.138:0/549905970 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0056040 con 0x55b795cb8430 2024-08-20T01:18:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- 172.21.15.138:0/549905970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 msgr2=0x55b795cbe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0030a00 tx=0x7f4ce0031fd0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- 172.21.15.138:0/549905970 shutdown_connections 2024-08-20T01:18:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cbe2e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb7b10 0x55b795cdbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/549905970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb5940 0x55b795cdb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- 172.21.15.138:0/549905970 >> 172.21.15.138:0/549905970 conn(0x55b795cc3df0 msgr2=0x55b795cbc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- 172.21.15.138:0/549905970 shutdown_connections 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- 172.21.15.138:0/549905970 wait complete. 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.453+0000 7f4cf27f7740 1 -- start start 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 Processor -- start 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 0x55b795cc85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b0130 con 0x55b795cb5940 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b15a0 con 0x55b795cb7b10 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b0800 con 0x55b795cb8430 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4cf27f7740 10 client.0 _ll_register_callbacks cb 0x7f4cdc009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:33.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4ced000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4ce7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 0x55b795cc85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4ced000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51654/0 (socket says 172.21.15.138:51654) 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4ced000640 1 -- 172.21.15.138:0/4044177627 learned_addr learned my addr 172.21.15.138:0/4044177627 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.454+0000 7f4ceda00640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 -- 172.21.15.138:0/4044177627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 msgr2=0x55b795cc8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 -- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 msgr2=0x55b795cc85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 0x55b795cc85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 -- 172.21.15.138:0/4044177627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b795d46130 con 0x55b795cb5940 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 0x55b795cc85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:33.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ceda00640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ced000640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55b795cd6820 tx=0x7f4cdc012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26495db59c29a189 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ccea00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc020cb0 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4cdc005020 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ce0036070 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc033050 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4cf27f7740 10 client.16713 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.455+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55b795d47440 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ce0036070 con 0x55b795cb5940 2024-08-20T01:18:33.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f4cdc010430 con 0x55b795cb5940 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 10 client.16713 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 20 client.16713 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cc43c0 0x55b795d49c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 4 client.16713 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b795c2ac00 con 0x55b795cc43c0 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cc43c0 0x55b795d49c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4ccea00640 10 client.16713 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:33.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.456+0000 7f4ce7e00640 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cc43c0 0x55b795d49c50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45b1dfa8aadf2e4b server_cookie=1c94563dab0306e4 in_seq=0 out_seq=0 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4ccea00640 1 -- 172.21.15.138:0/4044177627 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b795c2ac00 con 0x55b795cc43c0 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4ccea00640 10 client.16713 handle_command_reply: tid=0 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 client.16713 shutdown 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 msgr2=0x55b795cc8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x55b795cd6820 tx=0x7f4cdc012f90 comp rx=0 tx=0).stop 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 shutdown_connections 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b795cc43c0 0x55b795d49c50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b795cb8430 0x55b795cc8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795cb7b10 0x55b795cc85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 --2- 172.21.15.138:0/4044177627 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b795cb5940 0x55b795cc8070 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4ccea00640 0 client.16713 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.496+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 >> 172.21.15.138:0/4044177627 conn(0x55b795cc3df0 msgr2=0x55b795ccefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 shutdown_connections 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cf27f7740 1 -- 172.21.15.138:0/4044177627 wait complete. 2024-08-20T01:18:33.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cf27f7740 20 client.16713 trim_cache size 0 max 16384 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 msgr2=0x7f4cd00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 0x7f4cd00b6200 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4ce002ade0 tx=0x7f4ce0003da0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 msgr2=0x7f4ce819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.497+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 shutdown_connections 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4cd00b3d40 0x7f4cd00b6200 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ce8108b80 0x7f4ce81a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4ce8108200 0x7f4ce819b460 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 --2- 172.21.15.138:0/1788115687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4ce8102800 0x7f4ce819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 >> 172.21.15.138:0/1788115687 conn(0x7f4ce80fe280 msgr2=0x7f4ce81047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 shutdown_connections 2024-08-20T01:18:33.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:33.498+0000 7f4cee400640 1 -- 172.21.15.138:0/1788115687 wait complete. 2024-08-20T01:18:34.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 411694 3.66 MB/sec execute 406 sec latency 304.270 ms 2024-08-20T01:18:35.146 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:34 smithi138 ceph-mon[18076]: pgmap v685: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-08-20T01:18:35.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 412997 3.66 MB/sec execute 407 sec latency 27.208 ms 2024-08-20T01:18:35.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:34 smithi167 ceph-mon[21965]: pgmap v685: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-08-20T01:18:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:34 smithi142 ceph-mon[21799]: pgmap v685: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 158 op/s 2024-08-20T01:18:36.184 INFO:tasks.workunit.client.0.smithi138.stdout: 1 413183 3.65 MB/sec execute 408 sec latency 491.496 ms 2024-08-20T01:18:37.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 413183 3.65 MB/sec execute 409 sec latency 1388.024 ms 2024-08-20T01:18:37.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:36 smithi167 ceph-mon[21965]: pgmap v686: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-08-20T01:18:37.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:36 smithi142 ceph-mon[21799]: pgmap v686: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-08-20T01:18:37.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:36 smithi138 ceph-mon[18076]: pgmap v686: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-08-20T01:18:38.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 413183 3.64 MB/sec execute 410 sec latency 2388.168 ms 2024-08-20T01:18:39.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 413183 3.63 MB/sec execute 411 sec latency 3388.276 ms 2024-08-20T01:18:39.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:38 smithi167 ceph-mon[21965]: pgmap v687: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:18:39.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:38 smithi142 ceph-mon[21799]: pgmap v687: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:18:39.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:38 smithi138 ceph-mon[18076]: pgmap v687: 129 pgs: 129 active+clean; 303 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:18:40.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 415365 3.65 MB/sec execute 412 sec latency 3439.962 ms 2024-08-20T01:18:41.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 415425 3.64 MB/sec execute 413 sec latency 671.968 ms 2024-08-20T01:18:41.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:40 smithi167 ceph-mon[21965]: pgmap v688: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-20T01:18:41.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:40 smithi142 ceph-mon[21799]: pgmap v688: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-20T01:18:41.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:40 smithi138 ceph-mon[18076]: pgmap v688: 129 pgs: 129 active+clean; 307 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 174 op/s 2024-08-20T01:18:42.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 416425 3.65 MB/sec execute 414 sec latency 1043.345 ms 2024-08-20T01:18:43.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 416732 3.64 MB/sec execute 415 sec latency 756.790 ms 2024-08-20T01:18:43.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:42 smithi167 ceph-mon[21965]: pgmap v689: 129 pgs: 129 active+clean; 309 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-08-20T01:18:43.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:42 smithi142 ceph-mon[21799]: pgmap v689: 129 pgs: 129 active+clean; 309 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-08-20T01:18:43.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:42 smithi138 ceph-mon[18076]: pgmap v689: 129 pgs: 129 active+clean; 309 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-08-20T01:18:44.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 416732 3.63 MB/sec execute 416 sec latency 1756.917 ms 2024-08-20T01:18:45.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 418926 3.65 MB/sec execute 417 sec latency 1808.645 ms 2024-08-20T01:18:45.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:44 smithi167 ceph-mon[21965]: pgmap v690: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-08-20T01:18:45.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:44 smithi142 ceph-mon[21799]: pgmap v690: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-08-20T01:18:45.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:44 smithi138 ceph-mon[18076]: pgmap v690: 129 pgs: 129 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 153 op/s 2024-08-20T01:18:46.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 419033 3.65 MB/sec execute 418 sec latency 555.995 ms 2024-08-20T01:18:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 -- 172.21.15.138:0/3156567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108200 msgr2=0x7f8df8108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 --2- 172.21.15.138:0/3156567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108200 0x7f8df8108640 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8de4009900 tx=0x7f8de402a410 comp rx=0 tx=0).stop 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 -- 172.21.15.138:0/3156567112 shutdown_connections 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 --2- 172.21.15.138:0/3156567112 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108b80 0x7f8df810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 --2- 172.21.15.138:0/3156567112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108200 0x7f8df8108640 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 --2- 172.21.15.138:0/3156567112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 0x7f8df8102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 -- 172.21.15.138:0/3156567112 >> 172.21.15.138:0/3156567112 conn(0x7f8df80fe280 msgr2=0x7f8df81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 -- 172.21.15.138:0/3156567112 shutdown_connections 2024-08-20T01:18:46.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.623+0000 7f8dfd000640 1 -- 172.21.15.138:0/3156567112 wait complete. 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 Processor -- start 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 -- start start 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 0x7f8df819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108b80 0x7f8df81a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8df8112850 con 0x7f8df8102800 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81126d0 con 0x7f8df8108b80 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.624+0000 7f8dfd000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81129d0 con 0x7f8df8108200 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38378/0 (socket says 172.21.15.138:38378) 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 0x7f8df819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 -- 172.21.15.138:0/1343770002 learned_addr learned my addr 172.21.15.138:0/1343770002 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df7e00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108b80 0x7f8df81a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 -- 172.21.15.138:0/1343770002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108b80 msgr2=0x7f8df81a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108b80 0x7f8df81a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 -- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 msgr2=0x7f8df819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 0x7f8df819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 -- 172.21.15.138:0/1343770002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df81a0fe0 con 0x7f8df8108200 2024-08-20T01:18:46.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8df6a00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8de4005ab0 tx=0x7f8de4003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8defe00640 1 -- 172.21.15.138:0/1343770002 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de403f070 con 0x7f8df8108200 2024-08-20T01:18:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df81a1270 con 0x7f8df8108200 2024-08-20T01:18:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8defe00640 1 -- 172.21.15.138:0/1343770002 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de4031b60 con 0x7f8df8108200 2024-08-20T01:18:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.625+0000 7f8defe00640 1 -- 172.21.15.138:0/1343770002 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de4035030 con 0x7f8df8108200 2024-08-20T01:18:46.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.626+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df81a7990 con 0x7f8df8108200 2024-08-20T01:18:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.628+0000 7f8e01673740 1 Processor -- start 2024-08-20T01:18:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 -- start start 2024-08-20T01:18:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8defe00640 1 -- 172.21.15.138:0/1343770002 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8de4042070 con 0x7f8df8108200 2024-08-20T01:18:46.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x559480829ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x5594808387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55948079f210 con 0x5594808363c0 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55948078e710 con 0x55948082a430 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.629+0000 7f8e01673740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559480792750 con 0x559480829b10 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x559480829ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38796/0 (socket says 172.21.15.138:38796) 2024-08-20T01:18:46.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 -- 172.21.15.138:0/2644885455 learned_addr learned my addr 172.21.15.138:0/2644885455 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:46.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df6a00640 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x5594808387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 -- 172.21.15.138:0/2644885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 msgr2=0x559480829ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x559480829ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 -- 172.21.15.138:0/2644885455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 msgr2=0x5594808387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x5594808387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.630+0000 7f8df7400640 1 -- 172.21.15.138:0/2644885455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948075ade0 con 0x55948082a430 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8df7400640 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009cf0 tx=0x7f8de8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b89b7abd6cc33dd4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8decc00640 1 -- 172.21.15.138:0/2644885455 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de801e070 con 0x55948082a430 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8decc00640 1 -- 172.21.15.138:0/2644885455 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de8019ae0 con 0x55948082a430 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8decc00640 1 -- 172.21.15.138:0/2644885455 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de800ea10 con 0x55948082a430 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8defe00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.631+0000 7f8defe00640 1 -- 172.21.15.138:0/1343770002 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f8de4104780 con 0x7f8df8108200 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8df7e00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 -- 172.21.15.138:0/2644885455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 msgr2=0x559480835e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009cf0 tx=0x7f8de8011810 comp rx=0 tx=0).stop 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 -- 172.21.15.138:0/2644885455 shutdown_connections 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x5594808387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55948082a430 0x559480835e80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 --2- 172.21.15.138:0/2644885455 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x559480829ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 -- 172.21.15.138:0/2644885455 >> 172.21.15.138:0/2644885455 conn(0x559480827940 msgr2=0x559480828ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 -- 172.21.15.138:0/2644885455 shutdown_connections 2024-08-20T01:18:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8e01673740 1 -- 172.21.15.138:0/2644885455 wait complete. 2024-08-20T01:18:46.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.632+0000 7f8df7e00640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005fd0 tx=0x7f8df0009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 -- start start 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 Processor -- start 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x559480835b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x559480832fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5594806c7ab0 con 0x55948082a430 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559480735140 con 0x559480829b10 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5594808390a0 con 0x5594808363c0 2024-08-20T01:18:46.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.633+0000 7f8e01673740 10 client.0 _ll_register_callbacks cb 0x5594807485d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x559480835b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x559480835b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38806/0 (socket says 172.21.15.138:38806) 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38388/0 (socket says 172.21.15.138:38388) 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7400640 1 -- 172.21.15.138:0/273623977 learned_addr learned my addr 172.21.15.138:0/273623977 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df6a00640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x559480832fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 -- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 msgr2=0x559480835b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x559480835b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 -- 172.21.15.138:0/273623977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 msgr2=0x559480832fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x559480832fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 -- 172.21.15.138:0/273623977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559480736170 con 0x5594808363c0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.634+0000 7f8df7e00640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8df0011e90 tx=0x7f8df0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0fc2e15947d0509 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8de1600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:18:46.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df000f870 con 0x5594808363c0 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f8df000fa10 con 0x5594808363c0 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de8019ae0 con 0x5594808363c0 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8df000fc10 con 0x5594808363c0 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8e01673740 10 client.26728 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.635+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5594804f1790 con 0x5594808363c0 2024-08-20T01:18:46.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.636+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de8019ae0 con 0x5594808363c0 2024-08-20T01:18:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.636+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f8df0026070 con 0x5594808363c0 2024-08-20T01:18:46.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.636+0000 7f8e01673740 10 client.26728 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.636+0000 7f8e01673740 10 client.26728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.636+0000 7f8e01673740 20 client.26728 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559480846020 0x559480848450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8e01673740 4 client.26728 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594807a1160 con 0x559480846020 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8df7400640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559480846020 0x559480848450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8df7400640 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559480846020 0x559480848450 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e1513a85b283ee6 server_cookie=ff4dc53e986fe4d in_seq=0 out_seq=0 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.637+0000 7f8de1600640 10 client.26728 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.638+0000 7f8de1600640 1 -- 172.21.15.138:0/273623977 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594807a1160 con 0x559480846020 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.638+0000 7f8de1600640 10 client.26728 handle_command_reply: tid=0 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.638+0000 7f8e01673740 1 client.26728 shutdown 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.638+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 msgr2=0x559480833500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.638+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8df0011e90 tx=0x7f8df0004ab0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 shutdown_connections 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559480846020 0x559480848450 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5594808363c0 0x559480833500 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x559480832fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 --2- 172.21.15.138:0/273623977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x559480835b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8de1600640 0 client.26728 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 >> 172.21.15.138:0/273623977 conn(0x559480827940 msgr2=0x55948082d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 shutdown_connections 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 1 -- 172.21.15.138:0/273623977 wait complete. 2024-08-20T01:18:46.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.639+0000 7f8e01673740 20 client.26728 trim_cache size 0 max 16384 2024-08-20T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 Processor -- start 2024-08-20T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 -- start start 2024-08-20T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 0x55948084dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x55948084e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81b4540 con 0x5594808363c0 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81b1de0 con 0x559480829b10 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8e01673740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81afc50 con 0x55948082a430 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8df7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38812/0 (socket says 172.21.15.138:38812) 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8df7400640 1 -- 172.21.15.138:0/1075724955 learned_addr learned my addr 172.21.15.138:0/1075724955 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8df6a00640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 0x55948084dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.655+0000 7f8df7e00640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x55948084e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 -- 172.21.15.138:0/1075724955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 msgr2=0x55948084dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 0x55948084dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 -- 172.21.15.138:0/1075724955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 msgr2=0x55948084e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x55948084e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 -- 172.21.15.138:0/1075724955 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55948056a620 con 0x559480829b10 2024-08-20T01:18:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df6a00640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 0x55948084dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8df7400640 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8de800e660 tx=0x7f8de800e690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04e58ed6e2ab3b9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8de3e00640 1 -- 172.21.15.138:0/1075724955 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de801e040 con 0x559480829b10 2024-08-20T01:18:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8de3e00640 1 -- 172.21.15.138:0/1075724955 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de80194b0 con 0x559480829b10 2024-08-20T01:18:46.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.656+0000 7f8de3e00640 1 -- 172.21.15.138:0/1075724955 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de800ea80 con 0x559480829b10 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 -- 172.21.15.138:0/1075724955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 msgr2=0x5594808268a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8de800e660 tx=0x7f8de800e690 comp rx=0 tx=0).stop 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 -- 172.21.15.138:0/1075724955 shutdown_connections 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5594808363c0 0x55948084e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55948082a430 0x55948084dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 --2- 172.21.15.138:0/1075724955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559480829b10 0x5594808268a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 -- 172.21.15.138:0/1075724955 >> 172.21.15.138:0/1075724955 conn(0x559480827940 msgr2=0x559480826190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 -- 172.21.15.138:0/1075724955 shutdown_connections 2024-08-20T01:18:46.660 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.657+0000 7f8e01673740 1 -- 172.21.15.138:0/1075724955 wait complete. 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 -- start start 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 Processor -- start 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x55948083c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594808363c0 0x5594808a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81b02a0 con 0x55948082a430 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81b1710 con 0x5594808363c0 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8df81b0970 con 0x559480829b10 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.658+0000 7f8e01673740 10 client.0 _ll_register_callbacks cb 0x7f8df0009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38400/0 (socket says 172.21.15.138:38400) 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 -- 172.21.15.138:0/2991752742 learned_addr learned my addr 172.21.15.138:0/2991752742 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:46.661 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7e00640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594808363c0 0x5594808a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df6a00640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x55948083c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 -- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594808363c0 msgr2=0x5594808a8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594808363c0 0x5594808a8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 -- 172.21.15.138:0/2991752742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 msgr2=0x55948083c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x55948083c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 -- 172.21.15.138:0/2991752742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594808a94b0 con 0x559480829b10 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8df7400640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005f50 tx=0x7f8de8009cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41c024f754765e70 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8de1600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de8033070 con 0x559480829b10 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f8de8019a20 con 0x559480829b10 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.659+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de80194b0 con 0x559480829b10 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8de803f050 con 0x559480829b10 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 10 client.26734 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:46.662 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5594808b0920 con 0x559480829b10 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8de80194b0 con 0x559480829b10 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f8de800ec30 con 0x559480829b10 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 10 client.26734 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 20 client.26734 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594808b0da0 0x5594808b3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 4 client.26734 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.660+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55948079cc00 con 0x5594808b0da0 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.661+0000 7f8df6a00640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594808b0da0 0x5594808b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.661+0000 7f8df6a00640 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594808b0da0 0x5594808b3190 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba96d7bb20021b49 server_cookie=a85121222b6bba5 in_seq=0 out_seq=0 2024-08-20T01:18:46.663 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.661+0000 7f8de1600640 10 client.26734 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.695+0000 7f8de1600640 1 -- 172.21.15.138:0/2991752742 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55948079cc00 con 0x5594808b0da0 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.695+0000 7f8de1600640 10 client.26734 handle_command_reply: tid=0 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.695+0000 7f8e01673740 1 client.26734 shutdown 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.695+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 msgr2=0x55948083bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.695+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005f50 tx=0x7f8de8009cb0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 shutdown_connections 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5594808b0da0 0x5594808b3190 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5594808363c0 0x5594808a8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55948082a430 0x55948083c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 --2- 172.21.15.138:0/2991752742 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559480829b10 0x55948083bc40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8de1600640 0 client.26734 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 >> 172.21.15.138:0/2991752742 conn(0x559480827940 msgr2=0x559480834e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 shutdown_connections 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 1 -- 172.21.15.138:0/2991752742 wait complete. 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.696+0000 7f8e01673740 20 client.26734 trim_cache size 0 max 16384 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:18:46.696 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 msgr2=0x7f8ddc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8df0005fd0 tx=0x7f8df0009500 comp rx=0 tx=0).stop 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 msgr2=0x7f8df819b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8de4005ab0 tx=0x7f8de4003c10 comp rx=0 tx=0).stop 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 shutdown_connections 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8ddc0b3d40 0x7f8ddc0b6200 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8df8108b80 0x7f8df81a0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8df8108200 0x7f8df819b480 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 --2- 172.21.15.138:0/1343770002 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8df8102800 0x7f8df819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 >> 172.21.15.138:0/1343770002 conn(0x7f8df80fe280 msgr2=0x7f8df8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 shutdown_connections 2024-08-20T01:18:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:46.697+0000 7f8dfd000640 1 -- 172.21.15.138:0/1343770002 wait complete. 2024-08-20T01:18:47.185 INFO:tasks.workunit.client.0.smithi138.stdout: 1 420275 3.65 MB/sec execute 419 sec latency 274.219 ms 2024-08-20T01:18:47.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:46 smithi167 ceph-mon[21965]: pgmap v691: 129 pgs: 129 active+clean; 310 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-08-20T01:18:47.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:46 smithi142 ceph-mon[21799]: pgmap v691: 129 pgs: 129 active+clean; 310 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-08-20T01:18:47.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:46 smithi138 ceph-mon[18076]: pgmap v691: 129 pgs: 129 active+clean; 310 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-08-20T01:18:48.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 420275 3.64 MB/sec execute 420 sec latency 1274.377 ms 2024-08-20T01:18:49.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 420275 3.63 MB/sec execute 421 sec latency 2274.533 ms 2024-08-20T01:18:49.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:48 smithi167 ceph-mon[21965]: pgmap v692: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-08-20T01:18:49.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:48 smithi142 ceph-mon[21799]: pgmap v692: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-08-20T01:18:49.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:48 smithi138 ceph-mon[18076]: pgmap v692: 129 pgs: 129 active+clean; 311 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 104 op/s 2024-08-20T01:18:50.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 422449 3.66 MB/sec execute 422 sec latency 2325.523 ms 2024-08-20T01:18:51.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:50 smithi138 ceph-mon[18076]: pgmap v693: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:18:51.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 423396 3.65 MB/sec execute 423 sec latency 282.706 ms 2024-08-20T01:18:51.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:50 smithi167 ceph-mon[21965]: pgmap v693: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:18:51.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:50 smithi142 ceph-mon[21799]: pgmap v693: 129 pgs: 129 active+clean; 314 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 139 op/s 2024-08-20T01:18:52.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 423824 3.65 MB/sec execute 424 sec latency 752.297 ms 2024-08-20T01:18:53.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 425747 3.67 MB/sec execute 425 sec latency 1123.436 ms 2024-08-20T01:18:53.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:52 smithi167 ceph-mon[21965]: pgmap v694: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-20T01:18:53.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:52 smithi142 ceph-mon[21799]: pgmap v694: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-20T01:18:53.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:52 smithi138 ceph-mon[18076]: pgmap v694: 129 pgs: 129 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 144 op/s 2024-08-20T01:18:54.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427083 3.68 MB/sec execute 426 sec latency 21.290 ms 2024-08-20T01:18:54.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:54 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:18:54.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:54 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:18:54.797 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:54 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:18:55.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427365 3.67 MB/sec execute 427 sec latency 683.377 ms 2024-08-20T01:18:55.891 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: pgmap v695: 129 pgs: 129 active+clean; 317 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:18:55.892 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:55 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: pgmap v695: 129 pgs: 129 active+clean; 317 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:18:55.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:55 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: pgmap v695: 129 pgs: 129 active+clean; 317 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 146 op/s 2024-08-20T01:18:55.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:18:55.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:18:55.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:55 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:18:56.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427365 3.66 MB/sec execute 428 sec latency 1683.419 ms 2024-08-20T01:18:57.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427365 3.65 MB/sec execute 429 sec latency 2683.485 ms 2024-08-20T01:18:57.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:56 smithi167 ceph-mon[21965]: pgmap v696: 129 pgs: 129 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-08-20T01:18:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:56 smithi142 ceph-mon[21799]: pgmap v696: 129 pgs: 129 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-08-20T01:18:57.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:56 smithi138 ceph-mon[18076]: pgmap v696: 129 pgs: 129 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 152 op/s 2024-08-20T01:18:58.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427365 3.65 MB/sec execute 430 sec latency 3683.624 ms 2024-08-20T01:18:59.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 427365 3.64 MB/sec execute 431 sec latency 4683.726 ms 2024-08-20T01:18:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:18:58 smithi167 ceph-mon[21965]: pgmap v697: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-08-20T01:18:59.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:18:58 smithi142 ceph-mon[21799]: pgmap v697: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-08-20T01:18:59.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:18:58 smithi138 ceph-mon[18076]: pgmap v697: 129 pgs: 129 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.6 MiB/s wr, 138 op/s 2024-08-20T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3339316657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 msgr2=0x7f4b9c103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3339316657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c103cf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009900 tx=0x7f4b9002a410 comp rx=0 tx=0).stop 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3339316657 shutdown_connections 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3339316657 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 0x7f4b9c10f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3339316657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 0x7f4b9c10c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3339316657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c103cf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3339316657 >> 172.21.15.138:0/3339316657 conn(0x7f4b9c078620 msgr2=0x7f4b9c0ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3339316657 shutdown_connections 2024-08-20T01:18:59.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.818+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3339316657 wait complete. 2024-08-20T01:18:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 Processor -- start 2024-08-20T01:18:59.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 -- start start 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 0x7f4b9c101af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 0x7f4b9c102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c112a10 con 0x7f4b9c104230 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c112890 con 0x7f4b9c103910 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.819+0000 7f4ba3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c112b90 con 0x7f4b9c10cd30 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53144/0 (socket says 172.21.15.138:53144) 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 -- 172.21.15.138:0/3224027887 learned_addr learned my addr 172.21.15.138:0/3224027887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 0x7f4b9c101af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 0x7f4b9c102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 -- 172.21.15.138:0/3224027887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 msgr2=0x7f4b9c102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 0x7f4b9c102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 -- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 msgr2=0x7f4b9c101af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 0x7f4b9c101af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 -- 172.21.15.138:0/3224027887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9c1a7320 con 0x7f4b9c103910 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.820+0000 7f4ba1400640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005ab0 tx=0x7f4b90003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.821+0000 7f4b9a000640 1 -- 172.21.15.138:0/3224027887 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b9003f070 con 0x7f4b9c103910 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.821+0000 7f4b9a000640 1 -- 172.21.15.138:0/3224027887 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b90031b60 con 0x7f4b9c103910 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.821+0000 7f4b9a000640 1 -- 172.21.15.138:0/3224027887 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b90035030 con 0x7f4b9c103910 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.821+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b9c1a75b0 con 0x7f4b9c103910 2024-08-20T01:18:59.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.821+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b9c1a7a50 con 0x7f4b9c103910 2024-08-20T01:18:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 Processor -- start 2024-08-20T01:18:59.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 -- start start 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b5ef0 0x5585c61c1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61c1e80 0x5585c61c4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.824+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585c612b210 con 0x5585c61c1e80 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585c611a710 con 0x5585c61b55d0 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585c611e750 con 0x5585c61b5ef0 2024-08-20T01:18:59.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4b9a000640 1 -- 172.21.15.138:0/3224027887 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4b90042070 con 0x7f4b9c103910 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53146/0 (socket says 172.21.15.138:53146) 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 -- 172.21.15.138:0/2864479338 learned_addr learned my addr 172.21.15.138:0/2864479338 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61c1e80 0x5585c61c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 -- 172.21.15.138:0/2864479338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b5ef0 msgr2=0x5585c61c1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b5ef0 0x5585c61c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 -- 172.21.15.138:0/2864479338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61c1e80 msgr2=0x5585c61c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61c1e80 0x5585c61c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 -- 172.21.15.138:0/2864479338 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c60e6de0 con 0x5585c61b55d0 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.825+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009fd0 tx=0x7f4b940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e13c46a0f3a0943 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.826+0000 7f4b8e600640 1 -- 172.21.15.138:0/2864479338 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94004d20 con 0x5585c61b55d0 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.826+0000 7f4b8e600640 1 -- 172.21.15.138:0/2864479338 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b94004050 con 0x5585c61b55d0 2024-08-20T01:18:59.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.826+0000 7f4b8e600640 1 -- 172.21.15.138:0/2864479338 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b94005260 con 0x5585c61b55d0 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4b9a000640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 0x7f4b840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/2864479338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 msgr2=0x5585c61b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4b94009fd0 tx=0x7f4b940118d0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/2864479338 shutdown_connections 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61c1e80 0x5585c61c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b5ef0 0x5585c61c1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/2864479338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b55d0 0x5585c61b59b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/2864479338 >> 172.21.15.138:0/2864479338 conn(0x5585c61b33b0 msgr2=0x5585c61b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4b9a000640 1 -- 172.21.15.138:0/3224027887 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4b90104a00 con 0x7f4b9c103910 2024-08-20T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba1400640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 0x7f4b840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/2864479338 shutdown_connections 2024-08-20T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.827+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/2864479338 wait complete. 2024-08-20T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.828+0000 7f4ba1400640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 0x7f4b840b6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5585c61b8750 tx=0x7f4b9002a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.828+0000 7f4ba6e3a740 1 -- start start 2024-08-20T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.828+0000 7f4ba6e3a740 1 Processor -- start 2024-08-20T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5585c6053ab0 con 0x5585c61b55d0 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5585c60c1140 con 0x5585c61b5ef0 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5585c61c4b60 con 0x5585c61c1e80 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba6e3a740 10 client.0 _ll_register_callbacks cb 0x5585c60d45d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42394/0 (socket says 172.21.15.138:42394) 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53148/0 (socket says 172.21.15.138:53148) 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba1e00640 1 -- 172.21.15.138:0/1715510749 learned_addr learned my addr 172.21.15.138:0/1715510749 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1715510749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 msgr2=0x5585c61bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1715510749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 msgr2=0x5585c61c1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.829+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1715510749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c60c2170 con 0x5585c61b5ef0 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c06b3a0 tx=0x7f4b88012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbaefccc0bf4e8cc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4b82a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:18:59.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88004030 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4b880041d0 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5585c60f1a30 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4b880043d0 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4ba6e3a740 10 client.26792 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.830+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5585c5e7d790 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5585c60f1a30 con 0x5585c61b5ef0 2024-08-20T01:18:59.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f4b88003b50 con 0x5585c61b5ef0 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 10 client.26792 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 10 client.26792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 20 client.26792 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c61d1d20 0x5585c61d4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 4 client.26792 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585c612d160 con 0x5585c61d1d20 2024-08-20T01:18:59.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.831+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c61d1d20 0x5585c61d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.832+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c61d1d20 0x5585c61d4190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed1a4f3f006620c server_cookie=761d25ad8eb66e0d in_seq=0 out_seq=0 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.832+0000 7f4b82a00640 10 client.26792 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.832+0000 7f4b82a00640 1 -- 172.21.15.138:0/1715510749 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5585c612d160 con 0x5585c61d1d20 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.832+0000 7f4b82a00640 10 client.26792 handle_command_reply: tid=0 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.832+0000 7f4ba6e3a740 1 client.26792 shutdown 2024-08-20T01:18:59.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 msgr2=0x5585c61bed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c06b3a0 tx=0x7f4b88012b60 comp rx=0 tx=0).stop 2024-08-20T01:18:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 shutdown_connections 2024-08-20T01:18:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c61d1d20 0x5585c61d4190 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bed80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1715510749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61c1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 >> 172.21.15.138:0/1715510749 conn(0x5585c61b33b0 msgr2=0x5585c61b8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4b82a00640 0 client.26792 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 shutdown_connections 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1715510749 wait complete. 2024-08-20T01:18:59.846 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.833+0000 7f4ba6e3a740 20 client.26792 trim_cache size 0 max 16384 2024-08-20T01:18:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.852+0000 7f4ba6e3a740 1 Processor -- start 2024-08-20T01:18:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.852+0000 7f4ba6e3a740 1 -- start start 2024-08-20T01:18:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b55d0 0x5585c61ae230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61c1e80 0x5585c61b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b45e0 con 0x5585c61b5ef0 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b1e80 con 0x5585c61c1e80 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1afcf0 con 0x5585c61b55d0 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61c1e80 0x5585c61b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:42404/0 (socket says 172.21.15.138:42404) 2024-08-20T01:18:59.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1489399836 learned_addr learned my addr 172.21.15.138:0/1489399836 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b55d0 0x5585c61ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1489399836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b55d0 msgr2=0x5585c61ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.853+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b55d0 0x5585c61ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1489399836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61c1e80 msgr2=0x5585c61b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61c1e80 0x5585c61b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.856 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4ba0a00640 1 -- 172.21.15.138:0/1489399836 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585c5ef6620 con 0x5585c61b5ef0 2024-08-20T01:18:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cf7d78e331fbc3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4b8dc00640 1 -- 172.21.15.138:0/1489399836 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88004000 con 0x5585c61b5ef0 2024-08-20T01:18:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4b8dc00640 1 -- 172.21.15.138:0/1489399836 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b880041a0 con 0x5585c61b5ef0 2024-08-20T01:18:59.857 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.854+0000 7f4b8dc00640 1 -- 172.21.15.138:0/1489399836 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003bc0 con 0x5585c61b5ef0 2024-08-20T01:18:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1489399836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 msgr2=0x5585c61b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4b880058c0 tx=0x7f4b8800f810 comp rx=0 tx=0).stop 2024-08-20T01:18:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1489399836 shutdown_connections 2024-08-20T01:18:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61c1e80 0x5585c61b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b5ef0 0x5585c61b4450 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/1489399836 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61b55d0 0x5585c61ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1489399836 >> 172.21.15.138:0/1489399836 conn(0x5585c61b33b0 msgr2=0x5585c61bd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1489399836 shutdown_connections 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.855+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/1489399836 wait complete. 2024-08-20T01:18:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 -- start start 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 Processor -- start 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b0340 con 0x5585c61b55d0 2024-08-20T01:18:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b17b0 con 0x5585c61b5ef0 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9c1b0a10 con 0x5585c61c1e80 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba6e3a740 10 client.0 _ll_register_callbacks cb 0x7f4b880097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53170/0 (socket says 172.21.15.138:53170) 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.856+0000 7f4ba0a00640 1 -- 172.21.15.138:0/3932589050 learned_addr learned my addr 172.21.15.138:0/3932589050 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:18:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 -- 172.21.15.138:0/3932589050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 msgr2=0x5585c61c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 -- 172.21.15.138:0/3932589050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 msgr2=0x5585c61bb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61bb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 -- 172.21.15.138:0/3932589050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585c61bc1b0 con 0x5585c61b5ef0 2024-08-20T01:18:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba0a00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000bc0 tx=0x7f4b88003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=243ccb0d957b42df server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4b82a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88003fa0 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b880041a0 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.857+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9453+0+0 (secure 0 0 0) 0x7f4b8802e030 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4b8803e050 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 10 client.26798 fetch_fsmap learned FSMap version 40 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5585c61ca470 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b880041a0 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f4b88004570 con 0x5585c61b5ef0 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 10 client.26798 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:18:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 10 client.26798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 20 client.26798 populate_metadata read hostname 'smithi138' 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c623ca70 0x5585c623ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 4 client.26798 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585c6128c00 con 0x5585c623ca70 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.858+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c623ca70 0x5585c623ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.859+0000 7f4ba1e00640 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c623ca70 0x5585c623ee60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7bad18d9559e36 server_cookie=be168b9a7bf73a55 in_seq=0 out_seq=0 2024-08-20T01:18:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.859+0000 7f4b82a00640 10 client.26798 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.895+0000 7f4b82a00640 1 -- 172.21.15.138:0/3932589050 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585c6128c00 con 0x5585c623ca70 2024-08-20T01:18:59.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.895+0000 7f4b82a00640 10 client.26798 handle_command_reply: tid=0 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 client.26798 shutdown 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 msgr2=0x5585c61bb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b88000bc0 tx=0x7f4b88003d50 comp rx=0 tx=0).stop 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 shutdown_connections 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5585c623ca70 0x5585c623ee60 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5585c61c1e80 0x5585c61c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585c61b5ef0 0x5585c61bb8c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4b82a00640 0 client.26798 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 --2- 172.21.15.138:0/3932589050 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5585c61b55d0 0x5585c61bb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 >> 172.21.15.138:0/3932589050 conn(0x5585c61b33b0 msgr2=0x5585c61c0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 shutdown_connections 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 1 -- 172.21.15.138:0/3932589050 wait complete. 2024-08-20T01:18:59.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.896+0000 7f4ba6e3a740 20 client.26798 trim_cache size 0 max 16384 2024-08-20T01:18:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:18:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:18:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:18:59.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:18:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 msgr2=0x7f4b840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.900 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 0x7f4b840b6200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5585c61b8750 tx=0x7f4b9002a410 comp rx=0 tx=0).stop 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 msgr2=0x7f4b9c1015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b90005ab0 tx=0x7f4b90003c10 comp rx=0 tx=0).stop 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 shutdown_connections 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4b840b3d40 0x7f4b840b6200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4b9c10cd30 0x7f4b9c102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4b9c104230 0x7f4b9c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 --2- 172.21.15.138:0/3224027887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b9c103910 0x7f4b9c1015b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:18:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 >> 172.21.15.138:0/3224027887 conn(0x7f4b9c078620 msgr2=0x7f4b9c0ff490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:18:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.897+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 shutdown_connections 2024-08-20T01:18:59.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:18:59.898+0000 7f4ba3e00640 1 -- 172.21.15.138:0/3224027887 wait complete. 2024-08-20T01:19:00.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 429499 3.66 MB/sec execute 432 sec latency 4735.028 ms 2024-08-20T01:19:01.186 INFO:tasks.workunit.client.0.smithi138.stdout: 1 430088 3.65 MB/sec execute 433 sec latency 227.022 ms 2024-08-20T01:19:01.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:00 smithi167 ceph-mon[21965]: pgmap v698: 129 pgs: 129 active+clean; 322 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:19:01.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:00 smithi142 ceph-mon[21799]: pgmap v698: 129 pgs: 129 active+clean; 322 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:19:01.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:00 smithi138 ceph-mon[18076]: pgmap v698: 129 pgs: 129 active+clean; 322 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.9 MiB/s wr, 154 op/s 2024-08-20T01:19:02.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 430252 3.65 MB/sec execute 434 sec latency 572.339 ms 2024-08-20T01:19:03.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 430914 3.65 MB/sec execute 435 sec latency 399.974 ms 2024-08-20T01:19:03.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:02 smithi167 ceph-mon[21965]: pgmap v699: 129 pgs: 129 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:19:03.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:02 smithi142 ceph-mon[21799]: pgmap v699: 129 pgs: 129 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:19:03.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:02 smithi138 ceph-mon[18076]: pgmap v699: 129 pgs: 129 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:19:04.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 430914 3.64 MB/sec execute 436 sec latency 1400.100 ms 2024-08-20T01:19:05.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 432976 3.66 MB/sec execute 437 sec latency 1450.717 ms 2024-08-20T01:19:05.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:05 smithi138 ceph-mon[18076]: osdmap e98: 12 total, 12 up, 12 in 2024-08-20T01:19:05.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:05 smithi138 ceph-mon[18076]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-20T01:19:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:05 smithi167 ceph-mon[21965]: osdmap e98: 12 total, 12 up, 12 in 2024-08-20T01:19:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:05 smithi167 ceph-mon[21965]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-20T01:19:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:05 smithi142 ceph-mon[21799]: osdmap e98: 12 total, 12 up, 12 in 2024-08-20T01:19:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:05 smithi142 ceph-mon[21799]: pgmap v701: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-08-20T01:19:06.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 433649 3.66 MB/sec execute 438 sec latency 339.590 ms 2024-08-20T01:19:07.068 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:06 smithi138 ceph-mon[18076]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-08-20T01:19:07.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 433837 3.65 MB/sec execute 439 sec latency 842.468 ms 2024-08-20T01:19:07.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:06 smithi167 ceph-mon[21965]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-08-20T01:19:07.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:06 smithi142 ceph-mon[21799]: pgmap v702: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 168 op/s 2024-08-20T01:19:08.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 434458 3.65 MB/sec execute 440 sec latency 659.897 ms 2024-08-20T01:19:09.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 434458 3.64 MB/sec execute 441 sec latency 1660.025 ms 2024-08-20T01:19:09.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:08 smithi167 ceph-mon[21965]: pgmap v703: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-08-20T01:19:09.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:08 smithi142 ceph-mon[21799]: pgmap v703: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-08-20T01:19:09.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:08 smithi138 ceph-mon[18076]: pgmap v703: 129 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 121 active+clean; 324 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-08-20T01:19:10.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 436655 3.66 MB/sec execute 442 sec latency 1711.659 ms 2024-08-20T01:19:11.078 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:10 smithi138 ceph-mon[18076]: pgmap v704: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-08-20T01:19:11.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 437781 3.67 MB/sec execute 443 sec latency 145.587 ms 2024-08-20T01:19:11.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:10 smithi167 ceph-mon[21965]: pgmap v704: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-08-20T01:19:11.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:10 smithi142 ceph-mon[21799]: pgmap v704: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 177 op/s 2024-08-20T01:19:12.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 438005 3.66 MB/sec execute 444 sec latency 671.498 ms 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 -- 172.21.15.138:0/1018472304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c102800 msgr2=0x7fbc7c102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 --2- 172.21.15.138:0/1018472304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c102800 0x7fbc7c102be0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c009900 tx=0x7fbc6c02a410 comp rx=0 tx=0).stop 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 -- 172.21.15.138:0/1018472304 shutdown_connections 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 --2- 172.21.15.138:0/1018472304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c108b80 0x7fbc7c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 --2- 172.21.15.138:0/1018472304 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108200 0x7fbc7c108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 --2- 172.21.15.138:0/1018472304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c102800 0x7fbc7c102be0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 -- 172.21.15.138:0/1018472304 >> 172.21.15.138:0/1018472304 conn(0x7fbc7c0fe280 msgr2=0x7fbc7c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 -- 172.21.15.138:0/1018472304 shutdown_connections 2024-08-20T01:19:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.024+0000 7fbc80c00640 1 -- 172.21.15.138:0/1018472304 wait complete. 2024-08-20T01:19:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.025+0000 7fbc80c00640 1 Processor -- start 2024-08-20T01:19:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.025+0000 7fbc80c00640 1 -- start start 2024-08-20T01:19:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.025+0000 7fbc80c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 0x7fbc7c19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc80c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 0x7fbc7c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc80c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 0x7fbc7c1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc80c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c112800 con 0x7fbc7c102800 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc80c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c112680 con 0x7fbc7c108200 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc80c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c112980 con 0x7fbc7c108b80 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 0x7fbc7c19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 0x7fbc7c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 0x7fbc7c19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53746/0 (socket says 172.21.15.138:53746) 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc7b400640 1 -- 172.21.15.138:0/3409707192 learned_addr learned my addr 172.21.15.138:0/3409707192 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc7be00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 0x7fbc7c1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 -- 172.21.15.138:0/3409707192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 msgr2=0x7fbc7c1a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 0x7fbc7c1a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 -- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 msgr2=0x7fbc7c19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 0x7fbc7c19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc73e00640 1 -- 172.21.15.138:0/3409707192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc7c1a10a0 con 0x7fbc7c108200 2024-08-20T01:19:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.026+0000 7fbc7be00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 0x7fbc7c1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc73e00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 0x7fbc7c19b540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e9f0 tx=0x7fbc6400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc78c00640 1 -- 172.21.15.138:0/3409707192 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6400ccc0 con 0x7fbc7c108200 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc7c1a1390 con 0x7fbc7c108200 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc78c00640 1 -- 172.21.15.138:0/3409707192 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc640108e0 con 0x7fbc7c108200 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc78c00640 1 -- 172.21.15.138:0/3409707192 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc64018880 con 0x7fbc7c108200 2024-08-20T01:19:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.027+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc7c1a7c30 con 0x7fbc7c108200 2024-08-20T01:19:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.031+0000 7fbc84ff7740 1 Processor -- start 2024-08-20T01:19:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.031+0000 7fbc84ff7740 1 -- start start 2024-08-20T01:19:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.031+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.031+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563675f51210 con 0x563675fdc0f0 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563675f40710 con 0x563675fdb7d0 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563675f44750 con 0x563675fe8080 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc78c00640 1 -- 172.21.15.138:0/3409707192 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fbc64010430 con 0x7fbc7c108200 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:44844/0 (socket says 172.21.15.138:44844) 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53754/0 (socket says 172.21.15.138:53754) 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 -- 172.21.15.138:0/4237191183 learned_addr learned my addr 172.21.15.138:0/4237191183 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:13.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 -- 172.21.15.138:0/4237191183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 msgr2=0x563675fea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 -- 172.21.15.138:0/4237191183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 msgr2=0x563675fe7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.032+0000 7fbc7be00640 1 -- 172.21.15.138:0/4237191183 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563675f0cde0 con 0x563675fdb7d0 2024-08-20T01:19:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc7be00640 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc74010450 tx=0x7fbc74010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b858e0a944aa77c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc6be00640 1 -- 172.21.15.138:0/4237191183 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc74017070 con 0x563675fdb7d0 2024-08-20T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc6be00640 1 -- 172.21.15.138:0/4237191183 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc74011c10 con 0x563675fdb7d0 2024-08-20T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc6be00640 1 -- 172.21.15.138:0/4237191183 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7401acc0 con 0x563675fdb7d0 2024-08-20T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc78c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 0x7fbc600b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4237191183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 msgr2=0x563675fdbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.033+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbc74010450 tx=0x7fbc74010910 comp rx=0 tx=0).stop 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc78c00640 1 -- 172.21.15.138:0/3409707192 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9541+0+0 (secure 0 0 0) 0x7fbc640e4790 con 0x7fbc7c108200 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4237191183 shutdown_connections 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4237191183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdbbb0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4237191183 >> 172.21.15.138:0/4237191183 conn(0x563675fd9600 msgr2=0x563675fda7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc7b400640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 0x7fbc600b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4237191183 shutdown_connections 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4237191183 wait complete. 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 -- start start 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc7b400640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 0x7fbc600b6280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c031440 tx=0x7fbc6c0058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.034+0000 7fbc84ff7740 1 Processor -- start 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fe5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fe5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563675e79ab0 con 0x563675fdc0f0 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563675ee7140 con 0x563675fdb7d0 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563675fead60 con 0x563675fe8080 2024-08-20T01:19:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc84ff7740 10 client.0 _ll_register_callbacks cb 0x563675efa5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53770/0 (socket says 172.21.15.138:53770) 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc73e00640 1 -- 172.21.15.138:0/923064412 learned_addr learned my addr 172.21.15.138:0/923064412 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc7b400640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fe5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.035+0000 7fbc7be00640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fe5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 -- 172.21.15.138:0/923064412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 msgr2=0x563675fe5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fe5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 -- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 msgr2=0x563675fe5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fe5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 -- 172.21.15.138:0/923064412 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563675ee8170 con 0x563675fdc0f0 2024-08-20T01:19:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc73e00640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbc640a7020 tx=0x7fbc64010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da98d37ae4688345 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc68c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640aafa0 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9541+0+0 (secure 0 0 0) 0x7fbc640a7230 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563675f17a30 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbc640af5a0 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc84ff7740 10 client.16761 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.036+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563675ca3790 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563675f17a30 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fbc640c5c00 con 0x563675fdc0f0 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 10 client.16761 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 20 client.16761 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563675ff7fb0 0x563675ffa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 4 client.16761 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563675f53160 con 0x563675ff7fb0 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.037+0000 7fbc7b400640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563675ff7fb0 0x563675ffa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.038+0000 7fbc7b400640 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563675ff7fb0 0x563675ffa3a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=685bfccb3cf4a6f8 server_cookie=14da04fe1793c276 in_seq=0 out_seq=0 2024-08-20T01:19:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.038+0000 7fbc68c00640 10 client.16761 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc68c00640 1 -- 172.21.15.138:0/923064412 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563675f53160 con 0x563675ff7fb0 2024-08-20T01:19:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc68c00640 10 client.16761 handle_command_reply: tid=0 2024-08-20T01:19:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 client.16761 shutdown 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 msgr2=0x563675fe5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fbc640a7020 tx=0x7fbc64010860 comp rx=0 tx=0).stop 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 shutdown_connections 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563675ff7fb0 0x563675ffa3a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fe8080 0x563675fe5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fdc0f0 0x563675fe5560 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/923064412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fe5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 >> 172.21.15.138:0/923064412 conn(0x563675fd9600 msgr2=0x563675fdf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc68c00640 0 client.16761 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 shutdown_connections 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 1 -- 172.21.15.138:0/923064412 wait complete. 2024-08-20T01:19:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.039+0000 7fbc84ff7740 20 client.16761 trim_cache size 0 max 16384 2024-08-20T01:19:13.044 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:12 smithi138 ceph-mon[18076]: pgmap v705: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-08-20T01:19:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 Processor -- start 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 -- start start 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdf920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675fffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676000100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b4610 con 0x563675fe8080 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b1eb0 con 0x563675fdb7d0 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.055+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1afd00 con 0x563675fdc0f0 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc7b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdf920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:44858/0 (socket says 172.21.15.138:44858) 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675fffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc7b400640 1 -- 172.21.15.138:0/4085366377 learned_addr learned my addr 172.21.15.138:0/4085366377 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc7be00640 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676000100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 -- 172.21.15.138:0/4085366377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 msgr2=0x563675fdf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 -- 172.21.15.138:0/4085366377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 msgr2=0x563676000100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676000100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 -- 172.21.15.138:0/4085366377 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563675d1c620 con 0x563675fdc0f0 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc73e00640 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675fffbc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc640e1b90 tx=0x7fbc64009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05d5f0982a2bb77 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.056+0000 7fbc6b400640 1 -- 172.21.15.138:0/4085366377 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640b4050 con 0x563675fdc0f0 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc6b400640 1 -- 172.21.15.138:0/4085366377 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc640a6b70 con 0x563675fdc0f0 2024-08-20T01:19:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc6b400640 1 -- 172.21.15.138:0/4085366377 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc640e4370 con 0x563675fdc0f0 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4085366377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 msgr2=0x563675fffbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675fffbc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc640e1b90 tx=0x7fbc64009700 comp rx=0 tx=0).stop 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4085366377 shutdown_connections 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676000100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675fffbc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/4085366377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675fdf920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4085366377 >> 172.21.15.138:0/4085366377 conn(0x563675fd9600 msgr2=0x563675ff9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4085366377 shutdown_connections 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.057+0000 7fbc84ff7740 1 -- 172.21.15.138:0/4085366377 wait complete. 2024-08-20T01:19:13.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 -- start start 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 Processor -- start 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675feb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676060b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b0370 con 0x563675fe8080 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b17e0 con 0x563675fdb7d0 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7c1b0a40 con 0x563675fdc0f0 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.058+0000 7fbc84ff7740 10 client.0 _ll_register_callbacks cb 0x7fbc640a6e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56302/0 (socket says 172.21.15.138:56302) 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc7be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676060b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 -- 172.21.15.138:0/571621585 learned_addr learned my addr 172.21.15.138:0/571621585 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:13.061 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc7b400640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675feb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 -- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 msgr2=0x563675feb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675feb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 -- 172.21.15.138:0/571621585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 msgr2=0x563676060b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676060b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 -- 172.21.15.138:0/571621585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636760610c0 con 0x563675fdc0f0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc73e00640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9db31a403c3912b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc68c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc6408e060 con 0x563675fdc0f0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc640a6b70 con 0x563675fdc0f0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.059+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9541+0+0 (secure 0 0 0) 0x7fbc640a72a0 con 0x563675fdc0f0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fbc640b4050 con 0x563675fdc0f0 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 10 client.26764 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:13.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563676062380 con 0x563675fdc0f0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc640a6b70 con 0x563675fdc0f0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fbc640c5ca0 con 0x563675fdc0f0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 10 client.26764 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 20 client.26764 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5636760627b0 0x563676064ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 4 client.26764 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.060+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563675fe0bb0 con 0x5636760627b0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.061+0000 7fbc7b400640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5636760627b0 0x563676064ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.061+0000 7fbc7b400640 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5636760627b0 0x563676064ba0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dacf3581c84a6e83 server_cookie=b9d8294f55a26b4d in_seq=0 out_seq=0 2024-08-20T01:19:13.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.061+0000 7fbc68c00640 10 client.26764 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc68c00640 1 -- 172.21.15.138:0/571621585 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563675fe0bb0 con 0x5636760627b0 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc68c00640 10 client.26764 handle_command_reply: tid=0 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 client.26764 shutdown 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 msgr2=0x563675feb960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc64000c00 tx=0x7fbc6400ce60 comp rx=0 tx=0).stop 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 shutdown_connections 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5636760627b0 0x563676064ba0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.097 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563675fe8080 0x563676060b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563675fdc0f0 0x563675feb960 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc68c00640 0 client.26764 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 --2- 172.21.15.138:0/571621585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563675fdb7d0 0x563675feb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 >> 172.21.15.138:0/571621585 conn(0x563675fd9600 msgr2=0x563675fe4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 shutdown_connections 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.098+0000 7fbc84ff7740 1 -- 172.21.15.138:0/571621585 wait complete. 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc84ff7740 20 client.26764 trim_cache size 0 max 16384 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:19:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 msgr2=0x7fbc600b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 0x7fbc600b6280 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c031440 tx=0x7fbc6c0058d0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 msgr2=0x7fbc7c19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 0x7fbc7c19b540 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc6400e9f0 tx=0x7fbc6400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 shutdown_connections 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fbc600b3dc0 0x7fbc600b6280 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbc7c108b80 0x7fbc7c1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc7c108200 0x7fbc7c19b540 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 --2- 172.21.15.138:0/3409707192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbc7c102800 0x7fbc7c19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.099+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 >> 172.21.15.138:0/3409707192 conn(0x7fbc7c0fe280 msgr2=0x7fbc7c1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.100+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 shutdown_connections 2024-08-20T01:19:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:13.100+0000 7fbc80c00640 1 -- 172.21.15.138:0/3409707192 wait complete. 2024-08-20T01:19:13.187 INFO:tasks.workunit.client.0.smithi138.stdout: 1 438205 3.65 MB/sec execute 445 sec latency 1553.760 ms 2024-08-20T01:19:13.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:12 smithi167 ceph-mon[21965]: pgmap v705: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-08-20T01:19:13.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:12 smithi142 ceph-mon[21799]: pgmap v705: 129 pgs: 129 active+clean; 326 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-08-20T01:19:14.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 440322 3.67 MB/sec execute 446 sec latency 12.366 ms 2024-08-20T01:19:15.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 441547 3.68 MB/sec execute 447 sec latency 311.276 ms 2024-08-20T01:19:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:15 smithi167 ceph-mon[21965]: osdmap e99: 12 total, 12 up, 12 in 2024-08-20T01:19:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:15 smithi167 ceph-mon[21965]: pgmap v707: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-08-20T01:19:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:15 smithi142 ceph-mon[21799]: osdmap e99: 12 total, 12 up, 12 in 2024-08-20T01:19:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:15 smithi142 ceph-mon[21799]: pgmap v707: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-08-20T01:19:15.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:15 smithi138 ceph-mon[18076]: osdmap e99: 12 total, 12 up, 12 in 2024-08-20T01:19:15.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:15 smithi138 ceph-mon[18076]: pgmap v707: 129 pgs: 129 active+clean; 331 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 175 op/s 2024-08-20T01:19:16.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 441547 3.67 MB/sec execute 448 sec latency 1311.323 ms 2024-08-20T01:19:17.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 441547 3.66 MB/sec execute 449 sec latency 2311.391 ms 2024-08-20T01:19:17.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:16 smithi167 ceph-mon[21965]: pgmap v708: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:19:17.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:16 smithi142 ceph-mon[21799]: pgmap v708: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:19:17.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:16 smithi138 ceph-mon[18076]: pgmap v708: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:19:18.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 441547 3.65 MB/sec execute 450 sec latency 3311.630 ms 2024-08-20T01:19:19.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 441547 3.65 MB/sec execute 451 sec latency 4311.736 ms 2024-08-20T01:19:19.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:18 smithi167 ceph-mon[21965]: pgmap v709: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-08-20T01:19:19.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:18 smithi142 ceph-mon[21799]: pgmap v709: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-08-20T01:19:19.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:18 smithi138 ceph-mon[18076]: pgmap v709: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 152 op/s 2024-08-20T01:19:20.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 443688 3.67 MB/sec execute 452 sec latency 4365.113 ms 2024-08-20T01:19:21.040 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:20 smithi138 ceph-mon[18076]: pgmap v710: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-08-20T01:19:21.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 444988 3.67 MB/sec execute 453 sec latency 36.826 ms 2024-08-20T01:19:21.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:20 smithi167 ceph-mon[21965]: pgmap v710: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-08-20T01:19:21.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:20 smithi142 ceph-mon[21799]: pgmap v710: 129 pgs: 129 active+clean; 335 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 128 op/s 2024-08-20T01:19:22.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 445096 3.66 MB/sec execute 454 sec latency 654.693 ms 2024-08-20T01:19:23.188 INFO:tasks.workunit.client.0.smithi138.stdout: 1 446975 3.68 MB/sec execute 455 sec latency 568.484 ms 2024-08-20T01:19:23.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:22 smithi167 ceph-mon[21965]: pgmap v711: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:19:23.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:22 smithi142 ceph-mon[21799]: pgmap v711: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:19:23.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:22 smithi138 ceph-mon[18076]: pgmap v711: 129 pgs: 129 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:19:24.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 448347 3.69 MB/sec execute 456 sec latency 21.012 ms 2024-08-20T01:19:25.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 448638 3.68 MB/sec execute 457 sec latency 660.425 ms 2024-08-20T01:19:25.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:24 smithi167 ceph-mon[21965]: pgmap v712: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-08-20T01:19:25.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:24 smithi142 ceph-mon[21799]: pgmap v712: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-08-20T01:19:25.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:24 smithi138 ceph-mon[18076]: pgmap v712: 129 pgs: 129 active+clean; 344 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 161 op/s 2024-08-20T01:19:26.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 448638 3.67 MB/sec execute 458 sec latency 1660.499 ms 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.228+0000 7fd2d2c00640 1 -- 172.21.15.138:0/1255359583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 msgr2=0x7fd2cc0777e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.228+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/1255359583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc0777e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009900 tx=0x7fd2c002a410 comp rx=0 tx=0).stop 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 -- 172.21.15.138:0/1255359583 shutdown_connections 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/1255359583 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 0x7fd2cc1157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/1255359583 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc112eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/1255359583 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc0777e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 -- 172.21.15.138:0/1255359583 >> 172.21.15.138:0/1255359583 conn(0x7fd2cc100450 msgr2=0x7fd2cc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 -- 172.21.15.138:0/1255359583 shutdown_connections 2024-08-20T01:19:26.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 -- 172.21.15.138:0/1255359583 wait complete. 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.229+0000 7fd2d2c00640 1 Processor -- start 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 -- start start 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 0x7fd2cc1a4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc118e50 con 0x7fd2cc077400 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc118cd0 con 0x7fd2cc077d20 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc118fd0 con 0x7fd2cc1133f0 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 0x7fd2cc1a4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59354/0 (socket says 172.21.15.138:59354) 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d1800640 1 -- 172.21.15.138:0/3066031633 learned_addr learned my addr 172.21.15.138:0/3066031633 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.230+0000 7fd2d0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3066031633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 msgr2=0x7fd2cc1a4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 0x7fd2cc1a4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 msgr2=0x7fd2cc19f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3066031633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc1a53b0 con 0x7fd2cc077d20 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc19f850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800e9f0 tx=0x7fd2b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2ca000640 1 -- 172.21.15.138:0/3066031633 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b800cd80 con 0x7fd2cc077d20 2024-08-20T01:19:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc1a56a0 con 0x7fd2cc077d20 2024-08-20T01:19:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2ca000640 1 -- 172.21.15.138:0/3066031633 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b80035f0 con 0x7fd2cc077d20 2024-08-20T01:19:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.231+0000 7fd2ca000640 1 -- 172.21.15.138:0/3066031633 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8010690 con 0x7fd2cc077d20 2024-08-20T01:19:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.232+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1abf40 con 0x7fd2cc077d20 2024-08-20T01:19:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 Processor -- start 2024-08-20T01:19:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 -- start start 2024-08-20T01:19:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f70ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 0x56127f71d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56127f684210 con 0x56127f70f0f0 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56127f673710 con 0x56127f71b080 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d700d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56127f677750 con 0x56127f70e7d0 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d1800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59360/0 (socket says 172.21.15.138:59360) 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d1800640 1 -- 172.21.15.138:0/3071828473 learned_addr learned my addr 172.21.15.138:0/3071828473 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2ca000640 1 -- 172.21.15.138:0/3066031633 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd2b80037a0 con 0x7fd2cc077d20 2024-08-20T01:19:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.235+0000 7fd2d2200640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f70ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 0x56127f71d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 -- 172.21.15.138:0/3071828473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 msgr2=0x56127f70ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f70ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 -- 172.21.15.138:0/3071828473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 msgr2=0x56127f71d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 0x56127f71d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 -- 172.21.15.138:0/3071828473 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127f63fde0 con 0x56127f70f0f0 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 0x56127f71d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00044a0 tx=0x7fd2c0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ebf13fb4a9dec0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2bea00640 1 -- 172.21.15.138:0/3071828473 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0040070 con 0x56127f70f0f0 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2bea00640 1 -- 172.21.15.138:0/3071828473 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c00047a0 con 0x56127f70f0f0 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.236+0000 7fd2bea00640 1 -- 172.21.15.138:0/3071828473 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0035040 con 0x56127f70f0f0 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 -- 172.21.15.138:0/3071828473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 msgr2=0x56127f71ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00044a0 tx=0x7fd2c0005c00 comp rx=0 tx=0).stop 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 -- 172.21.15.138:0/3071828473 shutdown_connections 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f71b080 0x56127f71d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f70f0f0 0x56127f71ab40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3071828473 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f70ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2d700d740 1 -- 172.21.15.138:0/3071828473 >> 172.21.15.138:0/3071828473 conn(0x56127f70c600 msgr2=0x56127f70d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.237+0000 7fd2ca000640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 0x7fd2b40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2ca000640 1 -- 172.21.15.138:0/3066031633 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7fd2b80e46c0 con 0x7fd2cc077d20 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d700d740 1 -- 172.21.15.138:0/3071828473 shutdown_connections 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d700d740 1 -- 172.21.15.138:0/3071828473 wait complete. 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d2200640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 0x7fd2b40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d700d740 1 -- start start 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d2200640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 0x7fd2b40b6200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc076590 tx=0x7fd2c4009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.238+0000 7fd2d700d740 1 Processor -- start 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f718020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f718aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56127f5acab0 con 0x56127f71b080 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56127f61a140 con 0x56127f70f0f0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56127f71dd00 con 0x56127f70e7d0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d700d740 10 client.0 _ll_register_callbacks cb 0x56127f62d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f718aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f718aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59370/0 (socket says 172.21.15.138:59370) 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d2200640 1 -- 172.21.15.138:0/3249118505 learned_addr learned my addr 172.21.15.138:0/3249118505 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53054/0 (socket says 172.21.15.138:53054) 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.239+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f718020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3249118505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 msgr2=0x56127f718020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f718020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3249118505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 msgr2=0x56127f718aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f718aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3249118505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127f61b170 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80e1a70 tx=0x7fd2b80aab00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a9af1fd3a9f6b31 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2b3400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80af530 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56127f64aa30 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7fd2b80e1cf0 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd2b80b3fe0 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d700d740 10 client.26825 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.240+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56127f3d6790 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56127f64aa30 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fd2b80e4b70 con 0x56127f70f0f0 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 10 client.26825 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:26.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 10 client.26825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 20 client.26825 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f72af80 0x56127f72d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 4 client.26825 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56127f686160 con 0x56127f72af80 2024-08-20T01:19:26.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.241+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f72af80 0x56127f72d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.242+0000 7fd2b3400640 10 client.26825 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.242+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f72af80 0x56127f72d370 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b3ac216308b24c server_cookie=2b1c6ec682e84866 in_seq=0 out_seq=0 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2b3400640 1 -- 172.21.15.138:0/3249118505 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56127f686160 con 0x56127f72af80 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2b3400640 10 client.26825 handle_command_reply: tid=0 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 client.26825 shutdown 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 msgr2=0x56127f718560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80e1a70 tx=0x7fd2b80aab00 comp rx=0 tx=0).stop 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 shutdown_connections 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f72af80 0x56127f72d370 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f718aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f718560 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3249118505 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f718020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 >> 172.21.15.138:0/3249118505 conn(0x56127f70c600 msgr2=0x56127f7121a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2b3400640 0 client.26825 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 shutdown_connections 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.243+0000 7fd2d700d740 1 -- 172.21.15.138:0/3249118505 wait complete. 2024-08-20T01:19:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.244+0000 7fd2d700d740 20 client.26825 trim_cache size 0 max 16384 2024-08-20T01:19:26.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.258+0000 7fd2d700d740 1 Processor -- start 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.258+0000 7fd2d700d740 1 -- start start 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f712990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f732c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b87b0 con 0x56127f71b080 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b6050 con 0x56127f70f0f0 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d700d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b3ec0 con 0x56127f70e7d0 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59382/0 (socket says 172.21.15.138:59382) 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d2200640 1 -- 172.21.15.138:0/3861064774 learned_addr learned my addr 172.21.15.138:0/3861064774 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f732c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d1800640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f712990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3861064774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 msgr2=0x56127f712990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f712990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3861064774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 msgr2=0x56127f7331c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d0e00640 1 -- 172.21.15.138:0/3861064774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56127f44f620 con 0x56127f70f0f0 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.259+0000 7fd2d2200640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f732c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2b801f000 tx=0x7fd2b80ab0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ebd03009b4104c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2be000640 1 -- 172.21.15.138:0/3861064774 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80af530 con 0x56127f70f0f0 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2be000640 1 -- 172.21.15.138:0/3861064774 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b80a6b70 con 0x56127f70f0f0 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2be000640 1 -- 172.21.15.138:0/3861064774 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b80a71c0 con 0x56127f70f0f0 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 -- 172.21.15.138:0/3861064774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 msgr2=0x56127f732c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f732c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd2b801f000 tx=0x7fd2b80ab0b0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 -- 172.21.15.138:0/3861064774 shutdown_connections 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f7331c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70f0f0 0x56127f732c80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 --2- 172.21.15.138:0/3861064774 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70e7d0 0x56127f712990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.260+0000 7fd2d700d740 1 -- 172.21.15.138:0/3861064774 >> 172.21.15.138:0/3861064774 conn(0x56127f70c600 msgr2=0x56127f70db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 -- 172.21.15.138:0/3861064774 shutdown_connections 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 -- 172.21.15.138:0/3861064774 wait complete. 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 -- start start 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 Processor -- start 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 0x56127f71e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 0x56127f71eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.261+0000 7fd2d700d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f793c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d700d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b4510 con 0x56127f71b080 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d700d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b5980 con 0x56127f70e7d0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d700d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc1b4be0 con 0x56127f70f0f0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d700d740 10 client.0 _ll_register_callbacks cb 0x7fd2b8003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f793c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 0x56127f71e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 0x56127f71eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d2200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f793c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:59396/0 (socket says 172.21.15.138:59396) 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d2200640 1 -- 172.21.15.138:0/2711208000 learned_addr learned my addr 172.21.15.138:0/2711208000 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 -- 172.21.15.138:0/2711208000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 msgr2=0x56127f71eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 0x56127f71eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 -- 172.21.15.138:0/2711208000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 msgr2=0x56127f793c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f793c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 -- 172.21.15.138:0/2711208000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56127f794200 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 0x56127f71eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2d1800640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 0x56127f71e580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000c00 tx=0x7fd2c0003be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad31efef9e39a893 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.262+0000 7fd2b3400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0040040 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2b80a6b70 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7fd2c0050620 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd2c0035040 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2d700d740 10 client.26831 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56127f795570 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2b80a6b70 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.263+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7fd2c0057030 con 0x56127f70e7d0 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 10 client.26831 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 20 client.26831 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f7958b0 0x56127f797ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 4 client.26831 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56127f681c00 con 0x56127f7958b0 2024-08-20T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f7958b0 0x56127f797ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2d0e00640 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f7958b0 0x56127f797ca0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd33091e8e6ef359 server_cookie=86b7a10a687f4244 in_seq=0 out_seq=0 2024-08-20T01:19:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.264+0000 7fd2b3400640 10 client.26831 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.306+0000 7fd2b3400640 1 -- 172.21.15.138:0/2711208000 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56127f681c00 con 0x56127f7958b0 2024-08-20T01:19:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.306+0000 7fd2b3400640 10 client.26831 handle_command_reply: tid=0 2024-08-20T01:19:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.306+0000 7fd2d700d740 1 client.26831 shutdown 2024-08-20T01:19:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 msgr2=0x56127f71e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.305 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 0x56127f71e580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0000c00 tx=0x7fd2c0003be0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 shutdown_connections 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56127f7958b0 0x56127f797ca0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56127f71b080 0x56127f793c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56127f70f0f0 0x56127f71eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 --2- 172.21.15.138:0/2711208000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56127f70e7d0 0x56127f71e580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2b3400640 0 client.26831 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 >> 172.21.15.138:0/2711208000 conn(0x56127f70c600 msgr2=0x56127f70ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 shutdown_connections 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 1 -- 172.21.15.138:0/2711208000 wait complete. 2024-08-20T01:19:26.306 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.307+0000 7fd2d700d740 20 client.26831 trim_cache size 0 max 16384 2024-08-20T01:19:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:19:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:19:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:19:26.307 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 msgr2=0x7fd2b40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 0x7fd2b40b6200 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc076590 tx=0x7fd2c4009500 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 msgr2=0x7fd2cc19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc19f850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800e9f0 tx=0x7fd2b800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 shutdown_connections 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd2b40b3d40 0x7fd2b40b6200 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd2cc1133f0 0x7fd2cc1a4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd2cc077d20 0x7fd2cc19f850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 --2- 172.21.15.138:0/3066031633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd2cc077400 0x7fd2cc19f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 >> 172.21.15.138:0/3066031633 conn(0x7fd2cc100450 msgr2=0x7fd2cc102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:26.308 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.308+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 shutdown_connections 2024-08-20T01:19:26.309 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:26.309+0000 7fd2d2c00640 1 -- 172.21.15.138:0/3066031633 wait complete. 2024-08-20T01:19:27.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 449293 3.67 MB/sec execute 459 sec latency 1770.869 ms 2024-08-20T01:19:28.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 451035 3.69 MB/sec execute 460 sec latency 14.690 ms 2024-08-20T01:19:28.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:27 smithi167 ceph-mon[21965]: pgmap v713: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-08-20T01:19:28.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:27 smithi142 ceph-mon[21799]: pgmap v713: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-08-20T01:19:28.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:27 smithi138 ceph-mon[18076]: pgmap v713: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 140 op/s 2024-08-20T01:19:29.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 452187 3.69 MB/sec execute 461 sec latency 114.248 ms 2024-08-20T01:19:30.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 454045 3.71 MB/sec execute 462 sec latency 164.569 ms 2024-08-20T01:19:30.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:29 smithi167 ceph-mon[21965]: pgmap v714: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-20T01:19:30.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:29 smithi142 ceph-mon[21799]: pgmap v714: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-20T01:19:30.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:29 smithi138 ceph-mon[18076]: pgmap v714: 129 pgs: 129 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 105 op/s 2024-08-20T01:19:31.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 455269 3.71 MB/sec execute 463 sec latency 29.015 ms 2024-08-20T01:19:32.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 455730 3.71 MB/sec execute 464 sec latency 391.747 ms 2024-08-20T01:19:32.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:31 smithi167 ceph-mon[21965]: pgmap v715: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-08-20T01:19:32.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:31 smithi142 ceph-mon[21799]: pgmap v715: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-08-20T01:19:32.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:31 smithi138 ceph-mon[18076]: pgmap v715: 129 pgs: 129 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 1.9 MiB/s wr, 153 op/s 2024-08-20T01:19:33.160 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 23.3 2024-08-20T01:19:33.161 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:19:33.189 INFO:tasks.workunit.client.0.smithi138.stdout: 1 457677 3.73 MB/sec execute 465 sec latency 764.124 ms 2024-08-20T01:19:33.576 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:19:34.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:33 smithi138 ceph-mon[18076]: pgmap v716: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-08-20T01:19:34.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 458517 3.74 MB/sec execute 466 sec latency 48.556 ms 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 -- 172.21.15.138:0/522562619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788113660 msgr2=0x7f3788115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 --2- 172.21.15.138:0/522562619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788113660 0x7f3788115a50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3780009ca0 tx=0x7f378002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 -- 172.21.15.138:0/522562619 shutdown_connections 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 --2- 172.21.15.138:0/522562619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788113660 0x7f3788115a50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 --2- 172.21.15.138:0/522562619 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788075b90 0x7f3788113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 --2- 172.21.15.138:0/522562619 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075270 0x7f3788075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 -- 172.21.15.138:0/522562619 >> 172.21.15.138:0/522562619 conn(0x7f37880fe390 msgr2=0x7f37881007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:34.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 -- 172.21.15.138:0/522562619 shutdown_connections 2024-08-20T01:19:34.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.209+0000 7f378e400640 1 -- 172.21.15.138:0/522562619 wait complete. 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.210+0000 7f378e400640 1 Processor -- start 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 -- start start 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788075270 0x7f378819f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075b90 0x7f378819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3788118f30 con 0x7f3788075b90 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3788118db0 con 0x7f3788075270 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f37881190b0 con 0x7f3788113660 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59372/0 (socket says 172.21.15.138:59372) 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378da00640 1 -- 172.21.15.138:0/3784930394 learned_addr learned my addr 172.21.15.138:0/3784930394 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.211+0000 7f378d000640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788075270 0x7f378819f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:34.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f3787e00640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075b90 0x7f378819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:34.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 -- 172.21.15.138:0/3784930394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788075270 msgr2=0x7f378819f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788075270 0x7f378819f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 -- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075b90 msgr2=0x7f378819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075b90 0x7f378819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 -- 172.21.15.138:0/3784930394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881a53d0 con 0x7f3788113660 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f378da00640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3780005e30 tx=0x7f37800317c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.212+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3780003b70 con 0x7f3788113660 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.213+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3780003d10 con 0x7f3788113660 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.213+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881a5660 con 0x7f3788113660 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.213+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378002f970 con 0x7f3788113660 2024-08-20T01:19:34.213 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.213+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881abd80 con 0x7f3788113660 2024-08-20T01:19:34.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:33 smithi167 ceph-mon[21965]: pgmap v716: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-08-20T01:19:34.214 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.214+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3788075650 con 0x7f3788113660 2024-08-20T01:19:34.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.216+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3780004b40 con 0x7f3788113660 2024-08-20T01:19:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.218+0000 7f3785600640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 0x7f37700b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.218+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f37801057b0 con 0x7f3788113660 2024-08-20T01:19:34.217 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.218+0000 7f378d000640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 0x7f37700b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:34.218 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.219+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f37800cc7d0 con 0x7f3788113660 2024-08-20T01:19:34.219 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.220+0000 7f378d000640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 0x7f37700b6320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f377c009b60 tx=0x7f377c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:34.240 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:33 smithi142 ceph-mon[21799]: pgmap v716: 129 pgs: 129 active+clean; 347 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-08-20T01:19:34.489 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.490+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37881a0450 con 0x7f3788113660 2024-08-20T01:19:34.490 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.490+0000 7f3785600640 1 -- 172.21.15.138:0/3784930394 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11933 (secure 0 0 0) 0x7f37881a0450 con 0x7f3788113660 2024-08-20T01:19:34.490 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:19:34.490 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":40,"btime":"2024-08-20T01:17:24:278890+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:17:24.228659+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:19:34.491 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 40 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 msgr2=0x7f37700b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 0x7f37700b6320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f377c009b60 tx=0x7f377c005f90 comp rx=0 tx=0).stop 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 msgr2=0x7f37881a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3780005e30 tx=0x7f37800317c0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 shutdown_connections 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f37700b3e60 0x7f37700b6320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.493 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3788113660 0x7f37881a4c00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3788075b90 0x7f378819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 --2- 172.21.15.138:0/3784930394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3788075270 0x7f378819f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 >> 172.21.15.138:0/3784930394 conn(0x7f37880fe390 msgr2=0x7f37880fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 shutdown_connections 2024-08-20T01:19:34.494 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:34.494+0000 7f378e400640 1 -- 172.21.15.138:0/3784930394 wait complete. 2024-08-20T01:19:35.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 459997 3.77 MB/sec execute 467 sec latency 52.970 ms 2024-08-20T01:19:35.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:34 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3784930394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:35.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:34 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3784930394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:35.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:34 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3784930394' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:35.415 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 143.3 --await-for 45.0' 2024-08-20T01:19:35.416 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 143.3 --await-for 45.0 2024-08-20T01:19:35.826 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:19:36.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460030 3.76 MB/sec execute 468 sec latency 977.944 ms 2024-08-20T01:19:36.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:35 smithi167 ceph-mon[21965]: pgmap v717: 129 pgs: 129 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-08-20T01:19:36.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:35 smithi142 ceph-mon[21799]: pgmap v717: 129 pgs: 129 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-08-20T01:19:36.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:35 smithi138 ceph-mon[18076]: pgmap v717: 129 pgs: 129 active+clean; 352 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.6 MiB/s wr, 210 op/s 2024-08-20T01:19:36.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.723+0000 7f0a2aa00640 1 -- 172.21.15.138:0/4143561330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108ca0 msgr2=0x7f0a2410f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.723+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/4143561330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108ca0 0x7f0a2410f4a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009be0 tx=0x7f0a1c02a6c0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 -- 172.21.15.138:0/4143561330 shutdown_connections 2024-08-20T01:19:36.723 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/4143561330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108ca0 0x7f0a2410f4a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/4143561330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24108320 0x7f0a24108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/4143561330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24102910 0x7f0a24102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 -- 172.21.15.138:0/4143561330 >> 172.21.15.138:0/4143561330 conn(0x7f0a240fe3c0 msgr2=0x7f0a241007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:36.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 -- 172.21.15.138:0/4143561330 shutdown_connections 2024-08-20T01:19:36.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.724+0000 7f0a2aa00640 1 -- 172.21.15.138:0/4143561330 wait complete. 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.725+0000 7f0a2aa00640 1 Processor -- start 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.725+0000 7f0a2aa00640 1 -- start start 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.725+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 0x7f0a24075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2aa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24108ca0 0x7f0a24075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112920 con 0x7f0a24108ca0 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241127a0 con 0x7f0a24108320 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2aa00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a24112aa0 con 0x7f0a24102910 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a29600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a29600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59382/0 (socket says 172.21.15.138:59382) 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a29600640 1 -- 172.21.15.138:0/3652707991 learned_addr learned my addr 172.21.15.138:0/3652707991 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24108ca0 0x7f0a24075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.726+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 0x7f0a24075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 -- 172.21.15.138:0/3652707991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 msgr2=0x7f0a24075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 0x7f0a24075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 -- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24108ca0 msgr2=0x7f0a24075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24108ca0 0x7f0a24075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.727 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 -- 172.21.15.138:0/3652707991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a241a7110 con 0x7f0a24102910 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 0x7f0a24075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a29600640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800e9c0 tx=0x7f0a1800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a22000640 1 -- 172.21.15.138:0/3652707991 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800cde0 con 0x7f0a24102910 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a22000640 1 -- 172.21.15.138:0/3652707991 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a180035f0 con 0x7f0a24102910 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a241a7400 con 0x7f0a24102910 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.727+0000 7f0a22000640 1 -- 172.21.15.138:0/3652707991 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a18010690 con 0x7f0a24102910 2024-08-20T01:19:36.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.728+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a241a7960 con 0x7f0a24102910 2024-08-20T01:19:36.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 Processor -- start 2024-08-20T01:19:36.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 -- start start 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a22000640 1 -- 172.21.15.138:0/3652707991 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0a18004000 con 0x7f0a24102910 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcae39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaf2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606bca83670 con 0x5606bcaefe80 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5606bca577c0 con 0x5606bcae3ef0 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.731+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606bca46cc0 con 0x5606bcae35d0 2024-08-20T01:19:36.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:59060/0 (socket says 172.21.15.138:59060) 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 -- 172.21.15.138:0/3839972266 learned_addr learned my addr 172.21.15.138:0/3839972266 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcae39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaf2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 -- 172.21.15.138:0/3839972266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 msgr2=0x5606bcae39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcae39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 -- 172.21.15.138:0/3839972266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 msgr2=0x5606bcaf2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaf2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.732+0000 7f0a29600640 1 -- 172.21.15.138:0/3839972266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bc9da110 con 0x5606bcae3ef0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a29600640 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a180e19c0 tx=0x7f0a180eaef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8037782cf289d16c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a16600640 1 -- 172.21.15.138:0/3839972266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180f3070 con 0x5606bcae3ef0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a16600640 1 -- 172.21.15.138:0/3839972266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a180e60d0 con 0x5606bcae3ef0 2024-08-20T01:19:36.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a16600640 1 -- 172.21.15.138:0/3839972266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180ef680 con 0x5606bcae3ef0 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a22000640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 0x7f0a0c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.733+0000 7f0a22000640 1 -- 172.21.15.138:0/3652707991 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f0a180e4730 con 0x7f0a24102910 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3839972266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 msgr2=0x5606bcaef940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0a180e19c0 tx=0x7f0a180eaef0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 0x7f0a0c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3839972266 shutdown_connections 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaf2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae3ef0 0x5606bcaef940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3839972266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcae39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3839972266 >> 172.21.15.138:0/3839972266 conn(0x5606bcae1400 msgr2=0x5606bcae25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3839972266 shutdown_connections 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.734+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3839972266 wait complete. 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 -- start start 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 0x7f0a0c0b6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c004af0 tx=0x7f0a1c005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 Processor -- start 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 0x5606bcaeced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5606bc9ec780 con 0x5606bcae35d0 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5606bca4ad00 con 0x5606bcaefe80 2024-08-20T01:19:36.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5606bc8a97a0 con 0x5606bcae3ef0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.735+0000 7f0a2eab6740 10 client.0 _ll_register_callbacks cb 0x5606bc9f4940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59400/0 (socket says 172.21.15.138:59400) 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:59072/0 (socket says 172.21.15.138:59072) 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 -- 172.21.15.138:0/4159336539 learned_addr learned my addr 172.21.15.138:0/4159336539 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a29600640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 0x5606bcaeced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 -- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 msgr2=0x5606bcaed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 -- 172.21.15.138:0/4159336539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 msgr2=0x5606bcaeced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 0x5606bcaeced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 -- 172.21.15.138:0/4159336539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bc9f99c0 con 0x5606bcae3ef0 2024-08-20T01:19:36.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a2a000640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a29600640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 0x5606bcaeced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a28c00640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a100126a0 tx=0x7f0a10012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adc3681838c31e5d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.736+0000 7f0a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1001f060 con 0x5606bcae3ef0 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a180e60d0 con 0x5606bcae3ef0 2024-08-20T01:19:36.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f0a10010d60 con 0x5606bcae3ef0 2024-08-20T01:19:36.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0a10026050 con 0x5606bcae3ef0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a2eab6740 10 client.26800 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606bc7b5b90 con 0x5606bcae3ef0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.737+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a180e60d0 con 0x5606bcae3ef0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f0a10019030 con 0x5606bcae3ef0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 10 client.26800 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 10 client.26800 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 20 client.26800 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcaff950 0x5606bcb01df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 4 client.26800 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606bca59710 con 0x5606bcaff950 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.738+0000 7f0a29600640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcaff950 0x5606bcb01df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.739+0000 7f0a0aa00640 10 client.26800 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.739+0000 7f0a29600640 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcaff950 0x5606bcb01df0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ef6b742f31a0cd8 server_cookie=18a9f61c1a11d30b in_seq=0 out_seq=0 2024-08-20T01:19:36.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.739+0000 7f0a0aa00640 1 -- 172.21.15.138:0/4159336539 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f0a1808be80 con 0x5606bcaff950 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.739+0000 7f0a0aa00640 10 client.26800 handle_command_reply: tid=0 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 client.26800 shutdown 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 msgr2=0x5606bcaed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0a100126a0 tx=0x7f0a10012b60 comp rx=0 tx=0).stop 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 shutdown_connections 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcaff950 0x5606bcb01df0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcaed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcaed410 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a0aa00640 0 client.26800 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/4159336539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae35d0 0x5606bcaeced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 >> 172.21.15.138:0/4159336539 conn(0x5606bcae1400 msgr2=0x5606bcae6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:36.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 shutdown_connections 2024-08-20T01:19:36.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 1 -- 172.21.15.138:0/4159336539 wait complete. 2024-08-20T01:19:36.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.740+0000 7f0a2eab6740 20 client.26800 trim_cache size 0 max 16384 2024-08-20T01:19:36.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.758+0000 7f0a2eab6740 1 Processor -- start 2024-08-20T01:19:36.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.758+0000 7f0a2eab6740 1 -- start start 2024-08-20T01:19:36.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcb07c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaea0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b4490 con 0x5606bcaefe80 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b1d30 con 0x5606bcae35d0 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241afba0 con 0x5606bcae3ef0 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaea0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a29600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2a000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaea0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44460/0 (socket says 172.21.15.138:44460) 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a29600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:59082/0 (socket says 172.21.15.138:59082) 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a2a000640 1 -- 172.21.15.138:0/3438992181 learned_addr learned my addr 172.21.15.138:0/3438992181 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:36.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcb07c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 -- 172.21.15.138:0/3438992181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 msgr2=0x5606bcb07700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 -- 172.21.15.138:0/3438992181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 msgr2=0x5606bcaea0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaea0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a28c00640 1 -- 172.21.15.138:0/3438992181 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bcae20e0 con 0x5606bcae3ef0 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.759+0000 7f0a29600640 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.760+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcb07c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a1000d3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29c11369debf88c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.760+0000 7f0a15c00640 1 -- 172.21.15.138:0/3438992181 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10005710 con 0x5606bcae3ef0 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.760+0000 7f0a15c00640 1 -- 172.21.15.138:0/3438992181 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a100054e0 con 0x5606bcae3ef0 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.760+0000 7f0a15c00640 1 -- 172.21.15.138:0/3438992181 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10010dc0 con 0x5606bcae3ef0 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3438992181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 msgr2=0x5606bcb07c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcb07c40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a1000d3c0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3438992181 shutdown_connections 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcaefe80 0x5606bcaea0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae3ef0 0x5606bcb07c40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3438992181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcae35d0 0x5606bcb07700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3438992181 >> 172.21.15.138:0/3438992181 conn(0x5606bcae1400 msgr2=0x5606bcaee500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:36.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3438992181 shutdown_connections 2024-08-20T01:19:36.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.761+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3438992181 wait complete. 2024-08-20T01:19:36.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.762+0000 7f0a2eab6740 1 -- start start 2024-08-20T01:19:36.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.762+0000 7f0a2eab6740 1 Processor -- start 2024-08-20T01:19:36.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae3ef0 0x5606bcaf4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcb696d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b01f0 con 0x5606bcae3ef0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b1660 con 0x5606bcaefe80 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0a241b08c0 con 0x5606bcae35d0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2eab6740 10 client.0 _ll_register_callbacks cb 0x7f0a10009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59416/0 (socket says 172.21.15.138:59416) 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 -- 172.21.15.138:0/3471676171 learned_addr learned my addr 172.21.15.138:0/3471676171 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a2a000640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcb696d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae3ef0 0x5606bcaf4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 -- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 msgr2=0x5606bcb696d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcb696d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 -- 172.21.15.138:0/3471676171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae3ef0 msgr2=0x5606bcaf4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.763+0000 7f0a29600640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae3ef0 0x5606bcaf4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a29600640 1 -- 172.21.15.138:0/3471676171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bcb69c10 con 0x5606bcae35d0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a29600640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a180ec730 tx=0x7f0a180046e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3e21f24156ff4c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a0aa00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a180f3070 con 0x5606bcae35d0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a100054e0 con 0x5606bcae35d0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f0a180a7530 con 0x5606bcae35d0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f0a180a6e00 con 0x5606bcae35d0 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a2eab6740 10 client.26806 fetch_fsmap learned FSMap version 40 2024-08-20T01:19:36.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.764+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5606bcb6af20 con 0x5606bcae35d0 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a100054e0 con 0x5606bcae35d0 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 40) ==== 8358+0+0 (secure 0 0 0) 0x7f0a180a71d0 con 0x5606bcae35d0 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 10 client.26806 fetch_fsmap finished waiting for FSMap version 40 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 10 client.26806 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 20 client.26806 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:36.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcb6b390 0x5606bcb6d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:36.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 4 client.26806 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.3, "await_for": 45.0}] 2024-08-20T01:19:36.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 143.3, "await_for": 45.0}) -- 0x5606bca551b0 con 0x5606bcb6b390 2024-08-20T01:19:36.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.765+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcb6b390 0x5606bcb6d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:36.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.766+0000 7f0a28c00640 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcb6b390 0x5606bcb6d780 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a72365ab70f6e0 server_cookie=a6f21096207b59a in_seq=0 out_seq=0 2024-08-20T01:19:36.767 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:36.766+0000 7f0a0aa00640 10 client.26806 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:37.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.75 MB/sec execute 469 sec latency 1736.480 ms 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a0aa00640 1 -- 172.21.15.138:0/3471676171 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5606bca551b0 con 0x5606bcb6b390 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a0aa00640 10 client.26806 handle_command_reply: tid=0 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 client.26806 shutdown 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 msgr2=0x5606bcaf3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a180ec730 tx=0x7f0a180046e0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 shutdown_connections 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5606bcb6b390 0x5606bcb6d780 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.254 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606bcaefe80 0x5606bcb696d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5606bcae3ef0 0x5606bcaf4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a0aa00640 0 client.26806 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 --2- 172.21.15.138:0/3471676171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5606bcae35d0 0x5606bcaf3fd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 >> 172.21.15.138:0/3471676171 conn(0x5606bcae1400 msgr2=0x5606bcaece60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.254+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 shutdown_connections 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.255+0000 7f0a2eab6740 1 -- 172.21.15.138:0/3471676171 wait complete. 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.255+0000 7f0a2eab6740 20 client.26806 trim_cache size 0 max 16384 2024-08-20T01:19:37.255 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 40, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 42, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "248f5e2c": { 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 42, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:19:37.256 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 143.3, 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:37.257 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.255+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 msgr2=0x7f0a0c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.255+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 0x7f0a0c0b6320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c004af0 tx=0x7f0a1c005fd0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 msgr2=0x7f0a240751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0a1800e9c0 tx=0x7f0a1800ee80 comp rx=0 tx=0).stop 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 shutdown_connections 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0a0c0b3e60 0x7f0a0c0b6320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.258 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0a24108ca0 0x7f0a24075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a24108320 0x7f0a24075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 --2- 172.21.15.138:0/3652707991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0a24102910 0x7f0a240751d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 >> 172.21.15.138:0/3652707991 conn(0x7f0a240fe3c0 msgr2=0x7f0a241007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 shutdown_connections 2024-08-20T01:19:37.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:37.256+0000 7f0a2aa00640 1 -- 172.21.15.138:0/3652707991 wait complete. 2024-08-20T01:19:38.007 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:37 smithi138 ceph-mon[18076]: pgmap v718: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-08-20T01:19:38.044 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '248f5e2c', quiesce took 4.9 seconds. Will release after: 18.4 2024-08-20T01:19:38.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.75 MB/sec execute 470 sec latency 1233.820 ms 2024-08-20T01:19:38.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:37 smithi167 ceph-mon[21965]: pgmap v718: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-08-20T01:19:38.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:37 smithi142 ceph-mon[21799]: pgmap v718: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.3 MiB/s wr, 187 op/s 2024-08-20T01:19:39.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.74 MB/sec execute 471 sec latency 2233.945 ms 2024-08-20T01:19:39.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:38 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:19:39.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:38 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:19:39.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:38 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 -- 172.21.15.138:0/520638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 msgr2=0x7f18a4111780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 --2- 172.21.15.138:0/520638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a4111780 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f189c009ca0 tx=0x7f189c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 -- 172.21.15.138:0/520638182 shutdown_connections 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 --2- 172.21.15.138:0/520638182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a4111780 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 --2- 172.21.15.138:0/520638182 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 0x7f18a410aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 --2- 172.21.15.138:0/520638182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a4104fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 -- 172.21.15.138:0/520638182 >> 172.21.15.138:0/520638182 conn(0x7f18a4100600 msgr2=0x7f18a4102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 -- 172.21.15.138:0/520638182 shutdown_connections 2024-08-20T01:19:39.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.431+0000 7f18aa000640 1 -- 172.21.15.138:0/520638182 wait complete. 2024-08-20T01:19:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.432+0000 7f18aa000640 1 Processor -- start 2024-08-20T01:19:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.432+0000 7f18aa000640 1 -- start start 2024-08-20T01:19:39.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a419b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 0x7f18a419b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a41a09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4114c90 con 0x7f18a410af80 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4114b10 con 0x7f18a4104c00 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18aa000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18a4114e10 con 0x7f18a410a600 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a41a09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a419b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a41a09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51336/0 (socket says 172.21.15.138:51336) 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3e00640 1 -- 172.21.15.138:0/1718358764 learned_addr learned my addr 172.21.15.138:0/1718358764 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a2a00640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 0x7f18a419b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 -- 172.21.15.138:0/1718358764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 msgr2=0x7f18a419b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 0x7f18a419b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 -- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 msgr2=0x7f18a41a09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a41a09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.433+0000 7f18a3400640 1 -- 172.21.15.138:0/1718358764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a41a1190 con 0x7f18a4104c00 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f18a3400640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a419b0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f189400e9f0 tx=0x7f189400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f189be00640 1 -- 172.21.15.138:0/1718358764 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189400ccc0 con 0x7f18a4104c00 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a41a1480 con 0x7f18a4104c00 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f189be00640 1 -- 172.21.15.138:0/1718358764 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18940108e0 con 0x7f18a4104c00 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f189be00640 1 -- 172.21.15.138:0/1718358764 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894018880 con 0x7f18a4104c00 2024-08-20T01:19:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.434+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a41a7b40 con 0x7f18a4104c00 2024-08-20T01:19:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.437+0000 7f18ace62740 1 Processor -- start 2024-08-20T01:19:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.437+0000 7f18ace62740 1 -- start start 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.437+0000 7f18ace62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.437+0000 7f18ace62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b517940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18ace62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b51a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18ace62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b481210 con 0x55eb7b517e80 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18ace62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b470710 con 0x55eb7b50bef0 2024-08-20T01:19:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18ace62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b474750 con 0x55eb7b50b5d0 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f189be00640 1 -- 172.21.15.138:0/1718358764 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1894010430 con 0x7f18a4104c00 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b517940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b51a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b517940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39936/0 (socket says 172.21.15.138:39936) 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3400640 1 -- 172.21.15.138:0/2344392982 learned_addr learned my addr 172.21.15.138:0/2344392982 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 -- 172.21.15.138:0/2344392982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 msgr2=0x55eb7b517940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b517940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 -- 172.21.15.138:0/2344392982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 msgr2=0x55eb7b51a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b51a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.438+0000 7f18a3e00640 1 -- 172.21.15.138:0/2344392982 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7b43cde0 con 0x55eb7b50b5d0 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f18a3e00640 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50b9b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f189c006010 tx=0x7f189c009a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=364572f0b9e547a2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f188fe00640 1 -- 172.21.15.138:0/2344392982 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c031cf0 con 0x55eb7b50b5d0 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f188fe00640 1 -- 172.21.15.138:0/2344392982 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f189c036030 con 0x55eb7b50b5d0 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f188fe00640 1 -- 172.21.15.138:0/2344392982 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0042a0 con 0x55eb7b50b5d0 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f189be00640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 0x7f18880b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f18ace62740 1 -- 172.21.15.138:0/2344392982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 msgr2=0x55eb7b50b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.439+0000 7f18ace62740 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50b9b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f189c006010 tx=0x7f189c009a10 comp rx=0 tx=0).stop 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f189be00640 1 -- 172.21.15.138:0/1718358764 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f18940e4780 con 0x7f18a4104c00 2024-08-20T01:19:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18a3400640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 0x7f18880b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 -- 172.21.15.138:0/2344392982 shutdown_connections 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b51a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b517940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 --2- 172.21.15.138:0/2344392982 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50b9b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 -- 172.21.15.138:0/2344392982 >> 172.21.15.138:0/2344392982 conn(0x55eb7b5093b0 msgr2=0x55eb7b50a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 -- 172.21.15.138:0/2344392982 shutdown_connections 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 -- 172.21.15.138:0/2344392982 wait complete. 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18a3400640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 0x7f18880b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f18940e4a20 tx=0x7f18940ea530 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 -- start start 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.440+0000 7f18ace62740 1 Processor -- start 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b517900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b5154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b3a9ab0 con 0x55eb7b517e80 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b417140 con 0x55eb7b50bef0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55eb7b51a980 con 0x55eb7b50b5d0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18ace62740 10 client.0 _ll_register_callbacks cb 0x55eb7b42a5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39944/0 (socket says 172.21.15.138:39944) 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a2a00640 1 -- 172.21.15.138:0/4160556786 learned_addr learned my addr 172.21.15.138:0/4160556786 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a3e00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b517900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a3e00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b5154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a2a00640 1 -- 172.21.15.138:0/4160556786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 msgr2=0x55eb7b517900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.441+0000 7f18a2a00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b517900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18a2a00640 1 -- 172.21.15.138:0/4160556786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 msgr2=0x55eb7b5154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18a2a00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b5154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18a2a00640 1 -- 172.21.15.138:0/4160556786 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7b418170 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18a3e00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b5154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18a2a00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1890012740 tx=0x7f1890012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4faf7d0655b5cfb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f188cc00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000f920 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb7b447a30 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f189000fac0 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f189000fcc0 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18ace62740 10 client.26864 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb7b1d3790 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.442+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55eb7b447a30 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f189001c420 con 0x55eb7b50bef0 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 10 client.26864 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 10 client.26864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 20 client.26864 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b527c30 0x55eb7b52a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 4 client.26864 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb7b483160 con 0x55eb7b527c30 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18a3e00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b527c30 0x55eb7b52a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f18a3e00640 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b527c30 0x55eb7b52a020 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb1fa33fc7dc0da server_cookie=7d86041ddcda53ac in_seq=0 out_seq=0 2024-08-20T01:19:39.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.443+0000 7f188cc00640 10 client.26864 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.444+0000 7f188cc00640 1 -- 172.21.15.138:0/4160556786 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb7b483160 con 0x55eb7b527c30 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.444+0000 7f188cc00640 10 client.26864 handle_command_reply: tid=0 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.444+0000 7f18ace62740 1 client.26864 shutdown 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.444+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 msgr2=0x55eb7b514d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.444+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1890012740 tx=0x7f1890012c00 comp rx=0 tx=0).stop 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 shutdown_connections 2024-08-20T01:19:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b527c30 0x55eb7b52a020 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b517e80 0x55eb7b5154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50bef0 0x55eb7b514d40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 --2- 172.21.15.138:0/4160556786 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b517900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 >> 172.21.15.138:0/4160556786 conn(0x55eb7b5093b0 msgr2=0x55eb7b50efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f188cc00640 0 client.26864 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 shutdown_connections 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 1 -- 172.21.15.138:0/4160556786 wait complete. 2024-08-20T01:19:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.445+0000 7f18ace62740 20 client.26864 trim_cache size 0 max 16384 2024-08-20T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 Processor -- start 2024-08-20T01:19:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 -- start start 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b517e80 0x55eb7b52fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41b4630 con 0x55eb7b50bef0 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41b1ea0 con 0x55eb7b517e80 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18ace62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41afd10 con 0x55eb7b50b5d0 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51364/0 (socket says 172.21.15.138:51364) 2024-08-20T01:19:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.460+0000 7f18a2a00640 1 -- 172.21.15.138:0/3185216518 learned_addr learned my addr 172.21.15.138:0/3185216518 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a3e00640 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b517e80 0x55eb7b52fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 -- 172.21.15.138:0/3185216518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 msgr2=0x55eb7b50ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 -- 172.21.15.138:0/3185216518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b517e80 msgr2=0x55eb7b52fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b517e80 0x55eb7b52fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 -- 172.21.15.138:0/3185216518 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb7b50bd20 con 0x55eb7b50bef0 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f18a2a00640 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1890009e60 tx=0x7f1890009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f38936a79ca7df6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f188f400640 1 -- 172.21.15.138:0/3185216518 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890005d30 con 0x55eb7b50bef0 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f188f400640 1 -- 172.21.15.138:0/3185216518 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1890012e50 con 0x55eb7b50bef0 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.461+0000 7f188f400640 1 -- 172.21.15.138:0/3185216518 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189001c680 con 0x55eb7b50bef0 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- 172.21.15.138:0/3185216518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 msgr2=0x55eb7b52f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1890009e60 tx=0x7f1890009f00 comp rx=0 tx=0).stop 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- 172.21.15.138:0/3185216518 shutdown_connections 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b517e80 0x55eb7b52fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b52f8f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 --2- 172.21.15.138:0/3185216518 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b50b5d0 0x55eb7b50ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- 172.21.15.138:0/3185216518 >> 172.21.15.138:0/3185216518 conn(0x55eb7b5093b0 msgr2=0x55eb7b50a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- 172.21.15.138:0/3185216518 shutdown_connections 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- 172.21.15.138:0/3185216518 wait complete. 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.462+0000 7f18ace62740 1 -- start start 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 Processor -- start 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b520150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b517e80 0x55eb7b5110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41b0360 con 0x55eb7b50bef0 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41b17d0 con 0x55eb7b50b5d0 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f18a41b0a30 con 0x55eb7b517e80 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18ace62740 10 client.0 _ll_register_callbacks cb 0x7f1890009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b520150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18a3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:39960/0 (socket says 172.21.15.138:39960) 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18a3e00640 1 -- 172.21.15.138:0/550115014 learned_addr learned my addr 172.21.15.138:0/550115014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.463+0000 7f18a3e00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b517e80 0x55eb7b5110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 -- 172.21.15.138:0/550115014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b517e80 msgr2=0x55eb7b5110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b517e80 0x55eb7b5110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 -- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 msgr2=0x55eb7b51fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 -- 172.21.15.138:0/550115014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7b24c620 con 0x55eb7b50bef0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a3e00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18a2a00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b520150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f189000a010 tx=0x7f189001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11455db37d02510 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f188cc00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:19:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890005110 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f18900052b0 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1890012e50 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.464+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1890038050 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 10 client.16824 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55eb7b591db0 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1890012e50 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f189002a450 con 0x55eb7b50bef0 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 10 client.16824 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 20 client.16824 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b5924c0 0x55eb7b5948e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 4 client.16824 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:19:39.468 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb7b47ec00 con 0x55eb7b5924c0 2024-08-20T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.465+0000 7f18a3e00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b5924c0 0x55eb7b5948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.466+0000 7f18a3e00640 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b5924c0 0x55eb7b5948e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77538fe44102290 server_cookie=737377bcf5a8124d in_seq=0 out_seq=0 2024-08-20T01:19:39.469 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.466+0000 7f188cc00640 10 client.16824 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:39.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f188cc00640 1 -- 172.21.15.138:0/550115014 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb7b47ec00 con 0x55eb7b5924c0 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f188cc00640 10 client.16824 handle_command_reply: tid=0 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 client.16824 shutdown 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 msgr2=0x55eb7b520150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b520150 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f189000a010 tx=0x7f189001ca80 comp rx=0 tx=0).stop 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 shutdown_connections 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55eb7b5924c0 0x55eb7b5948e0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55eb7b517e80 0x55eb7b5110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55eb7b50bef0 0x55eb7b520150 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f188cc00640 0 client.16824 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 --2- 172.21.15.138:0/550115014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eb7b50b5d0 0x55eb7b51fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 >> 172.21.15.138:0/550115014 conn(0x55eb7b5093b0 msgr2=0x55eb7b514a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 shutdown_connections 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 1 -- 172.21.15.138:0/550115014 wait complete. 2024-08-20T01:19:39.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.512+0000 7f18ace62740 20 client.16824 trim_cache size 0 max 16384 2024-08-20T01:19:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:19:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:19:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:19:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:19:39.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 msgr2=0x7f18880b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 0x7f18880b6200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f18940e4a20 tx=0x7f18940ea530 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 msgr2=0x7f18a419b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a419b0f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f189400e9f0 tx=0x7f189400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 shutdown_connections 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f18880b3d40 0x7f18880b6200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18a410af80 0x7f18a41a09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f18a410a600 0x7f18a419b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 --2- 172.21.15.138:0/1718358764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18a4104c00 0x7f18a419b0f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 >> 172.21.15.138:0/1718358764 conn(0x7f18a4100600 msgr2=0x7f18a41029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 shutdown_connections 2024-08-20T01:19:39.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:39.513+0000 7f18aa000640 1 -- 172.21.15.138:0/1718358764 wait complete. 2024-08-20T01:19:40.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.73 MB/sec execute 472 sec latency 3234.106 ms 2024-08-20T01:19:40.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:39 smithi167 ceph-mon[21965]: pgmap v719: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-08-20T01:19:40.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:39 smithi142 ceph-mon[21799]: pgmap v719: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-08-20T01:19:40.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:39 smithi138 ceph-mon[18076]: pgmap v719: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.0 MiB/s wr, 169 op/s 2024-08-20T01:19:41.190 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.72 MB/sec execute 473 sec latency 4234.207 ms 2024-08-20T01:19:41.989 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:41 smithi138 ceph-mon[18076]: pgmap v720: 129 pgs: 129 active+clean; 357 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-20T01:19:42.197 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.71 MB/sec execute 474 sec latency 5240.828 ms 2024-08-20T01:19:42.212 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:41 smithi167 ceph-mon[21965]: pgmap v720: 129 pgs: 129 active+clean; 357 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-20T01:19:42.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:41 smithi142 ceph-mon[21799]: pgmap v720: 129 pgs: 129 active+clean; 357 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.3 MiB/s wr, 199 op/s 2024-08-20T01:19:43.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.71 MB/sec execute 475 sec latency 6240.875 ms 2024-08-20T01:19:44.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.70 MB/sec execute 476 sec latency 7241.655 ms 2024-08-20T01:19:44.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:43 smithi167 ceph-mon[21965]: pgmap v721: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-08-20T01:19:44.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:43 smithi142 ceph-mon[21799]: pgmap v721: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-08-20T01:19:44.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:43 smithi138 ceph-mon[18076]: pgmap v721: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.0 MiB/s wr, 161 op/s 2024-08-20T01:19:45.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.69 MB/sec execute 477 sec latency 8241.721 ms 2024-08-20T01:19:46.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:45 smithi138 ceph-mon[18076]: pgmap v722: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-08-20T01:19:46.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.68 MB/sec execute 478 sec latency 9241.790 ms 2024-08-20T01:19:46.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:45 smithi167 ceph-mon[21965]: pgmap v722: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-08-20T01:19:46.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:45 smithi142 ceph-mon[21799]: pgmap v722: 129 pgs: 129 active+clean; 356 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.3 MiB/s wr, 152 op/s 2024-08-20T01:19:47.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.68 MB/sec execute 479 sec latency 10241.849 ms 2024-08-20T01:19:48.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.67 MB/sec execute 480 sec latency 11241.907 ms 2024-08-20T01:19:48.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:47 smithi167 ceph-mon[21965]: pgmap v723: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-08-20T01:19:48.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:47 smithi142 ceph-mon[21799]: pgmap v723: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-08-20T01:19:48.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:47 smithi138 ceph-mon[18076]: pgmap v723: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.2 MiB/s wr, 150 op/s 2024-08-20T01:19:49.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.66 MB/sec execute 481 sec latency 12242.007 ms 2024-08-20T01:19:50.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.65 MB/sec execute 482 sec latency 13242.135 ms 2024-08-20T01:19:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:49 smithi167 ceph-mon[21965]: pgmap v724: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 2.8 MiB/s wr, 120 op/s 2024-08-20T01:19:50.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:49 smithi142 ceph-mon[21799]: pgmap v724: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 2.8 MiB/s wr, 120 op/s 2024-08-20T01:19:50.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:49 smithi138 ceph-mon[18076]: pgmap v724: 129 pgs: 129 active+clean; 361 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 2.8 MiB/s wr, 120 op/s 2024-08-20T01:19:51.198 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.65 MB/sec execute 483 sec latency 14242.236 ms 2024-08-20T01:19:52.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.64 MB/sec execute 484 sec latency 15242.366 ms 2024-08-20T01:19:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:51 smithi167 ceph-mon[21965]: pgmap v725: 129 pgs: 129 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.6 MiB/s wr, 130 op/s 2024-08-20T01:19:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:51 smithi142 ceph-mon[21799]: pgmap v725: 129 pgs: 129 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.6 MiB/s wr, 130 op/s 2024-08-20T01:19:52.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:51 smithi138 ceph-mon[18076]: pgmap v725: 129 pgs: 129 active+clean; 370 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 3.6 MiB/s wr, 130 op/s 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 -- 172.21.15.138:0/2097880046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108b80 msgr2=0x7f122810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 --2- 172.21.15.138:0/2097880046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108b80 0x7f122810f380 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f122002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 -- 172.21.15.138:0/2097880046 shutdown_connections 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 --2- 172.21.15.138:0/2097880046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108b80 0x7f122810f380 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 --2- 172.21.15.138:0/2097880046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228108200 0x7f1228108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 --2- 172.21.15.138:0/2097880046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228102800 0x7f1228102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.639+0000 7f122ec00640 1 -- 172.21.15.138:0/2097880046 >> 172.21.15.138:0/2097880046 conn(0x7f12280fe280 msgr2=0x7f12281006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.640+0000 7f122ec00640 1 -- 172.21.15.138:0/2097880046 shutdown_connections 2024-08-20T01:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.640+0000 7f122ec00640 1 -- 172.21.15.138:0/2097880046 wait complete. 2024-08-20T01:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 Processor -- start 2024-08-20T01:19:52.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 -- start start 2024-08-20T01:19:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 0x7f122819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108b80 0x7f12281a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1228112890 con 0x7f1228108200 2024-08-20T01:19:52.642 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1228112710 con 0x7f1228102800 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1228112a10 con 0x7f1228108b80 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46928/0 (socket says 172.21.15.138:46928) 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122ce00640 1 -- 172.21.15.138:0/1707515523 learned_addr learned my addr 172.21.15.138:0/1707515523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 0x7f122819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.641+0000 7f122e200640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108b80 0x7f12281a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 -- 172.21.15.138:0/1707515523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108b80 msgr2=0x7f12281a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108b80 0x7f12281a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 -- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 msgr2=0x7f122819b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 -- 172.21.15.138:0/1707515523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12281a0fc0 con 0x7f1228102800 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122ce00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 0x7f122819af20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9f0 tx=0x7f121c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.643 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f1226000640 1 -- 172.21.15.138:0/1707515523 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c00ccc0 con 0x7f1228102800 2024-08-20T01:19:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12281a12b0 con 0x7f1228102800 2024-08-20T01:19:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f1226000640 1 -- 172.21.15.138:0/1707515523 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c005020 con 0x7f1228102800 2024-08-20T01:19:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f1226000640 1 -- 172.21.15.138:0/1707515523 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c004a00 con 0x7f1228102800 2024-08-20T01:19:52.644 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.642+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12281a7970 con 0x7f1228102800 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.645+0000 7f1233093740 1 Processor -- start 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.645+0000 7f1233093740 1 -- start start 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 0x55bc52c7be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c7e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc52be5210 con 0x55bc52c70430 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bc52bd4710 con 0x55bc52c6fb10 2024-08-20T01:19:52.647 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1233093740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc52bd8750 con 0x55bc52c7c3c0 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f1226000640 1 -- 172.21.15.138:0/1707515523 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f121c004ba0 con 0x7f1228102800 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42276/0 (socket says 172.21.15.138:42276) 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 -- 172.21.15.138:0/1230031890 learned_addr learned my addr 172.21.15.138:0/1230031890 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122ce00640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c7e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122d800640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 0x55bc52c7be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 -- 172.21.15.138:0/1230031890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 msgr2=0x55bc52c7e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c7e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.646+0000 7f122e200640 1 -- 172.21.15.138:0/1230031890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 msgr2=0x55bc52c7be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f122e200640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 0x55bc52c7be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.648 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f122e200640 1 -- 172.21.15.138:0/1230031890 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc52ba0de0 con 0x55bc52c6fb10 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f122d800640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 0x55bc52c7be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f122e200640 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f12200093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecdb8ed551924f4f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f121aa00640 1 -- 172.21.15.138:0/1230031890 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220031cf0 con 0x55bc52c6fb10 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f121aa00640 1 -- 172.21.15.138:0/1230031890 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1220036030 con 0x55bc52c6fb10 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.647+0000 7f121aa00640 1 -- 172.21.15.138:0/1230031890 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220004240 con 0x55bc52c6fb10 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 -- 172.21.15.138:0/1230031890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 msgr2=0x55bc52c6fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f12200093e0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1226000640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 0x7f12100b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 -- 172.21.15.138:0/1230031890 shutdown_connections 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c7e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c70430 0x55bc52c7be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 --2- 172.21.15.138:0/1230031890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c6fb10 0x55bc52c6fef0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 -- 172.21.15.138:0/1230031890 >> 172.21.15.138:0/1230031890 conn(0x55bc52c6d940 msgr2=0x55bc52c6eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 0x7f12100b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1226000640 1 -- 172.21.15.138:0/1707515523 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f121c0e43f0 con 0x7f1228102800 2024-08-20T01:19:52.649 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 -- 172.21.15.138:0/1230031890 shutdown_connections 2024-08-20T01:19:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.648+0000 7f1233093740 1 -- 172.21.15.138:0/1230031890 wait complete. 2024-08-20T01:19:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 -- start start 2024-08-20T01:19:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f122d800640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 0x7f12100b6200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55bc52c72c90 tx=0x7f121c005460 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.650 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 Processor -- start 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c70430 0x55bc52c78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c7c3c0 0x55bc52c79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bc52b0dab0 con 0x55bc52c6fb10 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bc52b7b140 con 0x55bc52c7c3c0 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bc52c7eec0 con 0x55bc52c70430 2024-08-20T01:19:52.653 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.649+0000 7f1233093740 10 client.0 _ll_register_callbacks cb 0x55bc52b8e5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46942/0 (socket says 172.21.15.138:46942) 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 -- 172.21.15.138:0/1713714177 learned_addr learned my addr 172.21.15.138:0/1713714177 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122ce00640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c70430 0x55bc52c78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c7c3c0 0x55bc52c79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 -- 172.21.15.138:0/1713714177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c70430 msgr2=0x55bc52c78fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c70430 0x55bc52c78fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 -- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c7c3c0 msgr2=0x55bc52c79510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c7c3c0 0x55bc52c79510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 -- 172.21.15.138:0/1713714177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc52b7c170 con 0x55bc52c6fb10 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f122e200640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1220009620 tx=0x7f1220003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b68fc78bd8af1aec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f120f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220058060 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f122002f610 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.650+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1220036030 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f122005f050 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 10 client.16833 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bc52937790 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1220036030 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f1220043070 con 0x55bc52c6fb10 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 10 client.16833 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 20 client.16833 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52c8bb60 0x55bc52c8df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.655 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 4 client.16833 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.651+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc52be7160 con 0x55bc52c8bb60 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.652+0000 7f122ce00640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52c8bb60 0x55bc52c8df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.653+0000 7f122ce00640 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52c8bb60 0x55bc52c8df50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5962738437939416 server_cookie=b266630ff64c96cf in_seq=0 out_seq=0 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.653+0000 7f120f400640 10 client.16833 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f120f400640 1 -- 172.21.15.138:0/1713714177 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bc52be7160 con 0x55bc52c8bb60 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f120f400640 10 client.16833 handle_command_reply: tid=0 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 client.16833 shutdown 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 msgr2=0x55bc52c7bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1220009620 tx=0x7f1220003d00 comp rx=0 tx=0).stop 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 shutdown_connections 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52c8bb60 0x55bc52c8df50 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c7c3c0 0x55bc52c79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f120f400640 0 client.16833 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:52.656 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c70430 0x55bc52c78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 --2- 172.21.15.138:0/1713714177 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c7bb90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 >> 172.21.15.138:0/1713714177 conn(0x55bc52c6d940 msgr2=0x55bc52c734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 shutdown_connections 2024-08-20T01:19:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 1 -- 172.21.15.138:0/1713714177 wait complete. 2024-08-20T01:19:52.657 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.654+0000 7f1233093740 20 client.16833 trim_cache size 0 max 16384 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.669+0000 7f1233093740 1 Processor -- start 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.669+0000 7f1233093740 1 -- start start 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.669+0000 7f1233093740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c6fb10 0x55bc52c6f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.669+0000 7f1233093740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f1233093740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f1233093740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b4340 con 0x55bc52c7c3c0 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f1233093740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b1bb0 con 0x55bc52c70430 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f1233093740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12281afa20 con 0x55bc52c6fb10 2024-08-20T01:19:52.669 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46946/0 (socket says 172.21.15.138:46946) 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 -- 172.21.15.138:0/1230602508 learned_addr learned my addr 172.21.15.138:0/1230602508 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122ce00640 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c6fb10 0x55bc52c6f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 -- 172.21.15.138:0/1230602508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c6fb10 msgr2=0x55bc52c6f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c6fb10 0x55bc52c6f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 -- 172.21.15.138:0/1230602508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 msgr2=0x55bc52c744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 -- 172.21.15.138:0/1230602508 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc529b0620 con 0x55bc52c7c3c0 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.670+0000 7f122e200640 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f122002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f81d1f5aa14be2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.670 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f121a000640 1 -- 172.21.15.138:0/1230602508 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220003b30 con 0x55bc52c7c3c0 2024-08-20T01:19:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f121a000640 1 -- 172.21.15.138:0/1230602508 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1220036070 con 0x55bc52c7c3c0 2024-08-20T01:19:52.671 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f121a000640 1 -- 172.21.15.138:0/1230602508 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122002f870 con 0x55bc52c7c3c0 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 -- 172.21.15.138:0/1230602508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 msgr2=0x55bc52c93720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f122002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 -- 172.21.15.138:0/1230602508 shutdown_connections 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c7c3c0 0x55bc52c93720 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 --2- 172.21.15.138:0/1230602508 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c6fb10 0x55bc52c6f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 -- 172.21.15.138:0/1230602508 >> 172.21.15.138:0/1230602508 conn(0x55bc52c6d940 msgr2=0x55bc52c73dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 -- 172.21.15.138:0/1230602508 shutdown_connections 2024-08-20T01:19:52.674 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.671+0000 7f1233093740 1 -- 172.21.15.138:0/1230602508 wait complete. 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 -- start start 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 Processor -- start 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c73780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c73cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b0070 con 0x55bc52c6fb10 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b14e0 con 0x55bc52c70430 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12281b0740 con 0x55bc52c7c3c0 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f1233093740 10 client.0 _ll_register_callbacks cb 0x7f1214009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.672+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46954/0 (socket says 172.21.15.138:46954) 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 -- 172.21.15.138:0/3887538562 learned_addr learned my addr 172.21.15.138:0/3887538562 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c73cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122ce00640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c73780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 -- 172.21.15.138:0/3887538562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 msgr2=0x55bc52c73cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.675 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c73cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 -- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 msgr2=0x55bc52c73780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c73780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 -- 172.21.15.138:0/3887538562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc52c74400 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122ce00640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c73780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f122e200640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1220003a70 tx=0x7f122002bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7e5be3db8f096b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f120f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1220069020 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f1220030a30 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.673+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1220036070 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1220074050 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 10 client.16839 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55bc52cf66f0 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1220036070 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f1220043070 con 0x55bc52c6fb10 2024-08-20T01:19:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 10 client.16839 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 10 client.16839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 20 client.16839 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52cf6d00 0x55bc52cf9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 4 client.16839 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.674+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc52be2c00 con 0x55bc52cf6d00 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.675+0000 7f122ce00640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52cf6d00 0x55bc52cf9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.675+0000 7f122ce00640 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52cf6d00 0x55bc52cf9120 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a0ea008c422f1f server_cookie=5d74358afaff01a3 in_seq=0 out_seq=0 2024-08-20T01:19:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.675+0000 7f120f400640 10 client.16839 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f120f400640 1 -- 172.21.15.138:0/3887538562 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bc52be2c00 con 0x55bc52cf6d00 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f120f400640 10 client.16839 handle_command_reply: tid=0 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 client.16839 shutdown 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 msgr2=0x55bc52c846c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1220003a70 tx=0x7f122002bfd0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 shutdown_connections 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bc52cf6d00 0x55bc52cf9120 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bc52c7c3c0 0x55bc52c73cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f120f400640 0 client.16839 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:52.688 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc52c70430 0x55bc52c73780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 --2- 172.21.15.138:0/3887538562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bc52c6fb10 0x55bc52c846c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 >> 172.21.15.138:0/3887538562 conn(0x55bc52c6d940 msgr2=0x55bc52c7a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 shutdown_connections 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 1 -- 172.21.15.138:0/3887538562 wait complete. 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.689+0000 7f1233093740 20 client.16839 trim_cache size 0 max 16384 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:19:52.689 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 msgr2=0x7f12100b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 0x7f12100b6200 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55bc52c72c90 tx=0x7f121c005460 comp rx=0 tx=0).stop 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 msgr2=0x7f122819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 0x7f122819af20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9f0 tx=0x7f121c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 shutdown_connections 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12100b3d40 0x7f12100b6200 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1228108b80 0x7f12281a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1228108200 0x7f122819b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 --2- 172.21.15.138:0/1707515523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1228102800 0x7f122819af20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 >> 172.21.15.138:0/1707515523 conn(0x7f12280fe280 msgr2=0x7f12281047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 shutdown_connections 2024-08-20T01:19:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:19:52.690+0000 7f122ec00640 1 -- 172.21.15.138:0/1707515523 wait complete. 2024-08-20T01:19:53.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.63 MB/sec execute 485 sec latency 16242.499 ms 2024-08-20T01:19:54.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.62 MB/sec execute 486 sec latency 17242.655 ms 2024-08-20T01:19:54.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:53 smithi167 ceph-mon[21965]: pgmap v726: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 3.6 MiB/s wr, 102 op/s 2024-08-20T01:19:54.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:53 smithi142 ceph-mon[21799]: pgmap v726: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 3.6 MiB/s wr, 102 op/s 2024-08-20T01:19:54.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:53 smithi138 ceph-mon[18076]: pgmap v726: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 232 KiB/s rd, 3.6 MiB/s wr, 102 op/s 2024-08-20T01:19:55.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.62 MB/sec execute 487 sec latency 18242.737 ms 2024-08-20T01:19:56.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.61 MB/sec execute 488 sec latency 19242.813 ms 2024-08-20T01:19:56.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:55 smithi167 ceph-mon[21965]: pgmap v727: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.5 MiB/s wr, 92 op/s 2024-08-20T01:19:56.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:55 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:19:56.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:55 smithi142 ceph-mon[21799]: pgmap v727: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.5 MiB/s wr, 92 op/s 2024-08-20T01:19:56.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:55 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:19:56.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:55 smithi138 ceph-mon[18076]: pgmap v727: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 3.5 MiB/s wr, 92 op/s 2024-08-20T01:19:56.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:55 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:19:56.444 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:19:56.819 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:19:57.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.60 MB/sec execute 489 sec latency 20242.948 ms 2024-08-20T01:19:57.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:19:57.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:19:57.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:56 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:19:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:19:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:19:57.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:56 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:19:57.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:19:57.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:19:57.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:56 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 -- 172.21.15.138:0/3704282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 msgr2=0x7f894010ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 --2- 172.21.15.138:0/3704282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894010ef30 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f892c009900 tx=0x7f892c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 -- 172.21.15.138:0/3704282043 shutdown_connections 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 --2- 172.21.15.138:0/3704282043 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f894010f470 0x7f8940115c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 --2- 172.21.15.138:0/3704282043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894010ef30 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 --2- 172.21.15.138:0/3704282043 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f89401090e0 0x7f89401094c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 -- 172.21.15.138:0/3704282043 >> 172.21.15.138:0/3704282043 conn(0x7f8940078760 msgr2=0x7f894007ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:57.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 -- 172.21.15.138:0/3704282043 shutdown_connections 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.591+0000 7f8948400640 1 -- 172.21.15.138:0/3704282043 wait complete. 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.592+0000 7f8948400640 1 Processor -- start 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.592+0000 7f8948400640 1 -- start start 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 0x7f894019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894010f470 0x7f89401a4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f89401190c0 con 0x7f894010eaf0 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8940118f40 con 0x7f89401090e0 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8948400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8940119240 con 0x7f894010f470 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8945000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8945000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46978/0 (socket says 172.21.15.138:46978) 2024-08-20T01:19:57.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8945000640 1 -- 172.21.15.138:0/1070761552 learned_addr learned my addr 172.21.15.138:0/1070761552 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8945a00640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 0x7f894019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.593+0000 7f8946400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894010f470 0x7f89401a4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 -- 172.21.15.138:0/1070761552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894010f470 msgr2=0x7f89401a4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894010f470 0x7f89401a4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 -- 172.21.15.138:0/1070761552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 msgr2=0x7f894019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 0x7f894019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 -- 172.21.15.138:0/1070761552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89401a5660 con 0x7f894010eaf0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945a00640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 0x7f894019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8945000640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c03f070 con 0x7f894010eaf0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f892c031b60 con 0x7f894010eaf0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c035030 con 0x7f894010eaf0 2024-08-20T01:19:57.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.594+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89401a58f0 con 0x7f894010eaf0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.595+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89401ac010 con 0x7f894010eaf0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.596+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89401094c0 con 0x7f894010eaf0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.598+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f892c042070 con 0x7f894010eaf0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.599+0000 7f893e000640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 0x7f89280b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.599+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f892c1049d0 con 0x7f894010eaf0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.599+0000 7f8945a00640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 0x7f89280b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.600+0000 7f8945a00640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 0x7f89280b6280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8934009fc0 tx=0x7f8934005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:57.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.603+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f892c0cb8e0 con 0x7f894010eaf0 2024-08-20T01:19:57.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.820+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8940110cc0 con 0x7f894010eaf0 2024-08-20T01:19:57.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.821+0000 7f893e000640 1 -- 172.21.15.138:0/1070761552 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11934 (secure 0 0 0) 0x7f892c0cb1e0 con 0x7f894010eaf0 2024-08-20T01:19:57.819 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:19:57.820 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":41,"btime":"2024-08-20T01:19:37:843771+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":41,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:19:37.552766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:19:57.820 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 41 2024-08-20T01:19:57.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 msgr2=0x7f89280b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:57.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 0x7f89280b6280 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8934009fc0 tx=0x7f8934005f90 comp rx=0 tx=0).stop 2024-08-20T01:19:57.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 msgr2=0x7f894019faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:57.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f892c005ab0 tx=0x7f892c003c10 comp rx=0 tx=0).stop 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 shutdown_connections 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89280b3dc0 0x7f89280b6280 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f894010f470 0x7f89401a4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f894010eaf0 0x7f894019faf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 --2- 172.21.15.138:0/1070761552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89401090e0 0x7f894019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.823+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 >> 172.21.15.138:0/1070761552 conn(0x7f8940078760 msgr2=0x7f894010c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.824+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 shutdown_connections 2024-08-20T01:19:57.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:57.824+0000 7f8948400640 1 -- 172.21.15.138:0/1070761552 wait complete. 2024-08-20T01:19:58.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.59 MB/sec execute 490 sec latency 21243.038 ms 2024-08-20T01:19:58.290 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:58 smithi138 ceph-mon[18076]: pgmap v728: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.8 MiB/s wr, 70 op/s 2024-08-20T01:19:58.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:19:58 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1070761552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:58.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:58 smithi167 ceph-mon[21965]: pgmap v728: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.8 MiB/s wr, 70 op/s 2024-08-20T01:19:58.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:19:58 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1070761552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:58.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:58 smithi142 ceph-mon[21799]: pgmap v728: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 2.8 MiB/s wr, 70 op/s 2024-08-20T01:19:58.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:19:58 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1070761552' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:19:58.789 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 248f5e2c --release --await' 2024-08-20T01:19:58.790 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 248f5e2c --release --await 2024-08-20T01:19:59.159 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:19:59.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460033 3.59 MB/sec execute 491 sec latency 22243.075 ms 2024-08-20T01:19:59.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 -- 172.21.15.138:0/959732845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810aea0 msgr2=0x7f20981116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 --2- 172.21.15.138:0/959732845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810aea0 0x7f20981116a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2090009ca0 tx=0x7f209002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 -- 172.21.15.138:0/959732845 shutdown_connections 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 --2- 172.21.15.138:0/959732845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810aea0 0x7f20981116a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 --2- 172.21.15.138:0/959732845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810a520 0x7f209810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 --2- 172.21.15.138:0/959732845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f2098104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 -- 172.21.15.138:0/959732845 >> 172.21.15.138:0/959732845 conn(0x7f2098100540 msgr2=0x7f2098102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 -- 172.21.15.138:0/959732845 shutdown_connections 2024-08-20T01:19:59.788 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.788+0000 7f209ee00640 1 -- 172.21.15.138:0/959732845 wait complete. 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.789+0000 7f209ee00640 1 Processor -- start 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 -- start start 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 0x7f209819b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810aea0 0x7f20981a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2098114bb0 con 0x7f209810a520 2024-08-20T01:19:59.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2098114a30 con 0x7f209810aea0 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209ee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2098114d30 con 0x7f2098104b10 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 0x7f209819b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 0x7f209819b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51734/0 (socket says 172.21.15.138:51734) 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54334/0 (socket says 172.21.15.138:54334) 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209d000640 1 -- 172.21.15.138:0/2123660279 learned_addr learned my addr 172.21.15.138:0/2123660279 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.790+0000 7f209e400640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810aea0 0x7f20981a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 -- 172.21.15.138:0/2123660279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810aea0 msgr2=0x7f20981a07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810aea0 0x7f20981a07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 -- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 msgr2=0x7f209819b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 0x7f209819b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 -- 172.21.15.138:0/2123660279 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20981a0f70 con 0x7f2098104b10 2024-08-20T01:19:59.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209da00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9f0 tx=0x7f208c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f2096000640 1 -- 172.21.15.138:0/2123660279 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c00ccc0 con 0x7f2098104b10 2024-08-20T01:19:59.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.791+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20981a1260 con 0x7f2098104b10 2024-08-20T01:19:59.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.792+0000 7f2096000640 1 -- 172.21.15.138:0/2123660279 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c005020 con 0x7f2098104b10 2024-08-20T01:19:59.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.792+0000 7f2096000640 1 -- 172.21.15.138:0/2123660279 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c004a00 con 0x7f2098104b10 2024-08-20T01:19:59.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.792+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20981a7920 con 0x7f2098104b10 2024-08-20T01:19:59.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.795+0000 7f20a2e34740 1 Processor -- start 2024-08-20T01:19:59.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.795+0000 7f2096000640 1 -- 172.21.15.138:0/2123660279 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f208c004ba0 con 0x7f2098104b10 2024-08-20T01:19:59.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.795+0000 7f20a2e34740 1 -- start start 2024-08-20T01:19:59.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570b0847670 con 0x5570b08a67a0 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5570b081bd70 con 0x5570b08a5e80 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f20a2e34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570b080b270 con 0x5570b08b2730 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f209da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f209e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f209da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51748/0 (socket says 172.21.15.138:51748) 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f209da00640 1 -- 172.21.15.138:0/4029400760 learned_addr learned my addr 172.21.15.138:0/4029400760 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:59.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.796+0000 7f209d000640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 -- 172.21.15.138:0/4029400760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 msgr2=0x5570b08b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 -- 172.21.15.138:0/4029400760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 msgr2=0x5570b08a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 -- 172.21.15.138:0/4029400760 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b07a7870 con 0x5570b08a67a0 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209d000640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209e400640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f209da00640 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e16b0 tx=0x7f208c0ec620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed29b5cfe61bc388 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.797+0000 7f2096000640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 0x7f20800b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.798+0000 7f208aa00640 1 -- 172.21.15.138:0/4029400760 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0f8070 con 0x5570b08a67a0 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.798+0000 7f209e400640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 0x7f20800b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.798+0000 7f2096000640 1 -- 172.21.15.138:0/2123660279 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f208c0e43f0 con 0x7f2098104b10 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.798+0000 7f208aa00640 1 -- 172.21.15.138:0/4029400760 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c060220 con 0x5570b08a67a0 2024-08-20T01:19:59.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.798+0000 7f208aa00640 1 -- 172.21.15.138:0/4029400760 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0aa370 con 0x5570b08a67a0 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 -- 172.21.15.138:0/4029400760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 msgr2=0x5570b08b21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f209e400640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 0x7f20800b62a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f2090005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e16b0 tx=0x7f208c0ec620 comp rx=0 tx=0).stop 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 -- 172.21.15.138:0/4029400760 shutdown_connections 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b21f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 --2- 172.21.15.138:0/4029400760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 -- 172.21.15.138:0/4029400760 >> 172.21.15.138:0/4029400760 conn(0x5570b08a3cb0 msgr2=0x5570b08a4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 -- 172.21.15.138:0/4029400760 shutdown_connections 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.799+0000 7f20a2e34740 1 -- 172.21.15.138:0/4029400760 wait complete. 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 -- start start 2024-08-20T01:19:59.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 Processor -- start 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a5e80 0x5570b08b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 0x5570b08af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08af920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5570b0791ff0 con 0x5570b08a5e80 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5570b080f2b0 con 0x5570b08b2730 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.800+0000 7f20a2e34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5570b066dbb0 con 0x5570b08a67a0 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f20a2e34740 10 client.0 _ll_register_callbacks cb 0x5570b0516d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:59.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 0x5570b08af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a5e80 0x5570b08b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 0x5570b08af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54360/0 (socket says 172.21.15.138:54360) 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209d000640 1 -- 172.21.15.138:0/479907834 learned_addr learned my addr 172.21.15.138:0/479907834 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 -- 172.21.15.138:0/479907834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 msgr2=0x5570b08af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 0x5570b08af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 -- 172.21.15.138:0/479907834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a5e80 msgr2=0x5570b08b1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a5e80 0x5570b08b1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.801+0000 7f209e400640 1 -- 172.21.15.138:0/479907834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b056ab00 con 0x5570b08b2730 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f209e400640 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08af920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2090005e30 tx=0x7f209002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a95b84bec96d8f3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f207f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:19:59.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209002f610 con 0x5570b08b2730 2024-08-20T01:19:59.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f208c060220 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f209002f7b0 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f209002f9b0 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f20a2e34740 10 client.26906 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.802+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570b05e1960 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208c060220 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f2090051070 con 0x5570b08b2730 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 10 client.26906 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 10 client.26906 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 20 client.26906 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b08c1da0 0x5570b08c41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 4 client.26906 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570b081dcc0 con 0x5570b08c1da0 2024-08-20T01:19:59.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.803+0000 7f209da00640 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b08c1da0 0x5570b08c41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.804+0000 7f209da00640 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b08c1da0 0x5570b08c41e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5cb0e435507059 server_cookie=f9e462fe9ab0ed2d in_seq=0 out_seq=0 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.804+0000 7f207f400640 10 client.26906 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.804+0000 7f207f400640 1 -- 172.21.15.138:0/479907834 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570b081dcc0 con 0x5570b08c1da0 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.804+0000 7f207f400640 10 client.26906 handle_command_reply: tid=0 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 client.26906 shutdown 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 msgr2=0x5570b08af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08af920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2090005e30 tx=0x7f209002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 shutdown_connections 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b08c1da0 0x5570b08c41e0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08af920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a67a0 0x5570b08af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 --2- 172.21.15.138:0/479907834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a5e80 0x5570b08b1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f207f400640 0 client.26906 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:59.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 >> 172.21.15.138:0/479907834 conn(0x5570b08a3cb0 msgr2=0x5570b08a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 shutdown_connections 2024-08-20T01:19:59.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 1 -- 172.21.15.138:0/479907834 wait complete. 2024-08-20T01:19:59.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.805+0000 7f20a2e34740 20 client.26906 trim_cache size 0 max 16384 2024-08-20T01:19:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 Processor -- start 2024-08-20T01:19:59.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 -- start start 2024-08-20T01:19:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b4430 con 0x5570b08a67a0 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b1ca0 con 0x5570b08a5e80 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.823+0000 7f20a2e34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20981afb10 con 0x5570b08b2730 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51766/0 (socket says 172.21.15.138:51766) 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209d000640 1 -- 172.21.15.138:0/572679249 learned_addr learned my addr 172.21.15.138:0/572679249 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209da00640 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 -- 172.21.15.138:0/572679249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 msgr2=0x5570b08a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 -- 172.21.15.138:0/572679249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 msgr2=0x5570b08c9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08c9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.824+0000 7f209e400640 1 -- 172.21.15.138:0/572679249 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570b08a4730 con 0x5570b08b2730 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.825+0000 7f209e400640 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08c9ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2090043040 tx=0x7f209002b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bd15c63acce4c48 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.825+0000 7f208a000640 1 -- 172.21.15.138:0/572679249 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090004b40 con 0x5570b08b2730 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.825+0000 7f208a000640 1 -- 172.21.15.138:0/572679249 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090036050 con 0x5570b08b2730 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.825+0000 7f208a000640 1 -- 172.21.15.138:0/572679249 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090030c40 con 0x5570b08b2730 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- 172.21.15.138:0/572679249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 msgr2=0x5570b08c9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08c9ff0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2090043040 tx=0x7f209002b040 comp rx=0 tx=0).stop 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- 172.21.15.138:0/572679249 shutdown_connections 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08b2730 0x5570b08c9ff0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.825 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 --2- 172.21.15.138:0/572679249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08a5e80 0x5570b08a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- 172.21.15.138:0/572679249 >> 172.21.15.138:0/572679249 conn(0x5570b08a3cb0 msgr2=0x5570b08a2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- 172.21.15.138:0/572679249 shutdown_connections 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- 172.21.15.138:0/572679249 wait complete. 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.826+0000 7f20a2e34740 1 -- start start 2024-08-20T01:19:59.826 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 Processor -- start 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 0x5570b08badf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b0160 con 0x5570b08a67a0 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b15d0 con 0x5570b08b2730 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f20981b0830 con 0x5570b08a5e80 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f20a2e34740 10 client.0 _ll_register_callbacks cb 0x7f2090009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f209d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51776/0 (socket says 172.21.15.138:51776) 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f209d000640 1 -- 172.21.15.138:0/404239517 learned_addr learned my addr 172.21.15.138:0/404239517 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:19:59.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.827+0000 7f209da00640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 0x5570b08badf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209e400640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 -- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 msgr2=0x5570b08b8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08b8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 -- 172.21.15.138:0/404239517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 msgr2=0x5570b08b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 -- 172.21.15.138:0/404239517 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570b092ba90 con 0x5570b08a5e80 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f209da00640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 0x5570b08badf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e16b0 tx=0x7f208c092040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2bb4cf1e286866b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f207f400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c01c040 con 0x5570b08a5e80 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2090036050 con 0x5570b08a5e80 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.828+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9453+0+0 (secure 0 0 0) 0x7f208c0ec150 con 0x5570b08a5e80 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f208c0aa370 con 0x5570b08a5e80 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 10 client.26866 fetch_fsmap learned FSMap version 41 2024-08-20T01:19:59.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x5570b092cdd0 con 0x5570b08a5e80 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2090036050 con 0x5570b08a5e80 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 41) ==== 8362+0+0 (secure 0 0 0) 0x7f208c0a6e90 con 0x5570b08a5e80 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 10 client.26866 fetch_fsmap finished waiting for FSMap version 41 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 10 client.26866 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 20 client.26866 populate_metadata read hostname 'smithi138' 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b092d170 0x5570b092f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 4 client.26866 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "248f5e2c", "release": true, "await": true}] 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.829+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "248f5e2c", "release": true, "await": true}) -- 0x5570b0819760 con 0x5570b092d170 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.830+0000 7f209d000640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b092d170 0x5570b092f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:19:59.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.830+0000 7f209d000640 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b092d170 0x5570b092f560 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b748ae4d2c51c13 server_cookie=fa285505f32e7878 in_seq=0 out_seq=0 2024-08-20T01:19:59.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.830+0000 7f207f400640 10 client.26866 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:59.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f207f400640 1 -- 172.21.15.138:0/404239517 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5570b0819760 con 0x5570b092d170 2024-08-20T01:19:59.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f207f400640 10 client.26866 handle_command_reply: tid=0 2024-08-20T01:19:59.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 client.26866 shutdown 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 msgr2=0x5570b08badf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 0x5570b08badf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f208c0e16b0 tx=0x7f208c092040 comp rx=0 tx=0).stop 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 shutdown_connections 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5570b092d170 0x5570b092f560 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570b08b2730 0x5570b08b8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5570b08a67a0 0x5570b08b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f207f400640 0 client.26866 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 --2- 172.21.15.138:0/404239517 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5570b08a5e80 0x5570b08badf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 >> 172.21.15.138:0/404239517 conn(0x5570b08a3cb0 msgr2=0x5570b08a2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.846+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 shutdown_connections 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f20a2e34740 1 -- 172.21.15.138:0/404239517 wait complete. 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f20a2e34740 20 client.26866 trim_cache size 0 max 16384 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:19:59.846 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 41, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 44, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "248f5e2c": { 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 44, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:19:59.847 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 143.3, 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:19:59.848 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 msgr2=0x7f20800b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 0x7f20800b62a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2090006010 tx=0x7f2090005d80 comp rx=0 tx=0).stop 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 msgr2=0x7f209819aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.847+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e9f0 tx=0x7f208c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 shutdown_connections 2024-08-20T01:19:59.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20800b3de0 0x7f20800b62a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f209810aea0 0x7f20981a07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f209810a520 0x7f209819b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 --2- 172.21.15.138:0/2123660279 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2098104b10 0x7f209819aec0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 >> 172.21.15.138:0/2123660279 conn(0x7f2098100540 msgr2=0x7f2098103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 shutdown_connections 2024-08-20T01:19:59.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:19:59.848+0000 7f209ee00640 1 -- 172.21.15.138:0/2123660279 wait complete. 2024-08-20T01:20:00.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 460114 3.58 MB/sec execute 492 sec latency 22905.101 ms 2024-08-20T01:20:00.267 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:00 smithi138 ceph-mon[18076]: pgmap v729: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.2 MiB/s wr, 27 op/s 2024-08-20T01:20:00.267 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:00 smithi138 ceph-mon[18076]: overall HEALTH_OK 2024-08-20T01:20:00.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:00 smithi167 ceph-mon[21965]: pgmap v729: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.2 MiB/s wr, 27 op/s 2024-08-20T01:20:00.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:00 smithi167 ceph-mon[21965]: overall HEALTH_OK 2024-08-20T01:20:00.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:00 smithi142 ceph-mon[21799]: pgmap v729: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 1.2 MiB/s wr, 27 op/s 2024-08-20T01:20:00.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:00 smithi142 ceph-mon[21799]: overall HEALTH_OK 2024-08-20T01:20:00.714 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '248f5e2c', total seconds elapsed: 27.6 2024-08-20T01:20:01.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 461601 3.59 MB/sec execute 493 sec latency 356.400 ms 2024-08-20T01:20:01.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:01 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:01.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:01 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:01.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:01 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:02.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 462358 3.59 MB/sec execute 494 sec latency 64.506 ms 2024-08-20T01:20:02.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:02 smithi167 ceph-mon[21965]: pgmap v730: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.1 MiB/s wr, 26 op/s 2024-08-20T01:20:02.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:02 smithi142 ceph-mon[21799]: pgmap v730: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.1 MiB/s wr, 26 op/s 2024-08-20T01:20:02.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:02 smithi138 ceph-mon[18076]: pgmap v730: 129 pgs: 129 active+clean; 374 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 104 KiB/s rd, 1.1 MiB/s wr, 26 op/s 2024-08-20T01:20:03.199 INFO:tasks.workunit.client.0.smithi138.stdout: 1 463595 3.60 MB/sec execute 495 sec latency 278.233 ms 2024-08-20T01:20:04.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 463595 3.59 MB/sec execute 496 sec latency 1278.368 ms 2024-08-20T01:20:04.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:04 smithi167 ceph-mon[21965]: pgmap v731: 129 pgs: 129 active+clean; 376 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 503 KiB/s wr, 30 op/s 2024-08-20T01:20:04.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:04 smithi142 ceph-mon[21799]: pgmap v731: 129 pgs: 129 active+clean; 376 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 503 KiB/s wr, 30 op/s 2024-08-20T01:20:04.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:04 smithi138 ceph-mon[18076]: pgmap v731: 129 pgs: 129 active+clean; 376 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 503 KiB/s wr, 30 op/s 2024-08-20T01:20:05.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 465740 3.61 MB/sec execute 497 sec latency 1317.468 ms 2024-08-20T01:20:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 -- 172.21.15.138:0/1281593862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 msgr2=0x7f4fc410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1281593862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc410f3b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009ca0 tx=0x7f4fbc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 -- 172.21.15.138:0/1281593862 shutdown_connections 2024-08-20T01:20:05.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1281593862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc410f3b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1281593862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 0x7f4fc4105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1281593862 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc406ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 -- 172.21.15.138:0/1281593862 >> 172.21.15.138:0/1281593862 conn(0x7f4fc4077270 msgr2=0x7f4fc4077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.823+0000 7f4fcac00640 1 -- 172.21.15.138:0/1281593862 shutdown_connections 2024-08-20T01:20:05.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.824+0000 7f4fcac00640 1 -- 172.21.15.138:0/1281593862 wait complete. 2024-08-20T01:20:05.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 Processor -- start 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 -- start start 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 0x7f4fc419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc41a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4112850 con 0x7f4fc4104fd0 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41126d0 con 0x7f4fc4105970 2024-08-20T01:20:05.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fcac00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41129d0 con 0x7f4fc406a7a0 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc41a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54412/0 (socket says 172.21.15.138:54412) 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fc9800640 1 -- 172.21.15.138:0/1043795216 learned_addr learned my addr 172.21.15.138:0/1043795216 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc41a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57916/0 (socket says 172.21.15.138:57916) 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.825+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 0x7f4fc419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 -- 172.21.15.138:0/1043795216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 msgr2=0x7f4fc41a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc41a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 -- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 msgr2=0x7f4fc419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 0x7f4fc419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 -- 172.21.15.138:0/1043795216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc41a0f80 con 0x7f4fc406a7a0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc2000640 1 -- 172.21.15.138:0/1043795216 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb800cd80 con 0x7f4fc406a7a0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc2000640 1 -- 172.21.15.138:0/1043795216 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb80108e0 con 0x7f4fc406a7a0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.826+0000 7f4fc2000640 1 -- 172.21.15.138:0/1043795216 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8018980 con 0x7f4fc406a7a0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.827+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41a1270 con 0x7f4fc406a7a0 2024-08-20T01:20:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.827+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc41a7930 con 0x7f4fc406a7a0 2024-08-20T01:20:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.830+0000 7f4fcf262740 1 Processor -- start 2024-08-20T01:20:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.830+0000 7f4fcf262740 1 -- start start 2024-08-20T01:20:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff4430 0x55aa70fffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa710027b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa70f69210 con 0x55aa710003c0 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aa70f58710 con 0x55aa70ff4430 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fcf262740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa70f5c750 con 0x55aa70ff3b10 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fc2000640 1 -- 172.21.15.138:0/1043795216 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4fb8010430 con 0x7f4fc406a7a0 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54420/0 (socket says 172.21.15.138:54420) 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 -- 172.21.15.138:0/2963715356 learned_addr learned my addr 172.21.15.138:0/2963715356 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa710027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fc9800640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff4430 0x55aa70fffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 -- 172.21.15.138:0/2963715356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff4430 msgr2=0x55aa70fffe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff4430 0x55aa70fffe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 -- 172.21.15.138:0/2963715356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 msgr2=0x55aa710027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa710027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.831+0000 7f4fca200640 1 -- 172.21.15.138:0/2963715356 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa70f24de0 con 0x55aa70ff3b10 2024-08-20T01:20:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.832+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa710027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.832+0000 7f4fca200640 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009ca0 tx=0x7f4fbc0093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47eff4501562df83 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.832+0000 7f4fb6a00640 1 -- 172.21.15.138:0/2963715356 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc031cf0 con 0x55aa70ff3b10 2024-08-20T01:20:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.832+0000 7f4fb6a00640 1 -- 172.21.15.138:0/2963715356 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc036030 con 0x55aa70ff3b10 2024-08-20T01:20:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.832+0000 7f4fb6a00640 1 -- 172.21.15.138:0/2963715356 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0042a0 con 0x55aa70ff3b10 2024-08-20T01:20:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 -- 172.21.15.138:0/2963715356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 msgr2=0x55aa70ff3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009ca0 tx=0x7f4fbc0093e0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 -- 172.21.15.138:0/2963715356 shutdown_connections 2024-08-20T01:20:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa710027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff4430 0x55aa70fffe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2963715356 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff3ef0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fcf262740 1 -- 172.21.15.138:0/2963715356 >> 172.21.15.138:0/2963715356 conn(0x55aa70ff1940 msgr2=0x55aa70ff2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.833+0000 7f4fc2000640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 0x7f4fac0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.834+0000 7f4fcf262740 1 -- 172.21.15.138:0/2963715356 shutdown_connections 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.834+0000 7f4fc2000640 1 -- 172.21.15.138:0/1043795216 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9541+0+0 (secure 0 0 0) 0x7f4fb80e4700 con 0x7f4fc406a7a0 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.834+0000 7f4fcf262740 1 -- 172.21.15.138:0/2963715356 wait complete. 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.834+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 0x7f4fac0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.834+0000 7f4fcf262740 1 -- start start 2024-08-20T01:20:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.835+0000 7f4fcf262740 1 Processor -- start 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.835+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 0x7f4fac0b6280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc41049e0 tx=0x7f4fb0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa70ffd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55aa70e91ab0 con 0x55aa70ff3b10 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55aa70eff140 con 0x55aa710003c0 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55aa71002ec0 con 0x55aa70ff4430 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51806/0 (socket says 172.21.15.138:51806) 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fc9800640 1 -- 172.21.15.138:0/3049222837 learned_addr learned my addr 172.21.15.138:0/3049222837 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:54430/0 (socket says 172.21.15.138:54430) 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.836+0000 7f4fcf262740 10 client.0 _ll_register_callbacks cb 0x55aa70f125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fca200640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa70ffd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 -- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 msgr2=0x55aa70ffd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa70ffd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 -- 172.21.15.138:0/3049222837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 msgr2=0x55aa70fffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 -- 172.21.15.138:0/3049222837 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa70f00170 con 0x55aa70ff4430 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fca200640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa70ffd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc9800640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00128e0 tx=0x7f4fb0004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f316b575e36fa198 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fab400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb00044e0 con 0x55aa70ff4430 2024-08-20T01:20:05.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.837+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9541+0+0 (secure 0 0 0) 0x7f4fb0018030 con 0x55aa70ff4430 2024-08-20T01:20:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc036030 con 0x55aa70ff4430 2024-08-20T01:20:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4fb0024db0 con 0x55aa70ff4430 2024-08-20T01:20:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fcf262740 10 client.26878 fetch_fsmap learned FSMap version 42 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55aa70cbb790 con 0x55aa70ff4430 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc036030 con 0x55aa70ff4430 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f4fb0026850 con 0x55aa70ff4430 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fcf262740 10 client.26878 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.838+0000 7f4fcf262740 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fcf262740 20 client.26878 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7100fde0 0x55aa71012230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fcf262740 4 client.26878 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa70f6b160 con 0x55aa7100fde0 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fc9800640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7100fde0 0x55aa71012230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fc9800640 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7100fde0 0x55aa71012230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f489bc0bab0e0acb server_cookie=7d9187fdc4aa1278 in_seq=0 out_seq=0 2024-08-20T01:20:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.839+0000 7f4fab400640 10 client.26878 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.840+0000 7f4fab400640 1 -- 172.21.15.138:0/3049222837 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55aa70f6b160 con 0x55aa7100fde0 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.840+0000 7f4fab400640 10 client.26878 handle_command_reply: tid=0 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.840+0000 7f4fcf262740 1 client.26878 shutdown 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 msgr2=0x55aa70ffd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4fb00128e0 tx=0x7f4fb0004730 comp rx=0 tx=0).stop 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 shutdown_connections 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7100fde0 0x55aa71012230 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa70ffd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa70ffd010 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 --2- 172.21.15.138:0/3049222837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff3b10 0x55aa70fffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 >> 172.21.15.138:0/3049222837 conn(0x55aa70ff1940 msgr2=0x55aa70ff74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fab400640 0 client.26878 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 shutdown_connections 2024-08-20T01:20:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 1 -- 172.21.15.138:0/3049222837 wait complete. 2024-08-20T01:20:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.841+0000 7f4fcf262740 20 client.26878 trim_cache size 0 max 16384 2024-08-20T01:20:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.859+0000 7f4fcf262740 1 Processor -- start 2024-08-20T01:20:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.859+0000 7f4fcf262740 1 -- start start 2024-08-20T01:20:05.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa71018080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41b0160 con 0x55aa70ff4430 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ada00 con 0x55aa710003c0 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fcf262740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ab890 con 0x55aa70ff3b10 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fca200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa71018080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc8e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51810/0 (socket says 172.21.15.138:51810) 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc8e00640 1 -- 172.21.15.138:0/1284863894 learned_addr learned my addr 172.21.15.138:0/1284863894 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 -- 172.21.15.138:0/1284863894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 msgr2=0x55aa71018080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa71018080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 -- 172.21.15.138:0/1284863894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 msgr2=0x55aa71017b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc9800640 1 -- 172.21.15.138:0/1284863894 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa70d34620 con 0x55aa70ff3b10 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fca200640 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa71018080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.860+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.861+0000 7f4fc9800640 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff11a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c6a0 tx=0x7f4fb80e6b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfb2a0d2f1f1db87 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.861+0000 7f4fb6000640 1 -- 172.21.15.138:0/1284863894 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80af4b0 con 0x55aa70ff3b10 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.861+0000 7f4fb6000640 1 -- 172.21.15.138:0/1284863894 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb80b3f60 con 0x55aa70ff3b10 2024-08-20T01:20:05.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.861+0000 7f4fb6000640 1 -- 172.21.15.138:0/1284863894 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb80e6e20 con 0x55aa70ff3b10 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- 172.21.15.138:0/1284863894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 msgr2=0x55aa70ff11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff11a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800c6a0 tx=0x7f4fb80e6b50 comp rx=0 tx=0).stop 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- 172.21.15.138:0/1284863894 shutdown_connections 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa710003c0 0x55aa71018080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa70ff4430 0x55aa71017b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.864 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 --2- 172.21.15.138:0/1284863894 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff3b10 0x55aa70ff11a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- 172.21.15.138:0/1284863894 >> 172.21.15.138:0/1284863894 conn(0x55aa70ff1940 msgr2=0x55aa70ffcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- 172.21.15.138:0/1284863894 shutdown_connections 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- 172.21.15.138:0/1284863894 wait complete. 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 -- start start 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.862+0000 7f4fcf262740 1 Processor -- start 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa71008880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa70ff9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41abee0 con 0x55aa710003c0 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ad330 con 0x55aa70ff3b10 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41ac590 con 0x55aa70ff4430 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fcf262740 10 client.0 _ll_register_callbacks cb 0x7f4fb0009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57950/0 (socket says 172.21.15.138:57950) 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 -- 172.21.15.138:0/2247517299 learned_addr learned my addr 172.21.15.138:0/2247517299 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa71008880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fca200640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa70ff9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 -- 172.21.15.138:0/2247517299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 msgr2=0x55aa71008880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa71008880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 -- 172.21.15.138:0/2247517299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 msgr2=0x55aa70ff9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa70ff9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.863+0000 7f4fc9800640 1 -- 172.21.15.138:0/2247517299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa70ffa090 con 0x55aa70ff3b10 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fc9800640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9c0 tx=0x7f4fb8018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330fcb4e579aa7de server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8010a90 con 0x55aa70ff3b10 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fb80b3f60 con 0x55aa70ff3b10 2024-08-20T01:20:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9541+0+0 (secure 0 0 0) 0x7f4fb80e1c70 con 0x55aa70ff3b10 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f4fb80e1e70 con 0x55aa70ff3b10 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fcf262740 10 client.26927 fetch_fsmap learned FSMap version 42 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55aa7107a5e0 con 0x55aa70ff3b10 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb80b3f60 con 0x55aa70ff3b10 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f4fb80e7ce0 con 0x55aa70ff3b10 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fcf262740 10 client.26927 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.864+0000 7f4fcf262740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fcf262740 20 client.26927 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7107ad00 0x55aa7107d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fcf262740 4 client.26927 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa70f66c00 con 0x55aa7107ad00 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7107ad00 0x55aa7107d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fc8e00640 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7107ad00 0x55aa7107d120 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a93b07f6202f254 server_cookie=f91e01b5dd907247 in_seq=0 out_seq=0 2024-08-20T01:20:05.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.865+0000 7f4fab400640 10 client.26927 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.902+0000 7f4fab400640 1 -- 172.21.15.138:0/2247517299 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa70f66c00 con 0x55aa7107ad00 2024-08-20T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.902+0000 7f4fab400640 10 client.26927 handle_command_reply: tid=0 2024-08-20T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.902+0000 7f4fcf262740 1 client.26927 shutdown 2024-08-20T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 msgr2=0x55aa71008340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9c0 tx=0x7f4fb8018420 comp rx=0 tx=0).stop 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 shutdown_connections 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55aa7107ad00 0x55aa7107d120 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55aa710003c0 0x55aa70ff9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55aa70ff4430 0x55aa71008880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fab400640 0 client.26927 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 --2- 172.21.15.138:0/2247517299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aa70ff3b10 0x55aa71008340 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 >> 172.21.15.138:0/2247517299 conn(0x55aa70ff1940 msgr2=0x55aa70ff6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 shutdown_connections 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 1 -- 172.21.15.138:0/2247517299 wait complete. 2024-08-20T01:20:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.903+0000 7f4fcf262740 20 client.26927 trim_cache size 0 max 16384 2024-08-20T01:20:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:20:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:20:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:20:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 msgr2=0x7f4fac0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 0x7f4fac0b6280 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4fc41049e0 tx=0x7f4fb0005f90 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 msgr2=0x7f4fc419aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800e9f0 tx=0x7f4fb800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 shutdown_connections 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4fac0b3dc0 0x7f4fac0b6280 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4fc4105970 0x7f4fc41a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fc4104fd0 0x7f4fc419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 --2- 172.21.15.138:0/1043795216 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fc406a7a0 0x7f4fc419aee0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 >> 172.21.15.138:0/1043795216 conn(0x7f4fc4077270 msgr2=0x7f4fc4105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 shutdown_connections 2024-08-20T01:20:05.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:05.904+0000 7f4fcac00640 1 -- 172.21.15.138:0/1043795216 wait complete. 2024-08-20T01:20:06.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 466952 3.61 MB/sec execute 498 sec latency 30.126 ms 2024-08-20T01:20:06.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:06 smithi167 ceph-mon[21965]: pgmap v732: 129 pgs: 129 active+clean; 376 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-08-20T01:20:06.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:06 smithi167 ceph-mon[21965]: osdmap e100: 12 total, 12 up, 12 in 2024-08-20T01:20:06.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:06 smithi142 ceph-mon[21799]: pgmap v732: 129 pgs: 129 active+clean; 376 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-08-20T01:20:06.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:06 smithi142 ceph-mon[21799]: osdmap e100: 12 total, 12 up, 12 in 2024-08-20T01:20:06.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:06 smithi138 ceph-mon[18076]: pgmap v732: 129 pgs: 129 active+clean; 376 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 274 KiB/s wr, 52 op/s 2024-08-20T01:20:06.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:06 smithi138 ceph-mon[18076]: osdmap e100: 12 total, 12 up, 12 in 2024-08-20T01:20:07.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 467138 3.61 MB/sec execute 499 sec latency 879.078 ms 2024-08-20T01:20:08.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 467138 3.60 MB/sec execute 500 sec latency 1879.204 ms 2024-08-20T01:20:08.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:08 smithi167 ceph-mon[21965]: pgmap v734: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:08.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:08 smithi142 ceph-mon[21799]: pgmap v734: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:08.540 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:08 smithi138 ceph-mon[18076]: pgmap v734: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:09.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 467138 3.59 MB/sec execute 501 sec latency 2879.324 ms 2024-08-20T01:20:10.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 469312 3.61 MB/sec execute 502 sec latency 2922.062 ms 2024-08-20T01:20:10.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:10 smithi167 ceph-mon[21965]: pgmap v735: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:10.467 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:10 smithi138 ceph-mon[18076]: pgmap v735: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:10.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:10 smithi142 ceph-mon[21799]: pgmap v735: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 381 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 1002 KiB/s wr, 121 op/s 2024-08-20T01:20:11.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 470685 3.61 MB/sec execute 503 sec latency 161.694 ms 2024-08-20T01:20:12.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 470685 3.61 MB/sec execute 504 sec latency 1161.774 ms 2024-08-20T01:20:12.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:12 smithi167 ceph-mon[21965]: pgmap v736: 129 pgs: 129 active+clean; 383 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-08-20T01:20:12.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:12 smithi142 ceph-mon[21799]: pgmap v736: 129 pgs: 129 active+clean; 383 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-08-20T01:20:12.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:12 smithi138 ceph-mon[18076]: pgmap v736: 129 pgs: 129 active+clean; 383 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-08-20T01:20:13.200 INFO:tasks.workunit.client.0.smithi138.stdout: 1 472538 3.62 MB/sec execute 505 sec latency 1526.888 ms 2024-08-20T01:20:14.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 473924 3.63 MB/sec execute 506 sec latency 22.236 ms 2024-08-20T01:20:14.463 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:14 smithi167 ceph-mon[21965]: pgmap v737: 129 pgs: 129 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-20T01:20:14.491 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:14 smithi142 ceph-mon[21799]: pgmap v737: 129 pgs: 129 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-20T01:20:14.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:14 smithi138 ceph-mon[18076]: pgmap v737: 129 pgs: 129 active+clean; 387 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-08-20T01:20:15.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 474231 3.62 MB/sec execute 507 sec latency 766.895 ms 2024-08-20T01:20:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:15 smithi167 ceph-mon[21965]: osdmap e101: 12 total, 12 up, 12 in 2024-08-20T01:20:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:15 smithi142 ceph-mon[21799]: osdmap e101: 12 total, 12 up, 12 in 2024-08-20T01:20:15.788 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:15 smithi138 ceph-mon[18076]: osdmap e101: 12 total, 12 up, 12 in 2024-08-20T01:20:16.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 474234 3.62 MB/sec execute 508 sec latency 1749.426 ms 2024-08-20T01:20:16.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:16 smithi138 ceph-mon[18076]: pgmap v739: 129 pgs: 129 active+clean; 388 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-20T01:20:16.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:16 smithi167 ceph-mon[21965]: pgmap v739: 129 pgs: 129 active+clean; 388 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-20T01:20:16.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:16 smithi142 ceph-mon[21799]: pgmap v739: 129 pgs: 129 active+clean; 388 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-08-20T01:20:17.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 476471 3.64 MB/sec execute 509 sec latency 28.333 ms 2024-08-20T01:20:18.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 477780 3.64 MB/sec execute 510 sec latency 228.970 ms 2024-08-20T01:20:18.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:18 smithi167 ceph-mon[21965]: pgmap v740: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:18.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:18 smithi142 ceph-mon[21799]: pgmap v740: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:18.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:18 smithi138 ceph-mon[18076]: pgmap v740: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 -- 172.21.15.138:0/3619020346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 msgr2=0x7f568c10a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 --2- 172.21.15.138:0/3619020346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c10a850 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5678009900 tx=0x7f567802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 -- 172.21.15.138:0/3619020346 shutdown_connections 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 --2- 172.21.15.138:0/3619020346 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c10ad90 0x7f568c111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 --2- 172.21.15.138:0/3619020346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c10a850 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 --2- 172.21.15.138:0/3619020346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c104a10 0x7f568c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 -- 172.21.15.138:0/3619020346 >> 172.21.15.138:0/3619020346 conn(0x7f568c100410 msgr2=0x7f568c102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 -- 172.21.15.138:0/3619020346 shutdown_connections 2024-08-20T01:20:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.018+0000 7f568b200640 1 -- 172.21.15.138:0/3619020346 wait complete. 2024-08-20T01:20:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.019+0000 7f568b200640 1 Processor -- start 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.019+0000 7f568b200640 1 -- start start 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.019+0000 7f568b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c104a10 0x7f568c19af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.019+0000 7f568b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.019+0000 7f568b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c10ad90 0x7f568c1a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f568b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c114a10 con 0x7f568c10a410 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f568b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f568c114890 con 0x7f568c10ad90 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f568b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f568c114b90 con 0x7f568c104a10 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39398/0 (socket says 172.21.15.138:39398) 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 -- 172.21.15.138:0/2797944997 learned_addr learned my addr 172.21.15.138:0/2797944997 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689e00640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c104a10 0x7f568c19af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f568a800640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c10ad90 0x7f568c1a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 -- 172.21.15.138:0/2797944997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c104a10 msgr2=0x7f568c19af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c104a10 0x7f568c19af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 -- 172.21.15.138:0/2797944997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c10ad90 msgr2=0x7f568c1a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c10ad90 0x7f568c1a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.020+0000 7f5689400640 1 -- 172.21.15.138:0/2797944997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c1a0fb0 con 0x7f568c10a410 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f5689400640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5678005940 tx=0x7f56780040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f5682a00640 1 -- 172.21.15.138:0/2797944997 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567803f030 con 0x7f568c10a410 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f5682a00640 1 -- 172.21.15.138:0/2797944997 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56780044c0 con 0x7f568c10a410 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c1a1240 con 0x7f568c10a410 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f5682a00640 1 -- 172.21.15.138:0/2797944997 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5678035070 con 0x7f568c10a410 2024-08-20T01:20:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.021+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1a7960 con 0x7f568c10a410 2024-08-20T01:20:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 Processor -- start 2024-08-20T01:20:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 -- start start 2024-08-20T01:20:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5682a00640 1 -- 172.21.15.138:0/2797944997 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5678042030 con 0x7f568c10a410 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c82f325210 con 0x55c82f3b0430 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c82f314710 con 0x55c82f3bc3c0 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.024+0000 7f5693c29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c82f318750 con 0x55c82f3afb10 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38880/0 (socket says 172.21.15.138:38880) 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 -- 172.21.15.138:0/1292758040 learned_addr learned my addr 172.21.15.138:0/1292758040 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f5689e00640 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f5689400640 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 -- 172.21.15.138:0/1292758040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 msgr2=0x55c82f3be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 -- 172.21.15.138:0/1292758040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 msgr2=0x55c82f3bbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 -- 172.21.15.138:0/1292758040 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82f2e0de0 con 0x55c82f3afb10 2024-08-20T01:20:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f568a800640 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f56840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a0516a480380cc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f5677400640 1 -- 172.21.15.138:0/1292758040 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684004d20 con 0x55c82f3afb10 2024-08-20T01:20:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f5677400640 1 -- 172.21.15.138:0/1292758040 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5684004050 con 0x55c82f3afb10 2024-08-20T01:20:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.025+0000 7f5677400640 1 -- 172.21.15.138:0/1292758040 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684005280 con 0x55c82f3afb10 2024-08-20T01:20:19.029 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5682a00640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 0x7f56700b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5682a00640 1 -- 172.21.15.138:0/2797944997 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f56781046b0 con 0x7f568c10a410 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 -- 172.21.15.138:0/1292758040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 msgr2=0x55c82f3afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f56840118d0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5689e00640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 0x7f56700b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 -- 172.21.15.138:0/1292758040 shutdown_connections 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 --2- 172.21.15.138:0/1292758040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3afef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 -- 172.21.15.138:0/1292758040 >> 172.21.15.138:0/1292758040 conn(0x55c82f3ad940 msgr2=0x55c82f3aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 -- 172.21.15.138:0/1292758040 shutdown_connections 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.026+0000 7f5693c29740 1 -- 172.21.15.138:0/1292758040 wait complete. 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5689e00640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 0x7f56700b6280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c82f3b2c90 tx=0x7f567c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 -- start start 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 Processor -- start 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 0x55c82f3b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c82f24dab0 con 0x55c82f3afb10 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c82f2bb140 con 0x55c82f3b0430 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c82f3bf0a0 con 0x55c82f3bc3c0 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.027+0000 7f5693c29740 10 client.0 _ll_register_callbacks cb 0x55c82f2ce5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39422/0 (socket says 172.21.15.138:39422) 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 -- 172.21.15.138:0/1719678547 learned_addr learned my addr 172.21.15.138:0/1719678547 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689400640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 0x55c82f3b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f568a800640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 -- 172.21.15.138:0/1719678547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3bc3c0 msgr2=0x55c82f3b9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 -- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 msgr2=0x55c82f3b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 0x55c82f3b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 -- 172.21.15.138:0/1719678547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82f2bc170 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689400640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 0x55c82f3b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f5689e00640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f567c005f60 tx=0x7f567c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2057e81a552f09fd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.028+0000 7f566be00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c005020 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684004050 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f567c0051c0 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f567c0053c0 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 10 client.16893 fetch_fsmap learned FSMap version 42 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c82f077790 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5684004050 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f567c027070 con 0x55c82f3afb10 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 10 client.16893 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T01:20:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 20 client.16893 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f3cbf80 0x55c82f3ce390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 4 client.16893 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.029+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c82f327160 con 0x55c82f3cbf80 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.030+0000 7f5689400640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f3cbf80 0x55c82f3ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.030+0000 7f5689400640 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f3cbf80 0x55c82f3ce390 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be460f86da7686 server_cookie=9cddec85e8315d2f in_seq=0 out_seq=0 2024-08-20T01:20:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.030+0000 7f566be00640 10 client.16893 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f566be00640 1 -- 172.21.15.138:0/1719678547 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c82f327160 con 0x55c82f3cbf80 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f566be00640 10 client.16893 handle_command_reply: tid=0 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 client.16893 shutdown 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 msgr2=0x55c82f3bbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f567c005f60 tx=0x7f567c004730 comp rx=0 tx=0).stop 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 shutdown_connections 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f3cbf80 0x55c82f3ce390 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3b0430 0x55c82f3b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f566be00640 0 client.16893 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 --2- 172.21.15.138:0/1719678547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3bbb00 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 >> 172.21.15.138:0/1719678547 conn(0x55c82f3ad940 msgr2=0x55c82f3b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 shutdown_connections 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 1 -- 172.21.15.138:0/1719678547 wait complete. 2024-08-20T01:20:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.031+0000 7f5693c29740 20 client.16893 trim_cache size 0 max 16384 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.046+0000 7f5693c29740 1 Processor -- start 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.046+0000 7f5693c29740 1 -- start start 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.046+0000 7f5693c29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5693c29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3b0430 0x55c82f3d41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5693c29740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5693c29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b4400 con 0x55c82f3afb10 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5693c29740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b1ca0 con 0x55c82f3bc3c0 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5693c29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1afb10 con 0x55c82f3b0430 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5689e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5689e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39438/0 (socket says 172.21.15.138:39438) 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5689e00640 1 -- 172.21.15.138:0/523431444 learned_addr learned my addr 172.21.15.138:0/523431444 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5689400640 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3b0430 0x55c82f3d41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 -- 172.21.15.138:0/523431444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3b0430 msgr2=0x55c82f3d41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3b0430 0x55c82f3d41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 -- 172.21.15.138:0/523431444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 msgr2=0x55c82f3d3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 -- 172.21.15.138:0/523431444 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c82f0f0620 con 0x55c82f3bc3c0 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f5689e00640 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.047+0000 7f568a800640 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b3e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5684004970 tx=0x7f5684009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2de97183bac4d50 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5676a00640 1 -- 172.21.15.138:0/523431444 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684003a30 con 0x55c82f3bc3c0 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5676a00640 1 -- 172.21.15.138:0/523431444 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5684004e50 con 0x55c82f3bc3c0 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5676a00640 1 -- 172.21.15.138:0/523431444 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684005020 con 0x55c82f3bc3c0 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 -- 172.21.15.138:0/523431444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 msgr2=0x55c82f3b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b3e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5684004970 tx=0x7f5684009b30 comp rx=0 tx=0).stop 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 -- 172.21.15.138:0/523431444 shutdown_connections 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3b3e10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3b0430 0x55c82f3d41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 --2- 172.21.15.138:0/523431444 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3afb10 0x55c82f3d3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 -- 172.21.15.138:0/523431444 >> 172.21.15.138:0/523431444 conn(0x55c82f3ad940 msgr2=0x55c82f3b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.048+0000 7f5693c29740 1 -- 172.21.15.138:0/523431444 shutdown_connections 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 -- 172.21.15.138:0/523431444 wait complete. 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 -- start start 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 Processor -- start 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.052 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b0160 con 0x55c82f3b0430 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b15d0 con 0x55c82f3bc3c0 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f568c1b0830 con 0x55c82f3afb10 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.049+0000 7f5693c29740 10 client.0 _ll_register_callbacks cb 0x7f567c009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38904/0 (socket says 172.21.15.138:38904) 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 -- 172.21.15.138:0/2389300798 learned_addr learned my addr 172.21.15.138:0/2389300798 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f568a800640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 -- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 msgr2=0x55c82f3bd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3bd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 -- 172.21.15.138:0/2389300798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 msgr2=0x55c82f3bd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 -- 172.21.15.138:0/2389300798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c82f3bdac0 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689400640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f568a800640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f5689e00640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f567c005f60 tx=0x7f567c004450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=391425874f07ed84 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.050+0000 7f566be00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c036640 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5684004e50 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f567c0367e0 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f567c0369e0 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 10 client.26902 fetch_fsmap learned FSMap version 42 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c82f3c4120 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5684004e50 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f567c027050 con 0x55c82f3afb10 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 10 client.26902 fetch_fsmap finished waiting for FSMap version 42 2024-08-20T01:20:19.054 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 20 client.26902 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f437210 0x55c82f439630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 4 client.26902 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.051+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c82f322c00 con 0x55c82f437210 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.052+0000 7f5689400640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f437210 0x55c82f439630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.052+0000 7f5689400640 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f437210 0x55c82f439630 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a26f3dd3e0d957a server_cookie=afdbed0c5fe2acb1 in_seq=0 out_seq=0 2024-08-20T01:20:19.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.052+0000 7f566be00640 10 client.26902 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f566be00640 1 -- 172.21.15.138:0/2389300798 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c82f322c00 con 0x55c82f437210 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f566be00640 10 client.26902 handle_command_reply: tid=0 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 client.26902 shutdown 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 msgr2=0x55c82f3c4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f567c005f60 tx=0x7f567c004450 comp rx=0 tx=0).stop 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 shutdown_connections 2024-08-20T01:20:19.100 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c82f437210 0x55c82f439630 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c82f3bc3c0 0x55c82f3bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c82f3b0430 0x55c82f3bd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 --2- 172.21.15.138:0/2389300798 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c82f3afb10 0x55c82f3c4760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f566be00640 0 client.26902 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 >> 172.21.15.138:0/2389300798 conn(0x55c82f3ad940 msgr2=0x55c82f3ba570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 shutdown_connections 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 1 -- 172.21.15.138:0/2389300798 wait complete. 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.100+0000 7f5693c29740 20 client.26902 trim_cache size 0 max 16384 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:20:19.101 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 msgr2=0x7f56700b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 0x7f56700b6280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c82f3b2c90 tx=0x7f567c005f90 comp rx=0 tx=0).stop 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 msgr2=0x7f568c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5678005940 tx=0x7f56780040c0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 shutdown_connections 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f56700b3dc0 0x7f56700b6280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f568c10ad90 0x7f568c1a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f568c10a410 0x7f568c19b450 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 --2- 172.21.15.138:0/2797944997 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f568c104a10 0x7f568c19af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 >> 172.21.15.138:0/2797944997 conn(0x7f568c100410 msgr2=0x7f568c102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 shutdown_connections 2024-08-20T01:20:19.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:19.101+0000 7f568b200640 1 -- 172.21.15.138:0/2797944997 wait complete. 2024-08-20T01:20:19.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 478068 3.63 MB/sec execute 511 sec latency 1121.375 ms 2024-08-20T01:20:20.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 480136 3.65 MB/sec execute 512 sec latency 12.458 ms 2024-08-20T01:20:20.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:20 smithi167 ceph-mon[21965]: pgmap v741: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:20 smithi142 ceph-mon[21799]: pgmap v741: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:20.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:20 smithi138 ceph-mon[18076]: pgmap v741: 129 pgs: 129 active+clean; 393 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 107 op/s 2024-08-20T01:20:21.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 481360 3.66 MB/sec execute 513 sec latency 346.955 ms 2024-08-20T01:20:22.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 481360 3.65 MB/sec execute 514 sec latency 1347.059 ms 2024-08-20T01:20:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:22 smithi167 ceph-mon[21965]: pgmap v742: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-20T01:20:22.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:22 smithi142 ceph-mon[21799]: pgmap v742: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-20T01:20:22.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:22 smithi138 ceph-mon[18076]: pgmap v742: 129 pgs: 129 active+clean; 401 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 137 op/s 2024-08-20T01:20:23.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 481360 3.64 MB/sec execute 515 sec latency 2347.164 ms 2024-08-20T01:20:24.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 481360 3.63 MB/sec execute 516 sec latency 3347.213 ms 2024-08-20T01:20:24.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:24 smithi167 ceph-mon[21965]: pgmap v743: 129 pgs: 129 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-08-20T01:20:24.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:24 smithi142 ceph-mon[21799]: pgmap v743: 129 pgs: 129 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-08-20T01:20:24.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:24 smithi138 ceph-mon[18076]: pgmap v743: 129 pgs: 129 active+clean; 402 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 123 op/s 2024-08-20T01:20:25.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 483537 3.65 MB/sec execute 517 sec latency 3386.591 ms 2024-08-20T01:20:26.201 INFO:tasks.workunit.client.0.smithi138.stdout: 1 484995 3.66 MB/sec execute 518 sec latency 51.241 ms 2024-08-20T01:20:26.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:26 smithi167 ceph-mon[21965]: pgmap v744: 129 pgs: 129 active+clean; 405 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-08-20T01:20:26.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:26 smithi142 ceph-mon[21799]: pgmap v744: 129 pgs: 129 active+clean; 405 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-08-20T01:20:26.790 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:26 smithi138 ceph-mon[18076]: pgmap v744: 129 pgs: 129 active+clean; 405 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 142 op/s 2024-08-20T01:20:27.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 487135 3.68 MB/sec execute 519 sec latency 16.582 ms 2024-08-20T01:20:27.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:27 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:27.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:27 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:27.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:27 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:20:28.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 487829 3.67 MB/sec execute 520 sec latency 451.028 ms 2024-08-20T01:20:28.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:28 smithi167 ceph-mon[21965]: pgmap v745: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-08-20T01:20:28.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:28 smithi142 ceph-mon[21799]: pgmap v745: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-08-20T01:20:28.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:28 smithi138 ceph-mon[18076]: pgmap v745: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 130 op/s 2024-08-20T01:20:29.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 487829 3.67 MB/sec execute 521 sec latency 1451.160 ms 2024-08-20T01:20:30.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 488453 3.67 MB/sec execute 522 sec latency 1490.776 ms 2024-08-20T01:20:30.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:30 smithi167 ceph-mon[21965]: pgmap v746: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-20T01:20:30.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:30 smithi142 ceph-mon[21799]: pgmap v746: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-20T01:20:30.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:30 smithi138 ceph-mon[18076]: pgmap v746: 129 pgs: 129 active+clean; 406 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.1 MiB/s wr, 95 op/s 2024-08-20T01:20:31.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 488453 3.66 MB/sec execute 523 sec latency 1560.767 ms 2024-08-20T01:20:32.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 488453 3.65 MB/sec execute 524 sec latency 2560.855 ms 2024-08-20T01:20:32.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 -- 172.21.15.138:0/3432563375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c1083f0 msgr2=0x7f893c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 --2- 172.21.15.138:0/3432563375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c1083f0 0x7f893c108830 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8928009900 tx=0x7f892802a410 comp rx=0 tx=0).stop 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 -- 172.21.15.138:0/3432563375 shutdown_connections 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 --2- 172.21.15.138:0/3432563375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c108d70 0x7f893c10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 --2- 172.21.15.138:0/3432563375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c1083f0 0x7f893c108830 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 --2- 172.21.15.138:0/3432563375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 -- 172.21.15.138:0/3432563375 >> 172.21.15.138:0/3432563375 conn(0x7f893c0fe430 msgr2=0x7f893c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 -- 172.21.15.138:0/3432563375 shutdown_connections 2024-08-20T01:20:32.221 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.221+0000 7f8944400640 1 -- 172.21.15.138:0/3432563375 wait complete. 2024-08-20T01:20:32.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 Processor -- start 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 -- start start 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c19b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c1083f0 0x7f893c19b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f893c112a80 con 0x7f893c108d70 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f893c112900 con 0x7f893c1083f0 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.222+0000 7f8944400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f893c112c00 con 0x7f893c1029f0 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8941a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c19b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57568/0 (socket says 172.21.15.138:57568) 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 -- 172.21.15.138:0/3553934462 learned_addr learned my addr 172.21.15.138:0/3553934462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8941000640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c1083f0 0x7f893c19b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 -- 172.21.15.138:0/3553934462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 msgr2=0x7f893c19b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c19b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 -- 172.21.15.138:0/3553934462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c1083f0 msgr2=0x7f893c19b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c1083f0 0x7f893c19b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 -- 172.21.15.138:0/3553934462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c1a11e0 con 0x7f893c108d70 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8941a00640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c19b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.223+0000 7f8942400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f893400efd0 tx=0x7f893400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.224+0000 7f893a000640 1 -- 172.21.15.138:0/3553934462 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934003b70 con 0x7f893c108d70 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.224+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1a14d0 con 0x7f893c108d70 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.224+0000 7f893a000640 1 -- 172.21.15.138:0/3553934462 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8934003d10 con 0x7f893c108d70 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.224+0000 7f893a000640 1 -- 172.21.15.138:0/3553934462 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340108a0 con 0x7f893c108d70 2024-08-20T01:20:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.224+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c1a7b90 con 0x7f893c108d70 2024-08-20T01:20:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.227+0000 7f894743e740 1 Processor -- start 2024-08-20T01:20:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.227+0000 7f893a000640 1 -- 172.21.15.138:0/3553934462 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8934004020 con 0x7f893c108d70 2024-08-20T01:20:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.227+0000 7f894743e740 1 -- start start 2024-08-20T01:20:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.227+0000 7f894743e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x564758395d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.227+0000 7f894743e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f894743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x5647583a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f894743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56475830b210 con 0x5647583a21e0 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f894743e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5647582fa710 con 0x564758396250 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f894743e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647582fe750 con 0x564758395930 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x564758395d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x564758395d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55154/0 (socket says 172.21.15.138:55154) 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941000640 1 -- 172.21.15.138:0/1627372159 learned_addr learned my addr 172.21.15.138:0/1627372159 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941000640 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x5647583a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 -- 172.21.15.138:0/1627372159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 msgr2=0x564758395d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x564758395d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 -- 172.21.15.138:0/1627372159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 msgr2=0x5647583a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x5647583a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.228+0000 7f8941a00640 1 -- 172.21.15.138:0/1627372159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647582c6de0 con 0x564758396250 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f8941a00640 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a1ca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8930009fd0 tx=0x7f89300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d22ec594b0a7de9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f893a000640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 0x7f89240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f892ea00640 1 -- 172.21.15.138:0/1627372159 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893001e070 con 0x564758396250 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f892ea00640 1 -- 172.21.15.138:0/1627372159 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8930004310 con 0x564758396250 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f892ea00640 1 -- 172.21.15.138:0/1627372159 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893000f7f0 con 0x564758396250 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f893a000640 1 -- 172.21.15.138:0/3553934462 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f89340e4830 con 0x7f893c108d70 2024-08-20T01:20:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.229+0000 7f8941000640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 0x7f89240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- 172.21.15.138:0/1627372159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 msgr2=0x5647583a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a1ca0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8930009fd0 tx=0x7f89300118d0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f8941000640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 0x7f89240b6200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f8928005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- 172.21.15.138:0/1627372159 shutdown_connections 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x5647583a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a1ca0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 --2- 172.21.15.138:0/1627372159 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x564758395d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- 172.21.15.138:0/1627372159 >> 172.21.15.138:0/1627372159 conn(0x564758393760 msgr2=0x564758394900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- 172.21.15.138:0/1627372159 shutdown_connections 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- 172.21.15.138:0/1627372159 wait complete. 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.230+0000 7f894743e740 1 -- start start 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 Processor -- start 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758396250 0x56475839ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 0x56475839f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x564758233ab0 con 0x564758395930 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5647582a1140 con 0x5647583a21e0 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5647583a4ec0 con 0x564758396250 2024-08-20T01:20:32.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f894743e740 10 client.0 _ll_register_callbacks cb 0x5647582b45d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f8941a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f8941a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57594/0 (socket says 172.21.15.138:57594) 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f8941a00640 1 -- 172.21.15.138:0/2285427323 learned_addr learned my addr 172.21.15.138:0/2285427323 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f8942400640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 0x56475839f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.231+0000 7f8941000640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758396250 0x56475839ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 -- 172.21.15.138:0/2285427323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758396250 msgr2=0x56475839ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758396250 0x56475839ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 -- 172.21.15.138:0/2285427323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 msgr2=0x5647583a1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 -- 172.21.15.138:0/2285427323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647582a2170 con 0x5647583a21e0 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8941a00640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:32.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8942400640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 0x56475839f390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89340e1b80 tx=0x7f89340aaf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ff9706be2a225ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8923400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89340ab040 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8930004310 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f89340a72d0 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.232+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f89340cb050 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 10 client.26957 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56475805d790 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8930004310 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f893401f020 con 0x5647583a21e0 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 10 client.26957 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 20 client.26957 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5647583b1ea0 0x5647583b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 4 client.26957 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56475830d160 con 0x5647583b1ea0 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.233+0000 7f8941a00640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5647583b1ea0 0x5647583b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.234+0000 7f8941a00640 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5647583b1ea0 0x5647583b4290 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df3f44d2e885537a server_cookie=c05fd4aa2589ccf5 in_seq=0 out_seq=0 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.234+0000 7f8923400640 10 client.26957 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f8923400640 1 -- 172.21.15.138:0/2285427323 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56475830d160 con 0x5647583b1ea0 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f8923400640 10 client.26957 handle_command_reply: tid=0 2024-08-20T01:20:32.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 client.26957 shutdown 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 msgr2=0x56475839f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 0x56475839f390 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f89340e1b80 tx=0x7f89340aaf20 comp rx=0 tx=0).stop 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 shutdown_connections 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5647583b1ea0 0x5647583b4290 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647583a21e0 0x56475839f390 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758396250 0x56475839ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 --2- 172.21.15.138:0/2285427323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583a1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f8923400640 0 client.26957 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:32.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 >> 172.21.15.138:0/2285427323 conn(0x564758393760 msgr2=0x564758399300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 shutdown_connections 2024-08-20T01:20:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 1 -- 172.21.15.138:0/2285427323 wait complete. 2024-08-20T01:20:32.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.235+0000 7f894743e740 20 client.26957 trim_cache size 0 max 16384 2024-08-20T01:20:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 Processor -- start 2024-08-20T01:20:32.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 -- start start 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583ba000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647583a21e0 0x5647583aa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b4510 con 0x564758395930 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b1db0 con 0x564758396250 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.250+0000 7f894743e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1afc20 con 0x5647583a21e0 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57602/0 (socket says 172.21.15.138:57602) 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 -- 172.21.15.138:0/2513141633 learned_addr learned my addr 172.21.15.138:0/2513141633 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941000640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583ba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8942400640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647583a21e0 0x5647583aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 -- 172.21.15.138:0/2513141633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647583a21e0 msgr2=0x5647583aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647583a21e0 0x5647583aa170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 -- 172.21.15.138:0/2513141633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 msgr2=0x5647583ba000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583ba000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 -- 172.21.15.138:0/2513141633 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647580d6620 con 0x564758395930 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941000640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:32.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f8941a00640 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8930011dd0 tx=0x7f89300056c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ff8455f0970cc9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f892e000640 1 -- 172.21.15.138:0/2513141633 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893001e040 con 0x564758395930 2024-08-20T01:20:32.255 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f892e000640 1 -- 172.21.15.138:0/2513141633 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8930009b30 con 0x564758395930 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.251+0000 7f892e000640 1 -- 172.21.15.138:0/2513141633 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893000fae0 con 0x564758395930 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- 172.21.15.138:0/2513141633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 msgr2=0x5647583b9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8930011dd0 tx=0x7f89300056c0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- 172.21.15.138:0/2513141633 shutdown_connections 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5647583a21e0 0x5647583aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583ba000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 --2- 172.21.15.138:0/2513141633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x564758395930 0x5647583b9ac0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- 172.21.15.138:0/2513141633 >> 172.21.15.138:0/2513141633 conn(0x564758393760 msgr2=0x564758394890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- 172.21.15.138:0/2513141633 shutdown_connections 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- 172.21.15.138:0/2513141633 wait complete. 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.252+0000 7f894743e740 1 -- start start 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 Processor -- start 2024-08-20T01:20:32.256 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x564758414c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b0270 con 0x5647583a21e0 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b16e0 con 0x564758396250 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f893c1b0940 con 0x564758395930 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f894743e740 10 client.0 _ll_register_callbacks cb 0x7f8934004830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f8941a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f8941a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55184/0 (socket says 172.21.15.138:55184) 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f8942400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x564758414c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.253+0000 7f8941a00640 1 -- 172.21.15.138:0/4274860666 learned_addr learned my addr 172.21.15.138:0/4274860666 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941000640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 -- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 msgr2=0x5647583a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 -- 172.21.15.138:0/4274860666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 msgr2=0x564758414c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:32.257 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x564758414c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 -- 172.21.15.138:0/4274860666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564758415390 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8942400640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x564758414c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8941a00640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8930009f60 tx=0x7f8930009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18f89a826f27c11 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8923400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893001e040 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f893000f970 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.254+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8930009b30 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f893000fb70 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 10 client.26920 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56475841c820 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8930009b30 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f8930038550 con 0x564758395930 2024-08-20T01:20:32.258 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 10 client.26920 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 10 client.26920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 20 client.26920 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56475841cd50 0x56475841f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 4 client.26920 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.255+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564758308c00 con 0x56475841cd50 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.256+0000 7f8941000640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56475841cd50 0x56475841f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.256+0000 7f8923400640 10 client.26920 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:32.259 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.256+0000 7f8941000640 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56475841cd50 0x56475841f170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc192eccc23d3c64 server_cookie=ae95c89f1cf81b3b in_seq=0 out_seq=0 2024-08-20T01:20:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f8923400640 1 -- 172.21.15.138:0/4274860666 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564758308c00 con 0x56475841cd50 2024-08-20T01:20:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f8923400640 10 client.26920 handle_command_reply: tid=0 2024-08-20T01:20:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 client.26920 shutdown 2024-08-20T01:20:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 msgr2=0x5647583a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8930009f60 tx=0x7f8930009f90 comp rx=0 tx=0).stop 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 shutdown_connections 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56475841cd50 0x56475841f170 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5647583a21e0 0x564758414c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564758396250 0x5647583a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 --2- 172.21.15.138:0/4274860666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x564758395930 0x5647583a7bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f8923400640 0 client.26920 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 >> 172.21.15.138:0/4274860666 conn(0x564758393760 msgr2=0x5647583a0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 shutdown_connections 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 1 -- 172.21.15.138:0/4274860666 wait complete. 2024-08-20T01:20:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.310+0000 7f894743e740 20 client.26920 trim_cache size 0 max 16384 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 msgr2=0x7f89240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 0x7f89240b6200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8928000c00 tx=0x7f8928005a20 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 msgr2=0x7f893c1a0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f893400efd0 tx=0x7f893400c6a0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 shutdown_connections 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f89240b3d40 0x7f89240b6200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f893c108d70 0x7f893c1a0a10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f893c1083f0 0x7f893c19b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 --2- 172.21.15.138:0/3553934462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f893c1029f0 0x7f893c19b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 >> 172.21.15.138:0/3553934462 conn(0x7f893c0fe430 msgr2=0x7f893c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 shutdown_connections 2024-08-20T01:20:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:32.311+0000 7f8944400640 1 -- 172.21.15.138:0/3553934462 wait complete. 2024-08-20T01:20:32.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:32 smithi167 ceph-mon[21965]: pgmap v747: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-20T01:20:32.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:32 smithi142 ceph-mon[21799]: pgmap v747: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-20T01:20:32.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:32 smithi138 ceph-mon[18076]: pgmap v747: 129 pgs: 129 active+clean; 414 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-08-20T01:20:33.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 490569 3.67 MB/sec execute 525 sec latency 2646.462 ms 2024-08-20T01:20:34.202 INFO:tasks.workunit.client.0.smithi138.stdout: 1 491377 3.67 MB/sec execute 526 sec latency 341.458 ms 2024-08-20T01:20:34.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:34 smithi167 ceph-mon[21965]: pgmap v748: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-08-20T01:20:34.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:34 smithi142 ceph-mon[21799]: pgmap v748: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-08-20T01:20:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:34 smithi138 ceph-mon[18076]: pgmap v748: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-08-20T01:20:35.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 492000 3.67 MB/sec execute 527 sec latency 596.326 ms 2024-08-20T01:20:36.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 492000 3.66 MB/sec execute 528 sec latency 1596.399 ms 2024-08-20T01:20:36.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:36 smithi138 ceph-mon[18076]: pgmap v749: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-08-20T01:20:36.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:36 smithi167 ceph-mon[21965]: pgmap v749: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-08-20T01:20:36.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:36 smithi142 ceph-mon[21799]: pgmap v749: 129 pgs: 129 active+clean; 415 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-08-20T01:20:37.208 INFO:tasks.workunit.client.0.smithi138.stdout: 1 493530 3.67 MB/sec execute 529 sec latency 1746.832 ms 2024-08-20T01:20:38.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 494890 3.68 MB/sec execute 530 sec latency 14.813 ms 2024-08-20T01:20:38.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:38 smithi167 ceph-mon[21965]: pgmap v750: 129 pgs: 129 active+clean; 419 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-20T01:20:38.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:38 smithi142 ceph-mon[21799]: pgmap v750: 129 pgs: 129 active+clean; 419 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-20T01:20:38.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:38 smithi138 ceph-mon[18076]: pgmap v750: 129 pgs: 129 active+clean; 419 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-08-20T01:20:39.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 495544 3.68 MB/sec execute 531 sec latency 604.919 ms 2024-08-20T01:20:40.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 497723 3.69 MB/sec execute 532 sec latency 641.987 ms 2024-08-20T01:20:40.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:40 smithi167 ceph-mon[21965]: pgmap v751: 129 pgs: 129 active+clean; 423 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:20:40.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:40 smithi142 ceph-mon[21799]: pgmap v751: 129 pgs: 129 active+clean; 423 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:20:40.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:40 smithi138 ceph-mon[18076]: pgmap v751: 129 pgs: 129 active+clean; 423 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 127 op/s 2024-08-20T01:20:41.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 498919 3.70 MB/sec execute 533 sec latency 131.106 ms 2024-08-20T01:20:42.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 499092 3.69 MB/sec execute 534 sec latency 808.238 ms 2024-08-20T01:20:42.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:42 smithi142 ceph-mon[21799]: pgmap v752: 129 pgs: 129 active+clean; 428 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-08-20T01:20:42.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:42 smithi138 ceph-mon[18076]: pgmap v752: 129 pgs: 129 active+clean; 428 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-08-20T01:20:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:42 smithi167 ceph-mon[21965]: pgmap v752: 129 pgs: 129 active+clean; 428 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-08-20T01:20:43.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 499092 3.68 MB/sec execute 535 sec latency 1212.869 ms 2024-08-20T01:20:43.986 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 17.3 2024-08-20T01:20:43.987 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:20:44.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 499092 3.68 MB/sec execute 536 sec latency 2212.941 ms 2024-08-20T01:20:44.389 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:20:44.725 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:44 smithi138 ceph-mon[18076]: pgmap v753: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-08-20T01:20:44.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:44 smithi142 ceph-mon[21799]: pgmap v753: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-08-20T01:20:44.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:44 smithi167 ceph-mon[21965]: pgmap v753: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 138 op/s 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 -- 172.21.15.138:0/1542361502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410aea0 msgr2=0x7f0c841116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 --2- 172.21.15.138:0/1542361502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410aea0 0x7f0c841116a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009ca0 tx=0x7f0c7c02a8b0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 -- 172.21.15.138:0/1542361502 shutdown_connections 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 --2- 172.21.15.138:0/1542361502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410aea0 0x7f0c841116a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 --2- 172.21.15.138:0/1542361502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410a520 0x7f0c8410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 --2- 172.21.15.138:0/1542361502 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c84104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.017+0000 7f0c88a00640 1 -- 172.21.15.138:0/1542361502 >> 172.21.15.138:0/1542361502 conn(0x7f0c84100540 msgr2=0x7f0c84102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.018+0000 7f0c88a00640 1 -- 172.21.15.138:0/1542361502 shutdown_connections 2024-08-20T01:20:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.018+0000 7f0c88a00640 1 -- 172.21.15.138:0/1542361502 wait complete. 2024-08-20T01:20:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.018+0000 7f0c88a00640 1 Processor -- start 2024-08-20T01:20:45.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 -- start start 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c8419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410a520 0x7f0c8419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84114b60 con 0x7f0c8410a520 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0c841149e0 con 0x7f0c8410aea0 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c88a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0c84114ce0 con 0x7f0c84104b10 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c83200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.019+0000 7f0c82800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c8419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33338/0 (socket says 172.21.15.138:33338) 2024-08-20T01:20:45.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 -- 172.21.15.138:0/2418696772 learned_addr learned my addr 172.21.15.138:0/2418696772 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c81e00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410a520 0x7f0c8419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 -- 172.21.15.138:0/2418696772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 msgr2=0x7f0c8419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c8419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 -- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410a520 msgr2=0x7f0c8419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410a520 0x7f0c8419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 -- 172.21.15.138:0/2418696772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c841a0fe0 con 0x7f0c8410aea0 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c82800640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c8419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:45.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.020+0000 7f0c83200640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c005cc0 tx=0x7f0c7c030db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.021+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c003e00 con 0x7f0c8410aea0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.021+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c7c040070 con 0x7f0c8410aea0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.021+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c841a1270 con 0x7f0c8410aea0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.021+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c045a60 con 0x7f0c8410aea0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.021+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c841a7990 con 0x7f0c8410aea0 2024-08-20T01:20:45.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.022+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c84104ef0 con 0x7f0c8410aea0 2024-08-20T01:20:45.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.024+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0c7c045c20 con 0x7f0c8410aea0 2024-08-20T01:20:45.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.026+0000 7f0c7b400640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 0x7f0c680b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.026+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f0c7c1056d0 con 0x7f0c8410aea0 2024-08-20T01:20:45.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.026+0000 7f0c82800640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 0x7f0c680b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.026 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.027+0000 7f0c82800640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 0x7f0c680b62a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009fc0 tx=0x7f0c74005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.027 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.028+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f0c7c0cc6f0 con 0x7f0c8410aea0 2024-08-20T01:20:45.203 INFO:tasks.workunit.client.0.smithi138.stdout: 1 501254 3.70 MB/sec execute 537 sec latency 2258.598 ms 2024-08-20T01:20:45.302 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.303+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c8419c060 con 0x7f0c8410aea0 2024-08-20T01:20:45.303 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.304+0000 7f0c7b400640 1 -- 172.21.15.138:0/2418696772 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11934 (secure 0 0 0) 0x7f0c8419c060 con 0x7f0c8410aea0 2024-08-20T01:20:45.303 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:20:45.304 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":43,"btime":"2024-08-20T01:20:26:427904+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:20:26.156076+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:20:45.304 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 43 2024-08-20T01:20:45.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 msgr2=0x7f0c680b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 0x7f0c680b62a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0c74009fc0 tx=0x7f0c74005f90 comp rx=0 tx=0).stop 2024-08-20T01:20:45.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 msgr2=0x7f0c841a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c005cc0 tx=0x7f0c7c030db0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.306 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 shutdown_connections 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0c680b3de0 0x7f0c680b62a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c8410aea0 0x7f0c841a0810 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c8410a520 0x7f0c8419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 --2- 172.21.15.138:0/2418696772 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0c84104b10 0x7f0c8419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 >> 172.21.15.138:0/2418696772 conn(0x7f0c84100540 msgr2=0x7f0c84106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 shutdown_connections 2024-08-20T01:20:45.307 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:45.307+0000 7f0c88a00640 1 -- 172.21.15.138:0/2418696772 wait complete. 2024-08-20T01:20:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 -- 172.21.15.138:0/2517200648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1083f0 msgr2=0x7fed9c108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 --2- 172.21.15.138:0/2517200648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1083f0 0x7fed9c108830 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fed88009900 tx=0x7fed8802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 -- 172.21.15.138:0/2517200648 shutdown_connections 2024-08-20T01:20:45.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 --2- 172.21.15.138:0/2517200648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 --2- 172.21.15.138:0/2517200648 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1083f0 0x7fed9c108830 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 --2- 172.21.15.138:0/2517200648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1029f0 0x7fed9c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 -- 172.21.15.138:0/2517200648 >> 172.21.15.138:0/2517200648 conn(0x7fed9c0fe430 msgr2=0x7fed9c100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 -- 172.21.15.138:0/2517200648 shutdown_connections 2024-08-20T01:20:45.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.444+0000 7feda3400640 1 -- 172.21.15.138:0/2517200648 wait complete. 2024-08-20T01:20:45.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.445+0000 7feda3400640 1 Processor -- start 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 -- start start 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1029f0 0x7fed9c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1083f0 0x7fed9c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c112a10 con 0x7fed9c1083f0 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c112890 con 0x7fed9c108d70 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda3400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c112b90 con 0x7fed9c1029f0 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7fed9be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1083f0 0x7fed9c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.446+0000 7feda1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33356/0 (socket says 172.21.15.138:33356) 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 -- 172.21.15.138:0/322929537 learned_addr learned my addr 172.21.15.138:0/322929537 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1029f0 0x7fed9c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 -- 172.21.15.138:0/322929537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1029f0 msgr2=0x7fed9c0751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1029f0 0x7fed9c0751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 -- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1083f0 msgr2=0x7fed9c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1083f0 0x7fed9c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 -- 172.21.15.138:0/322929537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c1a7340 con 0x7fed9c108d70 2024-08-20T01:20:45.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda1400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed9400efd0 tx=0x7fed9400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7fed99600640 1 -- 172.21.15.138:0/322929537 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed94003b70 con 0x7fed9c108d70 2024-08-20T01:20:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7fed99600640 1 -- 172.21.15.138:0/322929537 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed94003d10 con 0x7fed9c108d70 2024-08-20T01:20:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7fed99600640 1 -- 172.21.15.138:0/322929537 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940108a0 con 0x7fed9c108d70 2024-08-20T01:20:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.447+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c1a7630 con 0x7fed9c108d70 2024-08-20T01:20:45.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.448+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1a7b50 con 0x7fed9c108d70 2024-08-20T01:20:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.451+0000 7feda6471740 1 Processor -- start 2024-08-20T01:20:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.451+0000 7fed99600640 1 -- 172.21.15.138:0/322929537 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fed94004020 con 0x7fed9c108d70 2024-08-20T01:20:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.451+0000 7feda6471740 1 -- start start 2024-08-20T01:20:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee74f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee74ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ee7466210 con 0x563ee74fd1e0 2024-08-20T01:20:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ee7455710 con 0x563ee74f0930 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda6471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ee7459750 con 0x563ee74f1250 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:60452/0 (socket says 172.21.15.138:60452) 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 -- 172.21.15.138:0/2814155021 learned_addr learned my addr 172.21.15.138:0/2814155021 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7fed9be00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee74f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7fed9be00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee74ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 -- 172.21.15.138:0/2814155021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 msgr2=0x563ee74f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee74f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 -- 172.21.15.138:0/2814155021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 msgr2=0x563ee74ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee74ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7feda0a00640 1 -- 172.21.15.138:0/2814155021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee7421de0 con 0x563ee74f1250 2024-08-20T01:20:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.452+0000 7fed9be00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee74ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:20:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.453+0000 7feda0a00640 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed90009cf0 tx=0x7fed90011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b29481569d1f6adf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.453+0000 7fed8dc00640 1 -- 172.21.15.138:0/2814155021 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90004230 con 0x563ee74f1250 2024-08-20T01:20:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.453+0000 7fed8dc00640 1 -- 172.21.15.138:0/2814155021 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed900043d0 con 0x563ee74f1250 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.453+0000 7fed8dc00640 1 -- 172.21.15.138:0/2814155021 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90019880 con 0x563ee74f1250 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.453+0000 7fed99600640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 0x7fed840b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7fed99600640 1 -- 172.21.15.138:0/322929537 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7fed940e47b0 con 0x7fed9c108d70 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7fed9be00640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 0x7fed840b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 -- 172.21.15.138:0/2814155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 msgr2=0x563ee74fcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed90009cf0 tx=0x7fed90011810 comp rx=0 tx=0).stop 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 -- 172.21.15.138:0/2814155021 shutdown_connections 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee74ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74fcca0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 --2- 172.21.15.138:0/2814155021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee74f0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 -- 172.21.15.138:0/2814155021 >> 172.21.15.138:0/2814155021 conn(0x563ee74ee760 msgr2=0x563ee74ef900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 -- 172.21.15.138:0/2814155021 shutdown_connections 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.454+0000 7feda6471740 1 -- 172.21.15.138:0/2814155021 wait complete. 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.455+0000 7fed9be00640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 0x7fed840b6280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed88005ab0 tx=0x7fed88005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.455+0000 7feda6471740 1 -- start start 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.455+0000 7feda6471740 1 Processor -- start 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ee738eab0 con 0x563ee74f0930 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ee73fc140 con 0x563ee74fd1e0 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ee74ffec0 con 0x563ee74f1250 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda6471740 10 client.0 _ll_register_callbacks cb 0x563ee740f5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.461 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7fed9be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33372/0 (socket says 172.21.15.138:33372) 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 -- 172.21.15.138:0/3035518316 learned_addr learned my addr 172.21.15.138:0/3035518316 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 -- 172.21.15.138:0/3035518316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 msgr2=0x563ee74f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 -- 172.21.15.138:0/3035518316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 msgr2=0x563ee74fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.456+0000 7feda1400640 1 -- 172.21.15.138:0/3035518316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee73fd170 con 0x563ee74fd1e0 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7feda1400640 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed940aaf50 tx=0x7fed94009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3536d4aa70ad72ab server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.462 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed940aafc0 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7fed940a7250 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed900043d0 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fed940af5c0 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7feda6471740 10 client.26981 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563ee71b8790 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed900043d0 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.457+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fed940ad070 con 0x563ee74fd1e0 2024-08-20T01:20:45.463 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 10 client.26981 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 10 client.26981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 20 client.26981 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee750cd70 0x563ee750f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 4 client.26981 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee7468160 con 0x563ee750cd70 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.458+0000 7feda0a00640 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee750cd70 0x563ee750f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.459+0000 7feda0a00640 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee750cd70 0x563ee750f160 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=956b962dbef5581c server_cookie=3034728ac5c6f1ed in_seq=0 out_seq=0 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.459+0000 7fed82000640 10 client.26981 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.459+0000 7fed82000640 1 -- 172.21.15.138:0/3035518316 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ee7468160 con 0x563ee750cd70 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.459+0000 7fed82000640 10 client.26981 handle_command_reply: tid=0 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 client.26981 shutdown 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 msgr2=0x563ee74fa350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fed940aaf50 tx=0x7fed94009960 comp rx=0 tx=0).stop 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 shutdown_connections 2024-08-20T01:20:45.464 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee750cd70 0x563ee750f160 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74fd1e0 0x563ee74fa350 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee74f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 --2- 172.21.15.138:0/3035518316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7fed82000640 0 client.26981 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 >> 172.21.15.138:0/3035518316 conn(0x563ee74ee760 msgr2=0x563ee74f4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 shutdown_connections 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 1 -- 172.21.15.138:0/3035518316 wait complete. 2024-08-20T01:20:45.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.460+0000 7feda6471740 20 client.26981 trim_cache size 0 max 16384 2024-08-20T01:20:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.477+0000 7feda6471740 1 Processor -- start 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 -- start start 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 0x563ee74fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 0x563ee75149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b45a0 con 0x563ee74f0930 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b1e40 con 0x563ee74f1250 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda6471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1afcb0 con 0x563ee74fd1e0 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7fed9be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 0x563ee74fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda0a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33920/0 (socket says 172.21.15.138:33920) 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda0a00640 1 -- 172.21.15.138:0/1982747427 learned_addr learned my addr 172.21.15.138:0/1982747427 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.478+0000 7feda1400640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 0x563ee75149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 -- 172.21.15.138:0/1982747427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 msgr2=0x563ee75149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 0x563ee75149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 -- 172.21.15.138:0/1982747427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 msgr2=0x563ee74fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 0x563ee74fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 -- 172.21.15.138:0/1982747427 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee7231620 con 0x563ee74f0930 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7fed9be00640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 0x563ee74fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda1400640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 0x563ee75149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:45.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7feda0a00640 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed90009cf0 tx=0x7fed90003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bde9a6bf7878ee server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7fed8d200640 1 -- 172.21.15.138:0/1982747427 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed900035f0 con 0x563ee74f0930 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7fed8d200640 1 -- 172.21.15.138:0/1982747427 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed90005a70 con 0x563ee74f0930 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.479+0000 7fed8d200640 1 -- 172.21.15.138:0/1982747427 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90019490 con 0x563ee74f0930 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 -- 172.21.15.138:0/1982747427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 msgr2=0x563ee74f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed90009cf0 tx=0x7fed90003f10 comp rx=0 tx=0).stop 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 -- 172.21.15.138:0/1982747427 shutdown_connections 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74fd1e0 0x563ee75149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f1250 0x563ee74fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 --2- 172.21.15.138:0/1982747427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74f0930 0x563ee74f9680 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 -- 172.21.15.138:0/1982747427 >> 172.21.15.138:0/1982747427 conn(0x563ee74ee760 msgr2=0x563ee74efab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 -- 172.21.15.138:0/1982747427 shutdown_connections 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.480+0000 7feda6471740 1 -- 172.21.15.138:0/1982747427 wait complete. 2024-08-20T01:20:45.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 -- start start 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 Processor -- start 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee7502ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee75759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b0300 con 0x563ee74fd1e0 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b1770 con 0x563ee74f0930 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.481+0000 7feda6471740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1b09d0 con 0x563ee74f1250 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda6471740 10 client.0 _ll_register_callbacks cb 0x7fed94004830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33402/0 (socket says 172.21.15.138:33402) 2024-08-20T01:20:45.485 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 -- 172.21.15.138:0/1929508645 learned_addr learned my addr 172.21.15.138:0/1929508645 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda1400640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee75759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7fed9be00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee7502ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 -- 172.21.15.138:0/1929508645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 msgr2=0x563ee7502ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee7502ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 -- 172.21.15.138:0/1929508645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 msgr2=0x563ee75759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee75759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 -- 172.21.15.138:0/1929508645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee7575ef0 con 0x563ee74f0930 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda1400640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee75759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7feda0a00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed900099c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a83605664477ff2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.482+0000 7fed82000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90039020 con 0x563ee74f0930 2024-08-20T01:20:45.486 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7fed900035f0 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed90005a70 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fed900052a0 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7feda6471740 10 client.26987 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563ee7577130 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed90005a70 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fed90025420 con 0x563ee74f0930 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7feda6471740 10 client.26987 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.483+0000 7feda6471740 10 client.26987 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7feda6471740 20 client.26987 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee75776d0 0x563ee7579af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7feda6471740 4 client.26987 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ee7463c00 con 0x563ee75776d0 2024-08-20T01:20:45.487 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7fed9be00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee75776d0 0x563ee7579af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7fed9be00640 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee75776d0 0x563ee7579af0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a13d2fcaed56f2 server_cookie=1febaaa85273f09a in_seq=0 out_seq=0 2024-08-20T01:20:45.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.484+0000 7fed82000640 10 client.26987 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:45.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:45 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2418696772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:20:45.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:45 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2418696772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:20:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7fed82000640 1 -- 172.21.15.138:0/1929508645 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ee7463c00 con 0x563ee75776d0 2024-08-20T01:20:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7fed82000640 10 client.26987 handle_command_reply: tid=0 2024-08-20T01:20:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 client.26987 shutdown 2024-08-20T01:20:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 msgr2=0x563ee7502590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed900099c0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 shutdown_connections 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ee75776d0 0x563ee7579af0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ee74fd1e0 0x563ee75759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7fed82000640 0 client.26987 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ee74f1250 0x563ee7502ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 --2- 172.21.15.138:0/1929508645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ee74f0930 0x563ee7502590 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.858+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 >> 172.21.15.138:0/1929508645 conn(0x563ee74ee760 msgr2=0x563ee74fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 shutdown_connections 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda6471740 1 -- 172.21.15.138:0/1929508645 wait complete. 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda6471740 20 client.26987 trim_cache size 0 max 16384 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:20:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 msgr2=0x7fed840b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 0x7fed840b6280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fed88005ab0 tx=0x7fed88005a20 comp rx=0 tx=0).stop 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 msgr2=0x7fed9c075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fed9400efd0 tx=0x7fed9400c6a0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 shutdown_connections 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fed840b3dc0 0x7fed840b6280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed9c108d70 0x7fed9c075c50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed9c1083f0 0x7fed9c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 --2- 172.21.15.138:0/322929537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fed9c1029f0 0x7fed9c0751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.859+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 >> 172.21.15.138:0/322929537 conn(0x7fed9c0fe430 msgr2=0x7fed9c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.860+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 shutdown_connections 2024-08-20T01:20:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:45.860+0000 7feda3400640 1 -- 172.21.15.138:0/322929537 wait complete. 2024-08-20T01:20:45.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:45 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2418696772' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:20:46.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502627 3.70 MB/sec execute 538 sec latency 18.838 ms 2024-08-20T01:20:46.274 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 137.3 --await-for 45.0' 2024-08-20T01:20:46.274 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 137.3 --await-for 45.0 2024-08-20T01:20:46.697 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:20:46.726 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:46 smithi138 ceph-mon[18076]: pgmap v754: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-08-20T01:20:46.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:46 smithi142 ceph-mon[21799]: pgmap v754: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-08-20T01:20:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:46 smithi167 ceph-mon[21965]: pgmap v754: 129 pgs: 129 active+clean; 429 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 149 op/s 2024-08-20T01:20:47.204 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502633 3.69 MB/sec execute 539 sec latency 973.918 ms 2024-08-20T01:20:47.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 -- 172.21.15.138:0/1355108330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc104ca0 msgr2=0x7f0bdc105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 --2- 172.21.15.138:0/1355108330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc105080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc009860 tx=0x7f0bcc02a340 comp rx=0 tx=0).stop 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 -- 172.21.15.138:0/1355108330 shutdown_connections 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 --2- 172.21.15.138:0/1355108330 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc10b030 0x7f0bdc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 --2- 172.21.15.138:0/1355108330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 --2- 172.21.15.138:0/1355108330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc105080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 -- 172.21.15.138:0/1355108330 >> 172.21.15.138:0/1355108330 conn(0x7f0bdc100730 msgr2=0x7f0bdc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 -- 172.21.15.138:0/1355108330 shutdown_connections 2024-08-20T01:20:47.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.376+0000 7f0be1600640 1 -- 172.21.15.138:0/1355108330 wait complete. 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.377+0000 7f0be1600640 1 Processor -- start 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.377+0000 7f0be1600640 1 -- start start 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc0773e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 0x7f0bdc077e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114cb0 con 0x7f0bdc10a6b0 2024-08-20T01:20:47.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114b30 con 0x7f0bdc10b030 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0be1600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc114e30 con 0x7f0bdc104ca0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc0773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33954/0 (socket says 172.21.15.138:33954) 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0bd3e00640 1 -- 172.21.15.138:0/2516238231 learned_addr learned my addr 172.21.15.138:0/2516238231 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.378+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 0x7f0bdc077e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2516238231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 msgr2=0x7f0bdc077e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 0x7f0bdc077e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 msgr2=0x7f0bdc077920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2516238231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bdc1ab750 con 0x7f0bdc104ca0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 0x7f0bdc077e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc0773e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc035040 tx=0x7f0bcc032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bd3400640 1 -- 172.21.15.138:0/2516238231 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc040020 con 0x7f0bdc104ca0 2024-08-20T01:20:47.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bdc1ab9e0 con 0x7f0bdc104ca0 2024-08-20T01:20:47.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bd3400640 1 -- 172.21.15.138:0/2516238231 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc030e10 con 0x7f0bdc104ca0 2024-08-20T01:20:47.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.379+0000 7f0bd3400640 1 -- 172.21.15.138:0/2516238231 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc037040 con 0x7f0bdc104ca0 2024-08-20T01:20:47.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.380+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bdc1abec0 con 0x7f0bdc104ca0 2024-08-20T01:20:47.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.382+0000 7f0be403e740 1 Processor -- start 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 -- start start 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bd3400640 1 -- 172.21.15.138:0/2516238231 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0bcc043020 con 0x7f0bdc104ca0 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 0x55b3577727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b35777e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b357712670 con 0x55b35777eca0 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3576e67c0 con 0x55b357772d10 2024-08-20T01:20:47.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0be403e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3576d5cc0 con 0x55b3577723f0 2024-08-20T01:20:47.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bdb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 0x55b3577727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33970/0 (socket says 172.21.15.138:33970) 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b35777e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.383+0000 7f0bd3e00640 1 -- 172.21.15.138:0/1338366536 learned_addr learned my addr 172.21.15.138:0/1338366536 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 -- 172.21.15.138:0/1338366536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 msgr2=0x55b3577727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 0x55b3577727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 -- 172.21.15.138:0/1338366536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 msgr2=0x55b35777e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b35777e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 -- 172.21.15.138:0/1338366536 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b357669110 con 0x55b35777eca0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bdb400640 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 0x55b3577727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40115f0 tx=0x7f0bc4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66689927209ceb90 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bcb400640 1 -- 172.21.15.138:0/1338366536 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc401e070 con 0x55b35777eca0 2024-08-20T01:20:47.385 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.384+0000 7f0bcb400640 1 -- 172.21.15.138:0/1338366536 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc400f660 con 0x55b35777eca0 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0bcb400640 1 -- 172.21.15.138:0/1338366536 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40037f0 con 0x55b35777eca0 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0bd3400640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 0x7f0bc00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0bd3400640 1 -- 172.21.15.138:0/2516238231 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f0bcc104230 con 0x7f0bdc104ca0 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 0x7f0bc00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 -- 172.21.15.138:0/1338366536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 msgr2=0x55b357781090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40115f0 tx=0x7f0bc4011ab0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 -- 172.21.15.138:0/1338366536 shutdown_connections 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b35777eca0 0x55b357781090 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b35777e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 --2- 172.21.15.138:0/1338366536 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b3577723f0 0x55b3577727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 -- 172.21.15.138:0/1338366536 >> 172.21.15.138:0/1338366536 conn(0x55b357770220 msgr2=0x55b3577713c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.385+0000 7f0be403e740 1 -- 172.21.15.138:0/1338366536 shutdown_connections 2024-08-20T01:20:47.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.386+0000 7f0be403e740 1 -- 172.21.15.138:0/1338366536 wait complete. 2024-08-20T01:20:47.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.386+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 0x7f0bc00b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005fd0 tx=0x7f0bd4010040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.386+0000 7f0be403e740 1 -- start start 2024-08-20T01:20:47.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.386+0000 7f0be403e740 1 Processor -- start 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35777bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b35777c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b35767b780 con 0x55b3577723f0 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b3576d9d00 con 0x55b35777eca0 2024-08-20T01:20:47.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b3575387a0 con 0x55b357772d10 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0be403e740 10 client.0 _ll_register_callbacks cb 0x55b357683940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b35777c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35777bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdb400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:33448/0 (socket says 172.21.15.138:33448) 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdb400640 1 -- 172.21.15.138:0/2872628745 learned_addr learned my addr 172.21.15.138:0/2872628745 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 -- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 msgr2=0x55b35777c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 -- 172.21.15.138:0/2872628745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 msgr2=0x55b35777bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35777bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bd3e00640 1 -- 172.21.15.138:0/2872628745 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3576889c0 con 0x55b357772d10 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35777bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:47.391 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.387+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b35777c130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0bc401c040 tx=0x7f0bc40052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b8a5cdc2384bfad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4026020 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc400f660 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f0bc40054e0 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0bc4031050 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0be403e740 10 client.26956 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.388+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b357444b90 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc400f660 con 0x55b357772d10 2024-08-20T01:20:47.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f0bc4021030 con 0x55b357772d10 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 10 client.26956 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 10 client.26956 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 20 client.26956 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b35778e5e0 0x55b3577909d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 4 client.26956 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3576e8710 con 0x55b35778e5e0 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.389+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b35778e5e0 0x55b3577909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.390+0000 7f0bbfe00640 10 client.26956 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.390+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b35778e5e0 0x55b3577909d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e68f861c185afb server_cookie=dc5ac341a42aa999 in_seq=0 out_seq=0 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.390+0000 7f0bbfe00640 1 -- 172.21.15.138:0/2872628745 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3576e8710 con 0x55b35778e5e0 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.390+0000 7f0bbfe00640 10 client.26956 handle_command_reply: tid=0 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.390+0000 7f0be403e740 1 client.26956 shutdown 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 msgr2=0x55b35777c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b35777c130 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0bc401c040 tx=0x7f0bc40052f0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 shutdown_connections 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b35778e5e0 0x55b3577909d0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b35777c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b35777c130 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 --2- 172.21.15.138:0/2872628745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35777bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0bbfe00640 0 client.26956 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 >> 172.21.15.138:0/2872628745 conn(0x55b357770220 msgr2=0x55b357775d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 shutdown_connections 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 1 -- 172.21.15.138:0/2872628745 wait complete. 2024-08-20T01:20:47.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.391+0000 7f0be403e740 20 client.26956 trim_cache size 0 max 16384 2024-08-20T01:20:47.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.408+0000 7f0be403e740 1 Processor -- start 2024-08-20T01:20:47.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 -- start start 2024-08-20T01:20:47.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b357796140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b357796680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b8a00 con 0x55b3577723f0 2024-08-20T01:20:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b62a0 con 0x55b35777eca0 2024-08-20T01:20:47.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0be403e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b4110 con 0x55b357772d10 2024-08-20T01:20:47.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33988/0 (socket says 172.21.15.138:33988) 2024-08-20T01:20:47.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2840592143 learned_addr learned my addr 172.21.15.138:0/2840592143 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b357796140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.409+0000 7f0bdb400640 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b357796680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2840592143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 msgr2=0x55b357796140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b357796140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2840592143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 msgr2=0x55b357796680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b357796680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 -- 172.21.15.138:0/2840592143 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b35777a0f0 con 0x55b3577723f0 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0c7dc0 tx=0x7f0bcc1066f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6804822460867a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bcaa00640 1 -- 172.21.15.138:0/2840592143 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0c6a10 con 0x55b3577723f0 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bcaa00640 1 -- 172.21.15.138:0/2840592143 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc107b30 con 0x55b3577723f0 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.410+0000 7f0bcaa00640 1 -- 172.21.15.138:0/2840592143 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0c62e0 con 0x55b3577723f0 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- 172.21.15.138:0/2840592143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 msgr2=0x55b35776f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc0c7dc0 tx=0x7f0bcc1066f0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- 172.21.15.138:0/2840592143 shutdown_connections 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b35777eca0 0x55b357796680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b357772d10 0x55b357796140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 --2- 172.21.15.138:0/2840592143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b35776f1f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- 172.21.15.138:0/2840592143 >> 172.21.15.138:0/2840592143 conn(0x55b357770220 msgr2=0x55b35777c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- 172.21.15.138:0/2840592143 shutdown_connections 2024-08-20T01:20:47.412 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- 172.21.15.138:0/2840592143 wait complete. 2024-08-20T01:20:47.416 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.411+0000 7f0be403e740 1 -- start start 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 Processor -- start 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b357782960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b35777eca0 0x55b3577868d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b4760 con 0x55b3577723f0 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b5bd0 con 0x55b357772d10 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0bdc1b4e30 con 0x55b35777eca0 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0be403e740 10 client.0 _ll_register_callbacks cb 0x7f0bc401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0bd3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b357782960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.417 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0bdaa00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33996/0 (socket says 172.21.15.138:33996) 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0bdaa00640 1 -- 172.21.15.138:0/267439794 learned_addr learned my addr 172.21.15.138:0/267439794 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.412+0000 7f0bdb400640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b35777eca0 0x55b3577868d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 -- 172.21.15.138:0/267439794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b35777eca0 msgr2=0x55b3577868d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b35777eca0 0x55b3577868d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 -- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 msgr2=0x55b357782960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b357782960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 -- 172.21.15.138:0/267439794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b357786fe0 con 0x55b3577723f0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bdaa00640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005940 tx=0x7f0bcc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0d01b8d7a48af4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bbfe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc0ca2c0 con 0x55b3577723f0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f0bcc0b20b0 con 0x55b3577723f0 2024-08-20T01:20:47.418 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bcc107b30 con 0x55b3577723f0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc0ca4a0 con 0x55b3577723f0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.413+0000 7f0be403e740 10 client.16956 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b3577f8ca0 con 0x55b3577723f0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bcc107b30 con 0x55b3577723f0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f0bcc0e3020 con 0x55b3577723f0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 10 client.16956 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 10 client.16956 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 20 client.16956 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b3577f9380 0x55b3577fb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 4 client.16956 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.3, "await_for": 45.0}] 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 137.3, "await_for": 45.0}) -- 0x55b3576e41b0 con 0x55b3577f9380 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.414+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b3577f9380 0x55b3577fb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.415+0000 7f0bd3e00640 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b3577f9380 0x55b3577fb7a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da4d8b0948937da9 server_cookie=226480f75f0c52f9 in_seq=0 out_seq=0 2024-08-20T01:20:47.419 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.415+0000 7f0bbfe00640 10 client.16956 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:47.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0bbfe00640 1 -- 172.21.15.138:0/267439794 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b3576e41b0 con 0x55b3577f9380 2024-08-20T01:20:47.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0bbfe00640 10 client.16956 handle_command_reply: tid=0 2024-08-20T01:20:47.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 client.16956 shutdown 2024-08-20T01:20:47.477 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 msgr2=0x55b3577823e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc005940 tx=0x7f0bcc004610 comp rx=0 tx=0).stop 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 shutdown_connections 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b3577f9380 0x55b3577fb7a0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b35777eca0 0x55b3577868d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b357772d10 0x55b357782960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 --2- 172.21.15.138:0/267439794 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b3577723f0 0x55b3577823e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0bbfe00640 0 client.16956 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 >> 172.21.15.138:0/267439794 conn(0x55b357770220 msgr2=0x55b35777bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 shutdown_connections 2024-08-20T01:20:47.478 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.478+0000 7f0be403e740 1 -- 172.21.15.138:0/267439794 wait complete. 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.479+0000 7f0be403e740 20 client.16956 trim_cache size 0 max 16384 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 43, 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 46, 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "a486140d": { 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 46, 2024-08-20T01:20:47.479 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 137.3, 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:20:47.480 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.479+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 msgr2=0x7f0bc00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.481 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.479+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 0x7f0bc00b6200 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4005fd0 tx=0x7f0bd4010040 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 msgr2=0x7f0bdc0773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc0773e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc035040 tx=0x7f0bcc032d50 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 shutdown_connections 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f0bc00b3d40 0x7f0bc00b6200 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bdc10b030 0x7f0bdc077e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0bdc10a6b0 0x7f0bdc077920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 --2- 172.21.15.138:0/2516238231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0bdc104ca0 0x7f0bdc0773e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 >> 172.21.15.138:0/2516238231 conn(0x7f0bdc100730 msgr2=0x7f0bdc102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 shutdown_connections 2024-08-20T01:20:47.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:20:47.480+0000 7f0be1600640 1 -- 172.21.15.138:0/2516238231 wait complete. 2024-08-20T01:20:48.205 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.69 MB/sec execute 540 sec latency 1215.673 ms 2024-08-20T01:20:48.463 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a486140d', quiesce took 4.5 seconds. Will release after: 12.8 2024-08-20T01:20:48.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:48 smithi142 ceph-mon[21799]: pgmap v755: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-08-20T01:20:48.765 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:48 smithi167 ceph-mon[21965]: pgmap v755: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-08-20T01:20:48.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:48 smithi138 ceph-mon[18076]: pgmap v755: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.4 MiB/s wr, 169 op/s 2024-08-20T01:20:49.205 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.68 MB/sec execute 541 sec latency 1753.539 ms 2024-08-20T01:20:50.205 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.67 MB/sec execute 542 sec latency 2753.647 ms 2024-08-20T01:20:50.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:50 smithi142 ceph-mon[21799]: pgmap v756: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-08-20T01:20:50.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:50 smithi138 ceph-mon[18076]: pgmap v756: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-08-20T01:20:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:50 smithi167 ceph-mon[21965]: pgmap v756: 129 pgs: 129 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 144 op/s 2024-08-20T01:20:51.205 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.67 MB/sec execute 543 sec latency 3753.812 ms 2024-08-20T01:20:52.210 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.66 MB/sec execute 544 sec latency 4758.735 ms 2024-08-20T01:20:52.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:52 smithi142 ceph-mon[21799]: pgmap v757: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-08-20T01:20:52.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:52 smithi138 ceph-mon[18076]: pgmap v757: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-08-20T01:20:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:52 smithi167 ceph-mon[21965]: pgmap v757: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-08-20T01:20:53.210 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.65 MB/sec execute 545 sec latency 5758.808 ms 2024-08-20T01:20:54.210 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.65 MB/sec execute 546 sec latency 6758.927 ms 2024-08-20T01:20:54.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:54 smithi142 ceph-mon[21799]: pgmap v758: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-08-20T01:20:54.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:54 smithi138 ceph-mon[18076]: pgmap v758: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-08-20T01:20:54.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:54 smithi167 ceph-mon[21965]: pgmap v758: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.6 MiB/s wr, 122 op/s 2024-08-20T01:20:55.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.64 MB/sec execute 547 sec latency 7759.073 ms 2024-08-20T01:20:56.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.63 MB/sec execute 548 sec latency 8759.149 ms 2024-08-20T01:20:56.712 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:56 smithi138 ceph-mon[18076]: pgmap v759: 129 pgs: 129 active+clean; 436 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-08-20T01:20:56.713 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:56 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:20:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:56 smithi167 ceph-mon[21965]: pgmap v759: 129 pgs: 129 active+clean; 436 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-08-20T01:20:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:56 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:20:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:56 smithi142 ceph-mon[21799]: pgmap v759: 129 pgs: 129 active+clean; 436 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-08-20T01:20:56.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:56 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:20:57.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.63 MB/sec execute 549 sec latency 9759.238 ms 2024-08-20T01:20:57.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:57 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:20:57.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:57 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:20:57.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:57 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:20:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:57 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:20:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:57 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:20:57.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:57 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:20:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:57 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:20:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:57 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:20:57.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:57 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:20:58.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.62 MB/sec execute 550 sec latency 10759.365 ms 2024-08-20T01:20:58.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:20:58 smithi142 ceph-mon[21799]: pgmap v760: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-08-20T01:20:58.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:20:58 smithi138 ceph-mon[18076]: pgmap v760: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-08-20T01:20:58.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:20:58 smithi167 ceph-mon[21965]: pgmap v760: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 1.6 MiB/s wr, 107 op/s 2024-08-20T01:20:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.979+0000 7f9a31000640 1 -- 172.21.15.138:0/595191016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 msgr2=0x7f9a2c10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:58.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.979+0000 7f9a31000640 1 --2- 172.21.15.138:0/595191016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c10f380 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9a24009ca0 tx=0x7f9a2402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 -- 172.21.15.138:0/595191016 shutdown_connections 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 --2- 172.21.15.138:0/595191016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c10f380 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 --2- 172.21.15.138:0/595191016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c108200 0x7f9a2c108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 --2- 172.21.15.138:0/595191016 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c102800 0x7f9a2c102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 -- 172.21.15.138:0/595191016 >> 172.21.15.138:0/595191016 conn(0x7f9a2c0fe280 msgr2=0x7f9a2c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 -- 172.21.15.138:0/595191016 shutdown_connections 2024-08-20T01:20:58.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.980+0000 7f9a31000640 1 -- 172.21.15.138:0/595191016 wait complete. 2024-08-20T01:20:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 Processor -- start 2024-08-20T01:20:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 -- start start 2024-08-20T01:20:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 0x7f9a2c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c108200 0x7f9a2c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c112850 con 0x7f9a2c108b80 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1126d0 con 0x7f9a2c102800 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a31000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2c1129d0 con 0x7f9a2c108200 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34704/0 (socket says 172.21.15.138:34704) 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a2be00640 1 -- 172.21.15.138:0/1313876190 learned_addr learned my addr 172.21.15.138:0/1313876190 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 0x7f9a2c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.981+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c108200 0x7f9a2c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 -- 172.21.15.138:0/1313876190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c108200 msgr2=0x7f9a2c19b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c108200 0x7f9a2c19b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 -- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 msgr2=0x7f9a2c1a07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 -- 172.21.15.138:0/1313876190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a2c1a0fc0 con 0x7f9a2c102800 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2be00640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 0x7f9a2c19af20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e9f0 tx=0x7f9a1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a23e00640 1 -- 172.21.15.138:0/1313876190 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c00ccc0 con 0x7f9a2c102800 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.982+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a2c1a12b0 con 0x7f9a2c102800 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.983+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a2c1a7970 con 0x7f9a2c102800 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.983+0000 7f9a23e00640 1 -- 172.21.15.138:0/1313876190 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0044d0 con 0x7f9a2c102800 2024-08-20T01:20:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.983+0000 7f9a23e00640 1 -- 172.21.15.138:0/1313876190 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c01c070 con 0x7f9a2c102800 2024-08-20T01:20:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.985+0000 7f9a3564a740 1 Processor -- start 2024-08-20T01:20:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.985+0000 7f9a3564a740 1 -- start start 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd21e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd223c0 0x55c21dd247b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c21dc8b210 con 0x55c21dd15b10 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c21dc7a710 con 0x55c21dd223c0 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a3564a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c21dc7e750 con 0x55c21dd16430 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34714/0 (socket says 172.21.15.138:34714) 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 -- 172.21.15.138:0/3073977904 learned_addr learned my addr 172.21.15.138:0/3073977904 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd223c0 0x55c21dd247b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 -- 172.21.15.138:0/3073977904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 msgr2=0x55c21dd21e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:20:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd21e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 -- 172.21.15.138:0/3073977904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd223c0 msgr2=0x55c21dd247b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd223c0 0x55c21dd247b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.986+0000 7f9a2be00640 1 -- 172.21.15.138:0/3073977904 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21dc46de0 con 0x55c21dd15b10 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a23e00640 1 -- 172.21.15.138:0/1313876190 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9a1c005020 con 0x7f9a2c102800 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a2be00640 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005fc0 tx=0x7f9a24009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e57b8b4ab144e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a20c00640 1 -- 172.21.15.138:0/3073977904 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a24031650 con 0x55c21dd15b10 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a20c00640 1 -- 172.21.15.138:0/3073977904 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a24036040 con 0x55c21dd15b10 2024-08-20T01:20:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a20c00640 1 -- 172.21.15.138:0/3073977904 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a240044c0 con 0x55c21dd15b10 2024-08-20T01:20:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a3564a740 1 -- 172.21.15.138:0/3073977904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 msgr2=0x55c21dd15ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.987+0000 7f9a3564a740 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005fc0 tx=0x7f9a24009880 comp rx=0 tx=0).stop 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 -- 172.21.15.138:0/3073977904 shutdown_connections 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd223c0 0x55c21dd247b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd21e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 --2- 172.21.15.138:0/3073977904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd15b10 0x55c21dd15ef0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 -- 172.21.15.138:0/3073977904 >> 172.21.15.138:0/3073977904 conn(0x55c21dd13940 msgr2=0x55c21dd14ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 -- 172.21.15.138:0/3073977904 shutdown_connections 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 -- 172.21.15.138:0/3073977904 wait complete. 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a23e00640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 0x7f9a100b6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a3564a740 1 -- start start 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a23e00640 1 -- 172.21.15.138:0/1313876190 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f9a1c0e42c0 con 0x7f9a2c102800 2024-08-20T01:20:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.988+0000 7f9a2be00640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 0x7f9a100b6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 Processor -- start 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd21ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a2be00640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 0x7f9a100b6110 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005e30 tx=0x7f9a2402a8c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd223c0 0x55c21dd1f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c21dbb3ab0 con 0x55c21dd223c0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c21dc21140 con 0x55c21dd15b10 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c21dd24ec0 con 0x55c21dd16430 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.989+0000 7f9a3564a740 10 client.0 _ll_register_callbacks cb 0x55c21dc345d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd223c0 0x55c21dd1f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd21ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38246/0 (socket says 172.21.15.138:38246) 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 -- 172.21.15.138:0/2803576313 learned_addr learned my addr 172.21.15.138:0/2803576313 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 -- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 msgr2=0x55c21dd21ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd21ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 -- 172.21.15.138:0/2803576313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd223c0 msgr2=0x55c21dd1f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd223c0 0x55c21dd1f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 -- 172.21.15.138:0/2803576313 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21dc22170 con 0x55c21dd16430 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a18012740 tx=0x7f9a18012c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaacfb89dabc16bf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a15600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.990+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1800f920 con 0x55c21dd16430 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a24036040 con 0x55c21dd16430 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f9a1800fac0 con 0x55c21dd16430 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9a1800fcc0 con 0x55c21dd16430 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 10 client.26971 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c21d9dd790 con 0x55c21dd16430 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a24036040 con 0x55c21dd16430 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f9a1801c420 con 0x55c21dd16430 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 10 client.26971 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 10 client.26971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 20 client.26971 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd31ef0 0x55c21dd34310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 4 client.26971 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.991+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c21dc8d160 con 0x55c21dd31ef0 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.992+0000 7f9a2b400640 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd31ef0 0x55c21dd34310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.992+0000 7f9a2b400640 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd31ef0 0x55c21dd34310 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31530465bd770e3 server_cookie=3105863a7f4971fe in_seq=0 out_seq=0 2024-08-20T01:20:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.992+0000 7f9a15600640 10 client.26971 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a15600640 1 -- 172.21.15.138:0/2803576313 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c21dc8d160 con 0x55c21dd31ef0 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a15600640 10 client.26971 handle_command_reply: tid=0 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 client.26971 shutdown 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 msgr2=0x55c21dd1ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a18012740 tx=0x7f9a18012c00 comp rx=0 tx=0).stop 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 shutdown_connections 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd31ef0 0x55c21dd34310 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd223c0 0x55c21dd1f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.993+0000 7f9a15600640 0 client.26971 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd16430 0x55c21dd1ef30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2803576313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd21ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 >> 172.21.15.138:0/2803576313 conn(0x55c21dd13940 msgr2=0x55c21dd194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 shutdown_connections 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 1 -- 172.21.15.138:0/2803576313 wait complete. 2024-08-20T01:20:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:58.994+0000 7f9a3564a740 20 client.26971 trim_cache size 0 max 16384 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 Processor -- start 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 -- start start 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 0x55c21dd39be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd15b10 0x55c21dd3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c012830 con 0x55c21dd16430 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c0100d0 con 0x55c21dd13940 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.009+0000 7f9a3564a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c00df40 con 0x55c21dd15b10 2024-08-20T01:20:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34736/0 (socket says 172.21.15.138:34736) 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 0x55c21dd39be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2be00640 1 -- 172.21.15.138:0/2133505834 learned_addr learned my addr 172.21.15.138:0/2133505834 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd15b10 0x55c21dd3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 -- 172.21.15.138:0/2133505834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd15b10 msgr2=0x55c21dd3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd15b10 0x55c21dd3a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 -- 172.21.15.138:0/2133505834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 msgr2=0x55c21dd2a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 -- 172.21.15.138:0/2133505834 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c21da56620 con 0x55c21dd13940 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2be00640 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a2b400640 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 0x55c21dd39be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0e1610 tx=0x7f9a1c092040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d8e033b16b67ccc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a17e00640 1 -- 172.21.15.138:0/2133505834 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0ca070 con 0x55c21dd13940 2024-08-20T01:20:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a17e00640 1 -- 172.21.15.138:0/2133505834 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0ab1d0 con 0x55c21dd13940 2024-08-20T01:20:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.010+0000 7f9a17e00640 1 -- 172.21.15.138:0/2133505834 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0c1070 con 0x55c21dd13940 2024-08-20T01:20:59.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 -- 172.21.15.138:0/2133505834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 msgr2=0x55c21dd39be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 0x55c21dd39be0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0e1610 tx=0x7f9a1c092040 comp rx=0 tx=0).stop 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 -- 172.21.15.138:0/2133505834 shutdown_connections 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd2a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd15b10 0x55c21dd3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 --2- 172.21.15.138:0/2133505834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd13940 0x55c21dd39be0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 -- 172.21.15.138:0/2133505834 >> 172.21.15.138:0/2133505834 conn(0x55c21dd1ff90 msgr2=0x55c21dd20d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 -- 172.21.15.138:0/2133505834 shutdown_connections 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.011+0000 7f9a3564a740 1 -- 172.21.15.138:0/2133505834 wait complete. 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 -- start start 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 Processor -- start 2024-08-20T01:20:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd27fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd94b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c00e590 con 0x55c21dd16430 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c00fa00 con 0x55c21dd15b10 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c00ec60 con 0x55c21dd13940 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.012+0000 7f9a3564a740 10 client.0 _ll_register_callbacks cb 0x7f9a18009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd94b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38264/0 (socket says 172.21.15.138:38264) 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 -- 172.21.15.138:0/1654578700 learned_addr learned my addr 172.21.15.138:0/1654578700 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd27fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 -- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 msgr2=0x55c21dd27fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd27fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 -- 172.21.15.138:0/1654578700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 msgr2=0x55c21dd94b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd94b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 -- 172.21.15.138:0/1654578700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c21dd952c0 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd27fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a2b400640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0aa7d0 tx=0x7f9a1c0a71c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2b671e4e53ce49 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a15600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c088020 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f9a1c0a6d60 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.013+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9a1c0ab1d0 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9a1c0c1070 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 10 client.26977 fetch_fsmap learned FSMap version 43 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c21dd9c730 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a1c0ab1d0 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f9a1c0e1890 con 0x55c21dd13940 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 10 client.26977 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:20:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 10 client.26977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 20 client.26977 populate_metadata read hostname 'smithi138' 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd223c0 0x55c21dd9ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 4 client.26977 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c21dc88c00 con 0x55c21dd223c0 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.014+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd223c0 0x55c21dd9ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.015+0000 7f9a2aa00640 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd223c0 0x55c21dd9ee90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82748b1f58e6378 server_cookie=ad71bf2f57b95155 in_seq=0 out_seq=0 2024-08-20T01:20:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.015+0000 7f9a15600640 10 client.26977 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a15600640 1 -- 172.21.15.138:0/1654578700 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c21dc88c00 con 0x55c21dd223c0 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a15600640 10 client.26977 handle_command_reply: tid=0 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 client.26977 shutdown 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 msgr2=0x55c21dd27a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0aa7d0 tx=0x7f9a1c0a71c0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 shutdown_connections 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c21dd223c0 0x55c21dd9ee90 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c21dd16430 0x55c21dd94b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c21dd15b10 0x55c21dd27fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 --2- 172.21.15.138:0/1654578700 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c21dd13940 0x55c21dd27a50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a15600640 0 client.26977 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 >> 172.21.15.138:0/1654578700 conn(0x55c21dd1ff90 msgr2=0x55c21dd2d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 shutdown_connections 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 1 -- 172.21.15.138:0/1654578700 wait complete. 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.044+0000 7f9a3564a740 20 client.26977 trim_cache size 0 max 16384 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:20:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:20:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 msgr2=0x7f9a100b6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 0x7f9a100b6110 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f9a24005e30 tx=0x7f9a2402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 msgr2=0x7f9a2c19af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:20:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 0x7f9a2c19af20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00e9f0 tx=0x7f9a1c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 shutdown_connections 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9a100b3c50 0x7f9a100b6110 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9a2c108b80 0x7f9a2c1a07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9a2c108200 0x7f9a2c19b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 --2- 172.21.15.138:0/1313876190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a2c102800 0x7f9a2c19af20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 >> 172.21.15.138:0/1313876190 conn(0x7f9a2c0fe280 msgr2=0x7f9a2c0750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 shutdown_connections 2024-08-20T01:20:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:20:59.045+0000 7f9a31000640 1 -- 172.21.15.138:0/1313876190 wait complete. 2024-08-20T01:20:59.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.61 MB/sec execute 551 sec latency 11759.494 ms 2024-08-20T01:21:00.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.61 MB/sec execute 552 sec latency 12759.633 ms 2024-08-20T01:21:00.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:00 smithi138 ceph-mon[18076]: pgmap v761: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 955 KiB/s wr, 69 op/s 2024-08-20T01:21:00.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:00 smithi167 ceph-mon[21965]: pgmap v761: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 955 KiB/s wr, 69 op/s 2024-08-20T01:21:00.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:00 smithi142 ceph-mon[21799]: pgmap v761: 129 pgs: 129 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 955 KiB/s wr, 69 op/s 2024-08-20T01:21:01.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.60 MB/sec execute 553 sec latency 13759.821 ms 2024-08-20T01:21:01.264 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:21:01.688 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:21:02.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.59 MB/sec execute 554 sec latency 14759.938 ms 2024-08-20T01:21:02.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 -- 172.21.15.138:0/1867741623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c10b040 msgr2=0x7f341c111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 --2- 172.21.15.138:0/1867741623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c10b040 0x7f341c111840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ca0 tx=0x7f341402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 -- 172.21.15.138:0/1867741623 shutdown_connections 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 --2- 172.21.15.138:0/1867741623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c10b040 0x7f341c111840 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 --2- 172.21.15.138:0/1867741623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10a6c0 0x7f341c10ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 --2- 172.21.15.138:0/1867741623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c104cb0 0x7f341c105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 -- 172.21.15.138:0/1867741623 >> 172.21.15.138:0/1867741623 conn(0x7f341c100720 msgr2=0x7f341c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 -- 172.21.15.138:0/1867741623 shutdown_connections 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.593+0000 7f3421200640 1 -- 172.21.15.138:0/1867741623 wait complete. 2024-08-20T01:21:02.593 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 Processor -- start 2024-08-20T01:21:02.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 -- start start 2024-08-20T01:21:02.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c104cb0 0x7f341c19b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:02.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:02.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 0x7f341c1a0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:02.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f341c114d50 con 0x7f341c104cb0 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f341c114bd0 con 0x7f341c10a6c0 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.594+0000 7f3421200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f341c114ed0 con 0x7f341c10b040 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3419e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c104cb0 0x7f341c19b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f341a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 0x7f341c1a0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:57216/0 (socket says 172.21.15.138:57216) 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f341a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 0x7f341c1a0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38280/0 (socket says 172.21.15.138:38280) 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 -- 172.21.15.138:0/187109288 learned_addr learned my addr 172.21.15.138:0/187109288 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 -- 172.21.15.138:0/187109288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 msgr2=0x7f341c1a0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 0x7f341c1a0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 -- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c104cb0 msgr2=0x7f341c19b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:02.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c104cb0 0x7f341c19b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 -- 172.21.15.138:0/187109288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341c1a11d0 con 0x7f341c10a6c0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.595+0000 7f3413e00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.596+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3404017030 con 0x7f341c10a6c0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.596+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341c1a14c0 con 0x7f341c10a6c0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.596+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3404004020 con 0x7f341c10a6c0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.596+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34040047a0 con 0x7f341c10a6c0 2024-08-20T01:21:02.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.596+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f341c1a7d50 con 0x7f341c10a6c0 2024-08-20T01:21:02.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.597+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f341c105090 con 0x7f341c10a6c0 2024-08-20T01:21:02.598 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.599+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3404009a30 con 0x7f341c10a6c0 2024-08-20T01:21:02.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.601+0000 7f3412a00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 0x7f34000b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:02.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.601+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9453+0+0 (secure 0 0 0) 0x7f34040e4120 con 0x7f341c10a6c0 2024-08-20T01:21:02.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.602+0000 7f3419e00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 0x7f34000b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:02.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.602+0000 7f3419e00640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 0x7f34000b6200 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f340c009bd0 tx=0x7f340c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:02.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.602+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f34040ab030 con 0x7f341c10a6c0 2024-08-20T01:21:02.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:02 smithi138 ceph-mon[18076]: pgmap v762: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-08-20T01:21:02.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.851+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f341c19c250 con 0x7f341c10a6c0 2024-08-20T01:21:02.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.852+0000 7f3412a00640 1 -- 172.21.15.138:0/187109288 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11934 (secure 0 0 0) 0x7f34040aa930 con 0x7f341c10a6c0 2024-08-20T01:21:02.851 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:21:02.851 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":43,"btime":"2024-08-20T01:20:26:427904+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:20:26.156076+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:21:02.851 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 43 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 msgr2=0x7f34000b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 0x7f34000b6200 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f340c009bd0 tx=0x7f340c005f90 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 msgr2=0x7f341c19b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f340400d950 tx=0x7f340400de10 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 shutdown_connections 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f34000b3d40 0x7f34000b6200 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f341c10b040 0x7f341c1a0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f341c10a6c0 0x7f341c19b660 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 --2- 172.21.15.138:0/187109288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f341c104cb0 0x7f341c19b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:02.854 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 >> 172.21.15.138:0/187109288 conn(0x7f341c100720 msgr2=0x7f341c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:02.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 shutdown_connections 2024-08-20T01:21:02.855 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:02.855+0000 7f3421200640 1 -- 172.21.15.138:0/187109288 wait complete. 2024-08-20T01:21:02.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:02 smithi167 ceph-mon[21965]: pgmap v762: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-08-20T01:21:02.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:02 smithi142 ceph-mon[21799]: pgmap v762: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.1 MiB/s wr, 82 op/s 2024-08-20T01:21:03.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.59 MB/sec execute 555 sec latency 15759.979 ms 2024-08-20T01:21:03.627 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id a486140d --release --await' 2024-08-20T01:21:03.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id a486140d --release --await 2024-08-20T01:21:03.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:03 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/187109288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:03.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:03 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/187109288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:03.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:03 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/187109288' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:04.007 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:21:04.211 INFO:tasks.workunit.client.0.smithi138.stdout: 1 502636 3.58 MB/sec execute 556 sec latency 16760.041 ms 2024-08-20T01:21:04.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.755+0000 7fe2d1c00640 1 -- 172.21.15.138:0/718967432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc104b10 msgr2=0x7fe2cc104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.755+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/718967432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc104b10 0x7fe2cc104ef0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009900 tx=0x7fe2bc02a410 comp rx=0 tx=0).stop 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 -- 172.21.15.138:0/718967432 shutdown_connections 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/718967432 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc10aea0 0x7fe2cc1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/718967432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10a520 0x7fe2cc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/718967432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc104b10 0x7fe2cc104ef0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 -- 172.21.15.138:0/718967432 >> 172.21.15.138:0/718967432 conn(0x7fe2cc100540 msgr2=0x7fe2cc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 -- 172.21.15.138:0/718967432 shutdown_connections 2024-08-20T01:21:04.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.756+0000 7fe2d1c00640 1 -- 172.21.15.138:0/718967432 wait complete. 2024-08-20T01:21:04.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.758+0000 7fe2d1c00640 1 Processor -- start 2024-08-20T01:21:04.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.758+0000 7fe2d1c00640 1 -- start start 2024-08-20T01:21:04.759 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.759+0000 7fe2d1c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 0x7fe2cc0743a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.760+0000 7fe2d1c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 0x7fe2cc0748e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.760+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 0x7fe2cc0743a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.760+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 0x7fe2cc0743a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38308/0 (socket says 172.21.15.138:38308) 2024-08-20T01:21:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.760+0000 7fe2cbe00640 1 -- 172.21.15.138:0/748377266 learned_addr learned my addr 172.21.15.138:0/748377266 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:04.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.760+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10aea0 0x7fe2cc06d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc114bb0 con 0x7fe2cc10aea0 2024-08-20T01:21:04.761 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc114a30 con 0x7fe2cc10a520 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc114d30 con 0x7fe2cc104b10 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2cb400640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 0x7fe2cc0748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2d1200640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10aea0 0x7fe2cc06d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2cb400640 1 -- 172.21.15.138:0/748377266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 msgr2=0x7fe2cc0743a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.761+0000 7fe2cb400640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 0x7fe2cc0743a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.762 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.762+0000 7fe2cb400640 1 -- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10aea0 msgr2=0x7fe2cc06d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.762+0000 7fe2cb400640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10aea0 0x7fe2cc06d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.762+0000 7fe2cb400640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc06deb0 con 0x7fe2cc10a520 2024-08-20T01:21:04.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.762+0000 7fe2cb400640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 0x7fe2cc0748e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005fd0 tx=0x7fe2b800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.763+0000 7fe2c8c00640 1 -- 172.21.15.138:0/748377266 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8009c30 con 0x7fe2cc10a520 2024-08-20T01:21:04.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.763+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc06e1a0 con 0x7fe2cc10a520 2024-08-20T01:21:04.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.763+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc0731e0 con 0x7fe2cc10a520 2024-08-20T01:21:04.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.763+0000 7fe2c8c00640 1 -- 172.21.15.138:0/748377266 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2b8009dd0 con 0x7fe2cc10a520 2024-08-20T01:21:04.764 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.763+0000 7fe2c8c00640 1 -- 172.21.15.138:0/748377266 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8010460 con 0x7fe2cc10a520 2024-08-20T01:21:04.764 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:04 smithi138 ceph-mon[18076]: pgmap v763: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 910 KiB/s wr, 52 op/s 2024-08-20T01:21:04.776 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.775+0000 7fe2c8c00640 1 -- 172.21.15.138:0/748377266 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe2b80035f0 con 0x7fe2cc10a520 2024-08-20T01:21:04.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.778+0000 7fe2c8c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 0x7fe2b00bec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.779+0000 7fe2c8c00640 1 -- 172.21.15.138:0/748377266 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7fe2b80e3a60 con 0x7fe2cc10a520 2024-08-20T01:21:04.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.779+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 0x7fe2b00bec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.780+0000 7fe2d5c17740 1 Processor -- start 2024-08-20T01:21:04.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.780+0000 7fe2d5c17740 1 -- start start 2024-08-20T01:21:04.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.780+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 0x7fe2b00bec10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c64a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c64cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b70c5df670 con 0x55b70c64a730 2024-08-20T01:21:04.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b70c5b3d70 con 0x55b70c63e7a0 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b70c5a3270 con 0x55b70c63de80 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38312/0 (socket says 172.21.15.138:38312) 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.781+0000 7fe2cbe00640 1 -- 172.21.15.138:0/1218595 learned_addr learned my addr 172.21.15.138:0/1218595 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cb400640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c64a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2d1200640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c64cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cbe00640 1 -- 172.21.15.138:0/1218595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 msgr2=0x55b70c64a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c64a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cbe00640 1 -- 172.21.15.138:0/1218595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 msgr2=0x55b70c64cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c64cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cbe00640 1 -- 172.21.15.138:0/1218595 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70c53f870 con 0x55b70c63de80 2024-08-20T01:21:04.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.782+0000 7fe2cb400640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c64a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:21:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.783+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005070 tx=0x7fe2bc03c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f44405756a9be4b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.783+0000 7fe2c1600640 1 -- 172.21.15.138:0/1218595 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc035050 con 0x55b70c63de80 2024-08-20T01:21:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.784+0000 7fe2c1600640 1 -- 172.21.15.138:0/1218595 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc04d070 con 0x55b70c63de80 2024-08-20T01:21:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.785+0000 7fe2d5c17740 1 -- 172.21.15.138:0/1218595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 msgr2=0x55b70c63e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.785+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc005070 tx=0x7fe2bc03c000 comp rx=0 tx=0).stop 2024-08-20T01:21:04.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 -- 172.21.15.138:0/1218595 shutdown_connections 2024-08-20T01:21:04.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c64cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c64a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/1218595 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c63e260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 -- 172.21.15.138:0/1218595 >> 172.21.15.138:0/1218595 conn(0x55b70c63bcb0 msgr2=0x55b70c63ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 -- 172.21.15.138:0/1218595 shutdown_connections 2024-08-20T01:21:04.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.786+0000 7fe2d5c17740 1 -- 172.21.15.138:0/1218595 wait complete. 2024-08-20T01:21:04.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.787+0000 7fe2d5c17740 1 -- start start 2024-08-20T01:21:04.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.787+0000 7fe2d5c17740 1 Processor -- start 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c6472b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c6477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b70c529ff0 con 0x55b70c64a730 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b70c5a72b0 con 0x55b70c63e7a0 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b70c405bb0 con 0x55b70c63de80 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.788+0000 7fe2d5c17740 10 client.0 _ll_register_callbacks cb 0x55b70c2aed70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:04.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.789+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.789+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38324/0 (socket says 172.21.15.138:38324) 2024-08-20T01:21:04.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.789+0000 7fe2cbe00640 1 -- 172.21.15.138:0/192357941 learned_addr learned my addr 172.21.15.138:0/192357941 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:04.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.789+0000 7fe2cb400640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c6472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.789+0000 7fe2d1200640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c6477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cb400640 1 -- 172.21.15.138:0/192357941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 msgr2=0x55b70c649e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cb400640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cb400640 1 -- 172.21.15.138:0/192357941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 msgr2=0x55b70c6477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cb400640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c6477f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cb400640 1 -- 172.21.15.138:0/192357941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70c302b00 con 0x55b70c63e7a0 2024-08-20T01:21:04.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.790+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.791+0000 7fe2cb400640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c6472b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80e0db0 tx=0x7fe2b80a9ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a8ed63324e85d69 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.791+0000 7fe2b6000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.791+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8014030 con 0x55b70c63e7a0 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.791+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7fe2b80a9550 con 0x55b70c63e7a0 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.792+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2bc04d070 con 0x55b70c63e7a0 2024-08-20T01:21:04.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.793+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe2b8020040 con 0x55b70c63e7a0 2024-08-20T01:21:04.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.793+0000 7fe2d5c17740 10 client.27038 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:04.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.793+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b70c379960 con 0x55b70c63e7a0 2024-08-20T01:21:04.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.793+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2bc04d070 con 0x55b70c63e7a0 2024-08-20T01:21:04.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe2b80a9050 con 0x55b70c63e7a0 2024-08-20T01:21:04.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 10 client.27038 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 10 client.27038 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 20 client.27038 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c66acf0 0x55b70c66d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 4 client.27038 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.794+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b70c5b5cc0 con 0x55b70c66acf0 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.795+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c66acf0 0x55b70c66d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.796+0000 7fe2b6000640 10 client.27038 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:04.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.796+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c66acf0 0x55b70c66d140 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61a05f3c4918e7ba server_cookie=f2c05ede1c2ad14f in_seq=0 out_seq=0 2024-08-20T01:21:04.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.797+0000 7fe2b6000640 1 -- 172.21.15.138:0/192357941 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b70c5b5cc0 con 0x55b70c66acf0 2024-08-20T01:21:04.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.797+0000 7fe2b6000640 10 client.27038 handle_command_reply: tid=0 2024-08-20T01:21:04.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.797+0000 7fe2d5c17740 1 client.27038 shutdown 2024-08-20T01:21:04.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.799+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 msgr2=0x55b70c6472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.799+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c6472b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80e0db0 tx=0x7fe2b80a9ec0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.799+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 shutdown_connections 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.799+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c66acf0 0x55b70c66d140 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c64a730 0x55b70c6477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c6472b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/192357941 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63de80 0x55b70c649e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 >> 172.21.15.138:0/192357941 conn(0x55b70c63bcb0 msgr2=0x55b70c641820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2b6000640 0 client.27038 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 shutdown_connections 2024-08-20T01:21:04.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 1 -- 172.21.15.138:0/192357941 wait complete. 2024-08-20T01:21:04.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.800+0000 7fe2d5c17740 20 client.27038 trim_cache size 0 max 16384 2024-08-20T01:21:04.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.836+0000 7fe2d5c17740 1 Processor -- start 2024-08-20T01:21:04.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.836+0000 7fe2d5c17740 1 -- start start 2024-08-20T01:21:04.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c63ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c672aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac012a80 con 0x55b70c63de80 2024-08-20T01:21:04.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac010320 con 0x55b70c63e7a0 2024-08-20T01:21:04.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac00e150 con 0x55b70c64a730 2024-08-20T01:21:04.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d1200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38334/0 (socket says 172.21.15.138:38334) 2024-08-20T01:21:04.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.837+0000 7fe2d1200640 1 -- 172.21.15.138:0/3167253947 learned_addr learned my addr 172.21.15.138:0/3167253947 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:04.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2cb400640 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c672aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2d1200640 1 -- 172.21.15.138:0/3167253947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 msgr2=0x55b70c672aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2d1200640 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c672aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2d1200640 1 -- 172.21.15.138:0/3167253947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 msgr2=0x55b70c63ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2d1200640 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c63ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.838+0000 7fe2d1200640 1 -- 172.21.15.138:0/3167253947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b70c647960 con 0x55b70c64a730 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.839+0000 7fe2d1200640 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009410 tx=0x7fe2c40098d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7195c12c774cd70 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.839+0000 7fe2c0c00640 1 -- 172.21.15.138:0/3167253947 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c401a020 con 0x55b70c64a730 2024-08-20T01:21:04.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.839+0000 7fe2c0c00640 1 -- 172.21.15.138:0/3167253947 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c4016d00 con 0x55b70c64a730 2024-08-20T01:21:04.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.839+0000 7fe2c0c00640 1 -- 172.21.15.138:0/3167253947 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4015d60 con 0x55b70c64a730 2024-08-20T01:21:04.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.840+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3167253947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 msgr2=0x55b70c672fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.840+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4009410 tx=0x7fe2c40098d0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3167253947 shutdown_connections 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c64a730 0x55b70c672fe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c63e7a0 0x55b70c672aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3167253947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c63ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3167253947 >> 172.21.15.138:0/3167253947 conn(0x55b70c63bcb0 msgr2=0x55b70c63a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3167253947 shutdown_connections 2024-08-20T01:21:04.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.841+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3167253947 wait complete. 2024-08-20T01:21:04.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.842+0000 7fe2d5c17740 1 -- start start 2024-08-20T01:21:04.842 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.842+0000 7fe2d5c17740 1 Processor -- start 2024-08-20T01:21:04.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.844+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c650cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.843 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.844+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.844+0000 7fe2d5c17740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 0x55b70c6d4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.844+0000 7fe2d5c17740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac00e7e0 con 0x55b70c63de80 2024-08-20T01:21:04.844 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2cb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2cbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c650cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2cb400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38340/0 (socket says 172.21.15.138:38340) 2024-08-20T01:21:04.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2cb400640 1 -- 172.21.15.138:0/3880907631 learned_addr learned my addr 172.21.15.138:0/3880907631 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:04.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac00fc50 con 0x55b70c64a730 2024-08-20T01:21:04.845 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2ac00eeb0 con 0x55b70c63e7a0 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2d5c17740 10 client.0 _ll_register_callbacks cb 0x7fe2b8005bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.845+0000 7fe2cb400640 1 -- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 msgr2=0x55b70c6d4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2d1200640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 0x55b70c6d4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2cb400640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 0x55b70c6d4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2cb400640 1 -- 172.21.15.138:0/3880907631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 msgr2=0x55b70c650cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2cb400640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c650cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2cb400640 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b70c6d5170 con 0x55b70c63e7a0 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c650cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:21:04.846 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.846+0000 7fe2d1200640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 0x55b70c6d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:21:04.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.847+0000 7fe2cb400640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80e3d80 tx=0x7fe2b80a9780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=671c840be5d2017b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:04.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.847+0000 7fe2b6000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:21:04.847 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.847+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b80cc040 con 0x55b70c63e7a0 2024-08-20T01:21:04.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.847+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7fe2b80a68e0 con 0x55b70c63e7a0 2024-08-20T01:21:04.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.847+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c4016d00 con 0x55b70c63e7a0 2024-08-20T01:21:04.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe2b8020040 con 0x55b70c63e7a0 2024-08-20T01:21:04.848 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2d5c17740 10 client.26998 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b70c6d64a0 con 0x55b70c63e7a0 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c4016d00 con 0x55b70c63e7a0 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7fe2b80a4a10 con 0x55b70c63e7a0 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2d5c17740 10 client.26998 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2d5c17740 10 client.26998 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.848+0000 7fe2d5c17740 20 client.26998 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.849+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c6d6a90 0x55b70c6d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.849+0000 7fe2d5c17740 4 client.26998 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a486140d", "release": true, "await": true}] 2024-08-20T01:21:04.849 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.849+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a486140d", "release": true, "await": true}) -- 0x55b70c64cf00 con 0x55b70c6d6a90 2024-08-20T01:21:04.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.849+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c6d6a90 0x55b70c6d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:04.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.851+0000 7fe2cbe00640 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c6d6a90 0x55b70c6d8eb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50609884a9f93cd8 server_cookie=20de66d84eeae773 in_seq=0 out_seq=0 2024-08-20T01:21:04.850 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.851+0000 7fe2b6000640 10 client.26998 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:04.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.859+0000 7fe2b6000640 1 -- 172.21.15.138:0/3880907631 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b70c64cf00 con 0x55b70c6d6a90 2024-08-20T01:21:04.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.859+0000 7fe2b6000640 10 client.26998 handle_command_reply: tid=0 2024-08-20T01:21:04.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.859+0000 7fe2d5c17740 1 client.26998 shutdown 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 msgr2=0x55b70c6511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe2b80e3d80 tx=0x7fe2b80a9780 comp rx=0 tx=0).stop 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 shutdown_connections 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b70c6d6a90 0x55b70c6d8eb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b70c64a730 0x55b70c6d4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b70c63e7a0 0x55b70c6511f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 --2- 172.21.15.138:0/3880907631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b70c63de80 0x55b70c650cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 >> 172.21.15.138:0/3880907631 conn(0x55b70c63bcb0 msgr2=0x55b70c63ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.860 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.860+0000 7fe2b6000640 0 client.26998 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:04.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.861+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 shutdown_connections 2024-08-20T01:21:04.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.861+0000 7fe2d5c17740 1 -- 172.21.15.138:0/3880907631 wait complete. 2024-08-20T01:21:04.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.861+0000 7fe2d5c17740 20 client.26998 trim_cache size 0 max 16384 2024-08-20T01:21:04.861 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 43, 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 48, 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "a486140d": { 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 48, 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:21:04.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 137.3, 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:04.863 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.862+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 msgr2=0x7fe2b00bec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.862+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 0x7fe2b00bec10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000c00 tx=0x7fe2bc031400 comp rx=0 tx=0).stop 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.862+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 msgr2=0x7fe2cc0748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.862+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 0x7fe2cc0748e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8005fd0 tx=0x7fe2b800dde0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 shutdown_connections 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe2b00bc750 0x7fe2b00bec10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe2cc10aea0 0x7fe2cc06d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.864 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe2cc10a520 0x7fe2cc0748e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 --2- 172.21.15.138:0/748377266 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe2cc104b10 0x7fe2cc0743a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:04.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 >> 172.21.15.138:0/748377266 conn(0x7fe2cc100540 msgr2=0x7fe2cc100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:04.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.863+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 shutdown_connections 2024-08-20T01:21:04.865 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:04.864+0000 7fe2d1c00640 1 -- 172.21.15.138:0/748377266 wait complete. 2024-08-20T01:21:04.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:04 smithi167 ceph-mon[21965]: pgmap v763: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 910 KiB/s wr, 52 op/s 2024-08-20T01:21:04.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:04 smithi142 ceph-mon[21799]: pgmap v763: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 910 KiB/s wr, 52 op/s 2024-08-20T01:21:05.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 503206 3.57 MB/sec execute 557 sec latency 17408.057 ms 2024-08-20T01:21:05.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:05 smithi138 ceph-mon[18076]: osdmap e102: 12 total, 12 up, 12 in 2024-08-20T01:21:05.842 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a486140d', total seconds elapsed: 21.9 2024-08-20T01:21:05.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:05 smithi167 ceph-mon[21965]: osdmap e102: 12 total, 12 up, 12 in 2024-08-20T01:21:05.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:05 smithi142 ceph-mon[21799]: osdmap e102: 12 total, 12 up, 12 in 2024-08-20T01:21:06.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 503951 3.58 MB/sec execute 558 sec latency 70.778 ms 2024-08-20T01:21:06.784 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:06 smithi138 ceph-mon[18076]: pgmap v765: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 900 KiB/s wr, 72 op/s 2024-08-20T01:21:06.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:06 smithi167 ceph-mon[21965]: pgmap v765: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 900 KiB/s wr, 72 op/s 2024-08-20T01:21:06.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:06 smithi142 ceph-mon[21799]: pgmap v765: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 435 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 900 KiB/s wr, 72 op/s 2024-08-20T01:21:07.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 504753 3.58 MB/sec execute 559 sec latency 392.195 ms 2024-08-20T01:21:08.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 505698 3.58 MB/sec execute 560 sec latency 508.011 ms 2024-08-20T01:21:08.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:08 smithi138 ceph-mon[18076]: pgmap v766: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 782 KiB/s wr, 80 op/s 2024-08-20T01:21:08.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:08 smithi167 ceph-mon[21965]: pgmap v766: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 782 KiB/s wr, 80 op/s 2024-08-20T01:21:08.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:08 smithi142 ceph-mon[21799]: pgmap v766: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 782 KiB/s wr, 80 op/s 2024-08-20T01:21:09.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 506183 3.58 MB/sec execute 561 sec latency 527.296 ms 2024-08-20T01:21:10.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 508355 3.60 MB/sec execute 562 sec latency 555.071 ms 2024-08-20T01:21:10.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:10 smithi138 ceph-mon[18076]: pgmap v767: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 839 KiB/s wr, 80 op/s 2024-08-20T01:21:10.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:10 smithi167 ceph-mon[21965]: pgmap v767: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 839 KiB/s wr, 80 op/s 2024-08-20T01:21:10.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:10 smithi142 ceph-mon[21799]: pgmap v767: 129 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 120 active+clean; 434 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 839 KiB/s wr, 80 op/s 2024-08-20T01:21:11.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 509568 3.60 MB/sec execute 563 sec latency 24.030 ms 2024-08-20T01:21:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 -- 172.21.15.138:0/1313565411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71180751f0 msgr2=0x7f71180755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 --2- 172.21.15.138:0/1313565411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71180751f0 0x7f71180755d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7108009900 tx=0x7f710802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 -- 172.21.15.138:0/1313565411 shutdown_connections 2024-08-20T01:21:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 --2- 172.21.15.138:0/1313565411 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f711810eff0 0x7f71181113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 --2- 172.21.15.138:0/1313565411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7118075b10 0x7f711810eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 --2- 172.21.15.138:0/1313565411 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71180751f0 0x7f71180755d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 -- 172.21.15.138:0/1313565411 >> 172.21.15.138:0/1313565411 conn(0x7f71180fe430 msgr2=0x7f7118100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 -- 172.21.15.138:0/1313565411 shutdown_connections 2024-08-20T01:21:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.168+0000 7f711ee00640 1 -- 172.21.15.138:0/1313565411 wait complete. 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.169+0000 7f711ee00640 1 Processor -- start 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.169+0000 7f711ee00640 1 -- start start 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7118075b10 0x7f711819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 0x7f71181a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114c20 con 0x7f711810eff0 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114aa0 con 0x7f71180751f0 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ee00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7118114da0 con 0x7f7118075b10 2024-08-20T01:21:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f7117e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f711ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 0x7f71181a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f7117e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56938/0 (socket says 172.21.15.138:56938) 2024-08-20T01:21:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.170+0000 7f7117e00640 1 -- 172.21.15.138:0/196914812 learned_addr learned my addr 172.21.15.138:0/196914812 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 -- 172.21.15.138:0/196914812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7118075b10 msgr2=0x7f711819b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7118075b10 0x7f711819b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 -- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 msgr2=0x7f71181a0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 0x7f71181a0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 -- 172.21.15.138:0/196914812 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71181a1220 con 0x7f71180751f0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f711ce00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 0x7f71181a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7117e00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7108005ab0 tx=0x7f7108003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f7114c00640 1 -- 172.21.15.138:0/196914812 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710803f070 con 0x7f71180751f0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71181a14b0 con 0x7f71180751f0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.171+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71181a7bd0 con 0x7f71180751f0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.172+0000 7f7114c00640 1 -- 172.21.15.138:0/196914812 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7108031d60 con 0x7f71180751f0 2024-08-20T01:21:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.172+0000 7f7114c00640 1 -- 172.21.15.138:0/196914812 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7108049060 con 0x7f71180751f0 2024-08-20T01:21:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.175+0000 7f7114c00640 1 -- 172.21.15.138:0/196914812 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7108042070 con 0x7f71180751f0 2024-08-20T01:21:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.175+0000 7f7121dce740 1 Processor -- start 2024-08-20T01:21:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.175+0000 7f7121dce740 1 -- start start 2024-08-20T01:21:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2ab7e80 0x55d1a2aba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2a21210 con 0x55d1a2ab7e80 2024-08-20T01:21:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2a10710 con 0x55d1a2aab5d0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7121dce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2a14750 con 0x55d1a2aabef0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56954/0 (socket says 172.21.15.138:56954) 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 -- 172.21.15.138:0/1545186014 learned_addr learned my addr 172.21.15.138:0/1545186014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7117e00640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7117400640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2ab7e80 0x55d1a2aba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 -- 172.21.15.138:0/1545186014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 msgr2=0x55d1a2ab7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 -- 172.21.15.138:0/1545186014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2ab7e80 msgr2=0x55d1a2aba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2ab7e80 0x55d1a2aba270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f711ce00640 1 -- 172.21.15.138:0/1545186014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a29dcde0 con 0x55d1a2aab5d0 2024-08-20T01:21:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.176+0000 7f7117e00640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:21:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f711ce00640 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7110011760 tx=0x7f7110011c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ab4c3209836079 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f7114c00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 0x7f70fc0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f710d600640 1 -- 172.21.15.138:0/1545186014 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110004d20 con 0x55d1a2aab5d0 2024-08-20T01:21:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f710d600640 1 -- 172.21.15.138:0/1545186014 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711000f9a0 con 0x55d1a2aab5d0 2024-08-20T01:21:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f710d600640 1 -- 172.21.15.138:0/1545186014 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100043b0 con 0x55d1a2aab5d0 2024-08-20T01:21:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f7114c00640 1 -- 172.21.15.138:0/196914812 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7f7108104870 con 0x7f71180751f0 2024-08-20T01:21:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f7117e00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 0x7f70fc0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f7121dce740 1 -- 172.21.15.138:0/1545186014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 msgr2=0x55d1a2aab9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.177+0000 7f7121dce740 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7110011760 tx=0x7f7110011c20 comp rx=0 tx=0).stop 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 -- 172.21.15.138:0/1545186014 shutdown_connections 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2ab7e80 0x55d1a2aba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 --2- 172.21.15.138:0/1545186014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2aab9b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 -- 172.21.15.138:0/1545186014 >> 172.21.15.138:0/1545186014 conn(0x55d1a2aa93b0 msgr2=0x55d1a2aaa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 -- 172.21.15.138:0/1545186014 shutdown_connections 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7117e00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 0x7f70fc0b6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7108101c70 tx=0x7f7108031400 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 -- 172.21.15.138:0/1545186014 wait complete. 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 -- start start 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.178+0000 7f7121dce740 1 Processor -- start 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ab5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2949ab0 con 0x55d1a2aab5d0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d1a29b7140 con 0x55d1a2aabef0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d1a2abab60 con 0x55d1a2ab7e80 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7121dce740 10 client.0 _ll_register_callbacks cb 0x55d1a29ca5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f711ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f711ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58290/0 (socket says 172.21.15.138:58290) 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f7117400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.179+0000 7f711ce00640 1 -- 172.21.15.138:0/963186107 learned_addr learned my addr 172.21.15.138:0/963186107 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ab5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 -- 172.21.15.138:0/963186107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 msgr2=0x55d1a2ab5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ab5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 -- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 msgr2=0x55d1a2ab52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 -- 172.21.15.138:0/963186107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a29b8170 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f7117400640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f711ce00640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7110006010 tx=0x7f7110011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f84f7a9a72cfdb8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f7102000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002e060 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d1a29e7a30 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.180+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7f7110004130 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7110035050 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 10 client.17004 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d1a2773790 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d1a29e7a30 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f7110018070 con 0x55d1a2aab5d0 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 10 client.17004 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 20 client.17004 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2ac7c60 0x55d1a2aca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 4 client.17004 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.181+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1a2a23160 con 0x55d1a2ac7c60 2024-08-20T01:21:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.182+0000 7f7117400640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2ac7c60 0x55d1a2aca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.182+0000 7f7117400640 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2ac7c60 0x55d1a2aca050 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3692c662e1f2bce5 server_cookie=8793f168e54ed9f7 in_seq=0 out_seq=0 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.182+0000 7f7102000640 10 client.17004 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7102000640 1 -- 172.21.15.138:0/963186107 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1a2a23160 con 0x55d1a2ac7c60 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7102000640 10 client.17004 handle_command_reply: tid=0 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 client.17004 shutdown 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 msgr2=0x55d1a2ab4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7110006010 tx=0x7f7110011440 comp rx=0 tx=0).stop 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 shutdown_connections 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2ac7c60 0x55d1a2aca050 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ab5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 --2- 172.21.15.138:0/963186107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4d90 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7102000640 0 client.17004 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 >> 172.21.15.138:0/963186107 conn(0x55d1a2aa93b0 msgr2=0x55d1a2aaef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 shutdown_connections 2024-08-20T01:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 1 -- 172.21.15.138:0/963186107 wait complete. 2024-08-20T01:21:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.183+0000 7f7121dce740 20 client.17004 trim_cache size 0 max 16384 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.210+0000 7f7121dce740 1 Processor -- start 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.210+0000 7f7121dce740 1 -- start start 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2acf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b4800 con 0x55d1a2aabef0 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b20a0 con 0x55d1a2aab5d0 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7121dce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71181aff10 con 0x55d1a2ab7e80 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58294/0 (socket says 172.21.15.138:58294) 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 -- 172.21.15.138:0/1960796141 learned_addr learned my addr 172.21.15.138:0/1960796141 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 -- 172.21.15.138:0/1960796141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 msgr2=0x55d1a2acf790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2acf790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 -- 172.21.15.138:0/1960796141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 msgr2=0x55d1a2ab4600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.211+0000 7f7117400640 1 -- 172.21.15.138:0/1960796141 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1a27ec620 con 0x55d1a2aabef0 2024-08-20T01:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7117400640 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55d1a2aca730 tx=0x7f7104005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31a9f56853eab1f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f710cc00640 1 -- 172.21.15.138:0/1960796141 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710401a330 con 0x55d1a2aabef0 2024-08-20T01:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f710cc00640 1 -- 172.21.15.138:0/1960796141 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710401a980 con 0x55d1a2aabef0 2024-08-20T01:21:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f710cc00640 1 -- 172.21.15.138:0/1960796141 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104011910 con 0x55d1a2aabef0 2024-08-20T01:21:12.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 509726 3.60 MB/sec execute 564 sec latency 823.717 ms 2024-08-20T01:21:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 -- 172.21.15.138:0/1960796141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 msgr2=0x55d1a2ab5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x55d1a2aca730 tx=0x7f7104005f60 comp rx=0 tx=0).stop 2024-08-20T01:21:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 -- 172.21.15.138:0/1960796141 shutdown_connections 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2ab7e80 0x55d1a2acf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aabef0 0x55d1a2ab5470 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 --2- 172.21.15.138:0/1960796141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.212+0000 7f7121dce740 1 -- 172.21.15.138:0/1960796141 >> 172.21.15.138:0/1960796141 conn(0x55d1a2aa93b0 msgr2=0x55d1a2ab4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.213+0000 7f7121dce740 1 -- 172.21.15.138:0/1960796141 shutdown_connections 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.213+0000 7f7121dce740 1 -- 172.21.15.138:0/1960796141 wait complete. 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.213+0000 7f7121dce740 1 -- start start 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.213+0000 7f7121dce740 1 Processor -- start 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2abfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ac0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b0560 con 0x55d1a2aab5d0 2024-08-20T01:21:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b19d0 con 0x55d1a2ab7e80 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71181b0c30 con 0x55d1a2aabef0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7121dce740 10 client.0 _ll_register_callbacks cb 0x7f7110011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58310/0 (socket says 172.21.15.138:58310) 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 -- 172.21.15.138:0/2793035049 learned_addr learned my addr 172.21.15.138:0/2793035049 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f7117400640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2abfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ac0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 -- 172.21.15.138:0/2793035049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 msgr2=0x55d1a2abfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2abfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 -- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2ab7e80 msgr2=0x55d1a2ac0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ac0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.214+0000 7f711ce00640 1 -- 172.21.15.138:0/2793035049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1a2ab0f90 con 0x55d1a2aab5d0 2024-08-20T01:21:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f711ce00640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7110005860 tx=0x7f7110005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dbc2a0ac1e8dcde server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7102000640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711002e070 con 0x55d1a2aab5d0 2024-08-20T01:21:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9541+0+0 (secure 0 0 0) 0x7f711000f790 con 0x55d1a2aab5d0 2024-08-20T01:21:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f710401a980 con 0x55d1a2aab5d0 2024-08-20T01:21:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7110004f50 con 0x55d1a2aab5d0 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7121dce740 10 client.17010 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d1a2b31910 con 0x55d1a2aab5d0 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.215+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f710401a980 con 0x55d1a2aab5d0 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f7110018040 con 0x55d1a2aab5d0 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 10 client.17010 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 10 client.17010 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 20 client.17010 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2b32000 0x55d1a2b34420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 4 client.17010 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1a2a1ec00 con 0x55d1a2b32000 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.216+0000 7f7117400640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2b32000 0x55d1a2b34420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.217+0000 7f7117400640 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2b32000 0x55d1a2b34420 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d250d43127f5d162 server_cookie=7d9d8c9eee81cc0e in_seq=0 out_seq=0 2024-08-20T01:21:12.220 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.217+0000 7f7102000640 10 client.17010 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7102000640 1 -- 172.21.15.138:0/2793035049 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1a2a1ec00 con 0x55d1a2b32000 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7102000640 10 client.17010 handle_command_reply: tid=0 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 client.17010 shutdown 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 msgr2=0x55d1a2ab2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7110005860 tx=0x7f7110005bd0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 shutdown_connections 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d1a2b32000 0x55d1a2b34420 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d1a2ab7e80 0x55d1a2ac0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d1a2aabef0 0x55d1a2abfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 --2- 172.21.15.138:0/2793035049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d1a2aab5d0 0x55d1a2ab2750 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 >> 172.21.15.138:0/2793035049 conn(0x55d1a2aa93b0 msgr2=0x55d1a2ab5580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.274+0000 7f7102000640 0 client.17010 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.275+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 shutdown_connections 2024-08-20T01:21:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.275+0000 7f7121dce740 1 -- 172.21.15.138:0/2793035049 wait complete. 2024-08-20T01:21:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.275+0000 7f7121dce740 20 client.17010 trim_cache size 0 max 16384 2024-08-20T01:21:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:21:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:21:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:21:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:21:12.276 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 msgr2=0x7f70fc0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 0x7f70fc0b6280 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7108101c70 tx=0x7f7108031400 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 msgr2=0x7f711819b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7108005ab0 tx=0x7f7108003c80 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 shutdown_connections 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70fc0b3dc0 0x7f70fc0b6280 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f711810eff0 0x7f71181a0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7118075b10 0x7f711819b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 --2- 172.21.15.138:0/196914812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71180751f0 0x7f711819b180 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 >> 172.21.15.138:0/196914812 conn(0x7f71180fe430 msgr2=0x7f71181111b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 shutdown_connections 2024-08-20T01:21:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:12.276+0000 7f711ee00640 1 -- 172.21.15.138:0/196914812 wait complete. 2024-08-20T01:21:12.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:12 smithi138 ceph-mon[18076]: pgmap v768: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-08-20T01:21:12.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:12 smithi167 ceph-mon[21965]: pgmap v768: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-08-20T01:21:12.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:12 smithi142 ceph-mon[21799]: pgmap v768: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 1.4 MiB/s wr, 127 op/s 2024-08-20T01:21:13.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 510436 3.59 MB/sec execute 565 sec latency 851.797 ms 2024-08-20T01:21:14.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 512107 3.61 MB/sec execute 566 sec latency 17.931 ms 2024-08-20T01:21:14.663 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:14 smithi138 ceph-mon[18076]: pgmap v769: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-08-20T01:21:14.663 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:14 smithi138 ceph-mon[18076]: osdmap e103: 12 total, 12 up, 12 in 2024-08-20T01:21:14.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:14 smithi167 ceph-mon[21965]: pgmap v769: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-08-20T01:21:14.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:14 smithi167 ceph-mon[21965]: osdmap e103: 12 total, 12 up, 12 in 2024-08-20T01:21:14.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:14 smithi142 ceph-mon[21799]: pgmap v769: 129 pgs: 129 active+clean; 437 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-08-20T01:21:14.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:14 smithi142 ceph-mon[21799]: osdmap e103: 12 total, 12 up, 12 in 2024-08-20T01:21:15.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 513272 3.61 MB/sec execute 567 sec latency 155.414 ms 2024-08-20T01:21:16.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 513272 3.60 MB/sec execute 568 sec latency 1155.499 ms 2024-08-20T01:21:16.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:16 smithi138 ceph-mon[18076]: pgmap v771: 129 pgs: 129 active+clean; 446 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-08-20T01:21:16.793 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:16 smithi142 ceph-mon[21799]: pgmap v771: 129 pgs: 129 active+clean; 446 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-08-20T01:21:16.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:16 smithi167 ceph-mon[21965]: pgmap v771: 129 pgs: 129 active+clean; 446 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 188 op/s 2024-08-20T01:21:17.212 INFO:tasks.workunit.client.0.smithi138.stdout: 1 513272 3.60 MB/sec execute 569 sec latency 2155.545 ms 2024-08-20T01:21:18.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 513272 3.59 MB/sec execute 570 sec latency 3155.661 ms 2024-08-20T01:21:18.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:18 smithi167 ceph-mon[21965]: pgmap v772: 129 pgs: 129 active+clean; 441 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-08-20T01:21:18.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:18 smithi142 ceph-mon[21799]: pgmap v772: 129 pgs: 129 active+clean; 441 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-08-20T01:21:19.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:18 smithi138 ceph-mon[18076]: pgmap v772: 129 pgs: 129 active+clean; 441 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.6 MiB/s wr, 194 op/s 2024-08-20T01:21:19.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 513272 3.59 MB/sec execute 571 sec latency 4155.799 ms 2024-08-20T01:21:20.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 515432 3.60 MB/sec execute 572 sec latency 4196.092 ms 2024-08-20T01:21:20.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:20 smithi167 ceph-mon[21965]: pgmap v773: 129 pgs: 129 active+clean; 441 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-08-20T01:21:20.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:20 smithi142 ceph-mon[21799]: pgmap v773: 129 pgs: 129 active+clean; 441 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-08-20T01:21:21.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:20 smithi138 ceph-mon[18076]: pgmap v773: 129 pgs: 129 active+clean; 441 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 194 op/s 2024-08-20T01:21:21.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 516709 3.61 MB/sec execute 573 sec latency 36.466 ms 2024-08-20T01:21:22.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 516816 3.60 MB/sec execute 574 sec latency 923.708 ms 2024-08-20T01:21:22.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:22 smithi167 ceph-mon[21965]: pgmap v774: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:21:22.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:22 smithi142 ceph-mon[21799]: pgmap v774: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:21:23.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:22 smithi138 ceph-mon[18076]: pgmap v774: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 159 op/s 2024-08-20T01:21:23.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 516816 3.59 MB/sec execute 575 sec latency 1923.852 ms 2024-08-20T01:21:24.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 516816 3.59 MB/sec execute 576 sec latency 2923.925 ms 2024-08-20T01:21:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:24 smithi167 ceph-mon[21965]: pgmap v775: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-08-20T01:21:24.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:24 smithi142 ceph-mon[21799]: pgmap v775: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-08-20T01:21:25.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:24 smithi138 ceph-mon[18076]: pgmap v775: 129 pgs: 129 active+clean; 443 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 154 op/s 2024-08-20T01:21:25.213 INFO:tasks.workunit.client.0.smithi138.stdout: 1 519009 3.61 MB/sec execute 577 sec latency 2952.740 ms 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 -- 172.21.15.138:0/2104114367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd806a7a0 msgr2=0x7f8fd806ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/2104114367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd806a7a0 0x7f8fd806ab80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009900 tx=0x7f8fc802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 -- 172.21.15.138:0/2104114367 shutdown_connections 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/2104114367 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd8105970 0x7f8fd810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/2104114367 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/2104114367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd806a7a0 0x7f8fd806ab80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 -- 172.21.15.138:0/2104114367 >> 172.21.15.138:0/2104114367 conn(0x7f8fd8077270 msgr2=0x7f8fd8077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 -- 172.21.15.138:0/2104114367 shutdown_connections 2024-08-20T01:21:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.412+0000 7f8fd7e00640 1 -- 172.21.15.138:0/2104114367 wait complete. 2024-08-20T01:21:25.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.413+0000 7f8fd7e00640 1 Processor -- start 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.413+0000 7f8fd7e00640 1 -- start start 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.413+0000 7f8fd7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 0x7f8fd81019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8105970 0x7f8fd8102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8112810 con 0x7f8fd8105970 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8112690 con 0x7f8fd8104fd0 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd8112990 con 0x7f8fd806a7a0 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8105970 0x7f8fd8102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 0x7f8fd81019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55490/0 (socket says 172.21.15.138:55490) 2024-08-20T01:21:25.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 -- 172.21.15.138:0/4096418362 learned_addr learned my addr 172.21.15.138:0/4096418362 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 -- 172.21.15.138:0/4096418362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 msgr2=0x7f8fd81019e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 0x7f8fd81019e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 -- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8105970 msgr2=0x7f8fd8102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8105970 0x7f8fd8102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6000640 1 -- 172.21.15.138:0/4096418362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fd81a7160 con 0x7f8fd8104fd0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.414+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 0x7f8fd81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fd6000640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400ea60 tx=0x7f8fc400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fcf400640 1 -- 172.21.15.138:0/4096418362 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc400cde0 con 0x7f8fd8104fd0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fd81a7450 con 0x7f8fd8104fd0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fcf400640 1 -- 172.21.15.138:0/4096418362 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc40035f0 con 0x7f8fd8104fd0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fcf400640 1 -- 172.21.15.138:0/4096418362 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4010690 con 0x7f8fd8104fd0 2024-08-20T01:21:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.415+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fd81a79f0 con 0x7f8fd8104fd0 2024-08-20T01:21:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.418+0000 7f8fe0685740 1 Processor -- start 2024-08-20T01:21:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.418+0000 7f8fe0685740 1 -- start start 2024-08-20T01:21:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.418+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.418+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 0x555b3cfdee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfe17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fe0685740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b3cf48210 con 0x555b3cfd3430 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fe0685740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555b3cf37710 con 0x555b3cfdf3c0 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fe0685740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b3cf3b750 con 0x555b3cfd2b10 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 0x555b3cfdee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39256/0 (socket says 172.21.15.138:39256) 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 -- 172.21.15.138:0/1405691762 learned_addr learned my addr 172.21.15.138:0/1405691762 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fcf400640 1 -- 172.21.15.138:0/4096418362 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8fc40037a0 con 0x7f8fd8104fd0 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 -- 172.21.15.138:0/1405691762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 msgr2=0x555b3cfe17b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfe17b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 -- 172.21.15.138:0/1405691762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 msgr2=0x555b3cfdee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 0x555b3cfdee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd7400640 1 -- 172.21.15.138:0/1405691762 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3cf03de0 con 0x555b3cfd2b10 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.419+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 0x555b3cfdee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.420+0000 7f8fd7400640 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009fd0 tx=0x7f8fd00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc4898f43343f15 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.420+0000 7f8fc3e00640 1 -- 172.21.15.138:0/1405691762 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0004d20 con 0x555b3cfd2b10 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.420+0000 7f8fc3e00640 1 -- 172.21.15.138:0/1405691762 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd0004050 con 0x555b3cfd2b10 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.420+0000 7f8fc3e00640 1 -- 172.21.15.138:0/1405691762 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0005280 con 0x555b3cfd2b10 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fcf400640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 0x7f8fbc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 -- 172.21.15.138:0/1405691762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 msgr2=0x555b3cfd2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009fd0 tx=0x7f8fd00118d0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fcf400640 1 -- 172.21.15.138:0/4096418362 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f8fc40e4670 con 0x7f8fd8104fd0 2024-08-20T01:21:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 -- 172.21.15.138:0/1405691762 shutdown_connections 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfe17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd3430 0x555b3cfdee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1405691762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfd2ef0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 -- 172.21.15.138:0/1405691762 >> 172.21.15.138:0/1405691762 conn(0x555b3cfd0940 msgr2=0x555b3cfd1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 0x7f8fbc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 -- 172.21.15.138:0/1405691762 shutdown_connections 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.421+0000 7f8fe0685740 1 -- 172.21.15.138:0/1405691762 wait complete. 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.422+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 0x7f8fbc0b6200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005ab0 tx=0x7f8fc8005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.422+0000 7f8fe0685740 1 -- start start 2024-08-20T01:21:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.422+0000 7f8fe0685740 1 Processor -- start 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfdbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfdc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x555b3ce70ab0 con 0x555b3cfd2b10 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x555b3cede140 con 0x555b3cfdf3c0 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x555b3cfe20a0 con 0x555b3cfd3430 2024-08-20T01:21:25.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fe0685740 10 client.0 _ll_register_callbacks cb 0x555b3cef15d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38834/0 (socket says 172.21.15.138:38834) 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fd6a00640 1 -- 172.21.15.138:0/2933703406 learned_addr learned my addr 172.21.15.138:0/2933703406 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fd6000640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfdbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.423+0000 7f8fd7400640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfdc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 -- 172.21.15.138:0/2933703406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 msgr2=0x555b3cfdbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfdbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 -- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 msgr2=0x555b3cfdc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfdc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 -- 172.21.15.138:0/2933703406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3cedf170 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009900 tx=0x7f8fc8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc51174a6e76411 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8035040 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd0004050 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.424+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f8fc80037f0 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8fc80039f0 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 10 client.17022 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555b3cc9a790 con 0x555b3cfd2b10 2024-08-20T01:21:25.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fd0004050 con 0x555b3cfd2b10 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f8fc8044030 con 0x555b3cfd2b10 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 10 client.17022 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 20 client.17022 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.425+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3cff77a0 0x555b3cff9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.426+0000 7f8fe0685740 4 client.17022 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.426+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b3cf4a160 con 0x555b3cff77a0 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.426+0000 7f8fd6000640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3cff77a0 0x555b3cff9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.426+0000 7f8fd6000640 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3cff77a0 0x555b3cff9b90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fc7c3c048eb6f94 server_cookie=cb65a138cce3e4f4 in_seq=0 out_seq=0 2024-08-20T01:21:25.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.426+0000 7f8fc0c00640 10 client.17022 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fc0c00640 1 -- 172.21.15.138:0/2933703406 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b3cf4a160 con 0x555b3cff77a0 2024-08-20T01:21:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fc0c00640 10 client.17022 handle_command_reply: tid=0 2024-08-20T01:21:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 client.17022 shutdown 2024-08-20T01:21:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 msgr2=0x555b3cfdeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8009900 tx=0x7f8fc8009260 comp rx=0 tx=0).stop 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 shutdown_connections 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3cff77a0 0x555b3cff9b90 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfdf3c0 0x555b3cfdc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfdbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 --2- 172.21.15.138:0/2933703406 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfd2b10 0x555b3cfdeb90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 >> 172.21.15.138:0/2933703406 conn(0x555b3cfd0940 msgr2=0x555b3cfd6470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fc0c00640 0 client.17022 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.427+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 shutdown_connections 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.428+0000 7f8fe0685740 1 -- 172.21.15.138:0/2933703406 wait complete. 2024-08-20T01:21:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.428+0000 7f8fe0685740 20 client.17022 trim_cache size 0 max 16384 2024-08-20T01:21:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.449+0000 7f8fe0685740 1 Processor -- start 2024-08-20T01:21:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.449+0000 7f8fe0685740 1 -- start start 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfddd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfff410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81b0100 con 0x555b3cfdf3c0 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ad9a0 con 0x555b3cfd2b10 2024-08-20T01:21:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fe0685740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ab810 con 0x555b3cfd3430 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55522/0 (socket says 172.21.15.138:55522) 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 -- 172.21.15.138:0/1024383364 learned_addr learned my addr 172.21.15.138:0/1024383364 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd7400640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6000640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 -- 172.21.15.138:0/1024383364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 msgr2=0x555b3cfddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 -- 172.21.15.138:0/1024383364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 msgr2=0x555b3cfff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6a00640 1 -- 172.21.15.138:0/1024383364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b3cd13620 con 0x555b3cfd2b10 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.450+0000 7f8fd6000640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:21:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.451+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3877ea17af74b029 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.451+0000 7f8fc3400640 1 -- 172.21.15.138:0/1024383364 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80035f0 con 0x555b3cfd2b10 2024-08-20T01:21:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.451+0000 7f8fc3400640 1 -- 172.21.15.138:0/1024383364 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc804c070 con 0x555b3cfd2b10 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.451+0000 7f8fc3400640 1 -- 172.21.15.138:0/1024383364 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc80515b0 con 0x555b3cfd2b10 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- 172.21.15.138:0/1024383364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 msgr2=0x555b3cfdc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8000c00 tx=0x7f8fc8004b40 comp rx=0 tx=0).stop 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- 172.21.15.138:0/1024383364 shutdown_connections 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd3430 0x555b3cfddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 --2- 172.21.15.138:0/1024383364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd2b10 0x555b3cfdc4b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- 172.21.15.138:0/1024383364 >> 172.21.15.138:0/1024383364 conn(0x555b3cfd0940 msgr2=0x555b3cfdd600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- 172.21.15.138:0/1024383364 shutdown_connections 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- 172.21.15.138:0/1024383364 wait complete. 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 -- start start 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.452+0000 7f8fe0685740 1 Processor -- start 2024-08-20T01:21:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 0x555b3cff0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfeb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81abe60 con 0x555b3cfdf3c0 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ad2d0 con 0x555b3cfd3430 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8fd81ac530 con 0x555b3cfd2b10 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fe0685740 10 client.0 _ll_register_callbacks cb 0x7f8fc8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 0x555b3cff0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39284/0 (socket says 172.21.15.138:39284) 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 -- 172.21.15.138:0/489577942 learned_addr learned my addr 172.21.15.138:0/489577942 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 0x555b3cff0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:55526/0 (socket says 172.21.15.138:55526) 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd7400640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfeb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 -- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 msgr2=0x555b3cff0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 0x555b3cff0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 -- 172.21.15.138:0/489577942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 msgr2=0x555b3cfeb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfeb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd6a00640 1 -- 172.21.15.138:0/489577942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b3cfebee0 con 0x555b3cfd2b10 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.453+0000 7f8fd7400640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfeb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fd6a00640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fc802a430 tx=0x7f8fc8005200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72e7c204cf48628e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fc0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:21:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fc8003d90 con 0x555b3cfd2b10 2024-08-20T01:21:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fc804c070 con 0x555b3cfd2b10 2024-08-20T01:21:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f8fc80515b0 con 0x555b3cfd2b10 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8fc80517b0 con 0x555b3cfd2b10 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fe0685740 10 client.27028 fetch_fsmap learned FSMap version 43 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.454+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555b3d062100 con 0x555b3cfd2b10 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fc804c070 con 0x555b3cfd2b10 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 43) ==== 8362+0+0 (secure 0 0 0) 0x7f8fc8044040 con 0x555b3cfd2b10 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 10 client.27028 fetch_fsmap finished waiting for FSMap version 43 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 10 client.27028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 20 client.27028 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3d0625d0 0x555b3d0649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 4 client.27028 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:21:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b3cf45c00 con 0x555b3d0625d0 2024-08-20T01:21:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.455+0000 7f8fd6000640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3d0625d0 0x555b3d0649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.456+0000 7f8fc0c00640 10 client.27028 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.456+0000 7f8fd6000640 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3d0625d0 0x555b3d0649c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=183b8a9ff9264cc2 server_cookie=cf0155df855795ee in_seq=0 out_seq=0 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.503+0000 7f8fc0c00640 1 -- 172.21.15.138:0/489577942 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b3cf45c00 con 0x555b3d0625d0 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.503+0000 7f8fc0c00640 10 client.27028 handle_command_reply: tid=0 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.503+0000 7f8fe0685740 1 client.27028 shutdown 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 msgr2=0x555b3cfeff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fc802a430 tx=0x7f8fc8005200 comp rx=0 tx=0).stop 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 shutdown_connections 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x555b3d0625d0 0x555b3d0649c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x555b3cfdf3c0 0x555b3cfeb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555b3cfd3430 0x555b3cff0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 --2- 172.21.15.138:0/489577942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x555b3cfd2b10 0x555b3cfeff00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fc0c00640 0 client.27028 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 >> 172.21.15.138:0/489577942 conn(0x555b3cfd0940 msgr2=0x555b3cfdde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 shutdown_connections 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 1 -- 172.21.15.138:0/489577942 wait complete. 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.504+0000 7f8fe0685740 20 client.27028 trim_cache size 0 max 16384 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:21:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 msgr2=0x7f8fbc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 0x7f8fbc0b6200 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8fc8005ab0 tx=0x7f8fc8005a20 comp rx=0 tx=0).stop 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 msgr2=0x7f8fd8101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400ea60 tx=0x7f8fc400ef20 comp rx=0 tx=0).stop 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 shutdown_connections 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f8fbc0b3d40 0x7f8fbc0b6200 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8fd8105970 0x7f8fd8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd8104fd0 0x7f8fd8101f20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 --2- 172.21.15.138:0/4096418362 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8fd806a7a0 0x7f8fd81019e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 >> 172.21.15.138:0/4096418362 conn(0x7f8fd8077270 msgr2=0x7f8fd8108340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 shutdown_connections 2024-08-20T01:21:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:25.505+0000 7f8fd7e00640 1 -- 172.21.15.138:0/4096418362 wait complete. 2024-08-20T01:21:26.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 520365 3.61 MB/sec execute 578 sec latency 42.664 ms 2024-08-20T01:21:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:26 smithi167 ceph-mon[21965]: pgmap v776: 129 pgs: 129 active+clean; 448 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-08-20T01:21:26.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:26 smithi142 ceph-mon[21799]: pgmap v776: 129 pgs: 129 active+clean; 448 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-08-20T01:21:27.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:26 smithi138 ceph-mon[18076]: pgmap v776: 129 pgs: 129 active+clean; 448 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 173 op/s 2024-08-20T01:21:27.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 520365 3.60 MB/sec execute 579 sec latency 1002.139 ms 2024-08-20T01:21:28.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 520365 3.60 MB/sec execute 580 sec latency 2002.281 ms 2024-08-20T01:21:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:28 smithi167 ceph-mon[21965]: pgmap v777: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-08-20T01:21:28.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:28 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:21:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:28 smithi142 ceph-mon[21799]: pgmap v777: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-08-20T01:21:28.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:28 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:21:29.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:28 smithi138 ceph-mon[18076]: pgmap v777: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 114 op/s 2024-08-20T01:21:29.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:28 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:21:29.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 520365 3.59 MB/sec execute 581 sec latency 3002.385 ms 2024-08-20T01:21:30.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 522555 3.61 MB/sec execute 582 sec latency 3029.745 ms 2024-08-20T01:21:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:30 smithi167 ceph-mon[21965]: pgmap v778: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-20T01:21:30.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:30 smithi142 ceph-mon[21799]: pgmap v778: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-20T01:21:31.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:30 smithi138 ceph-mon[18076]: pgmap v778: 129 pgs: 129 active+clean; 450 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 91 op/s 2024-08-20T01:21:31.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 523709 3.61 MB/sec execute 583 sec latency 59.249 ms 2024-08-20T01:21:32.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 523906 3.60 MB/sec execute 584 sec latency 813.821 ms 2024-08-20T01:21:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:32 smithi167 ceph-mon[21965]: pgmap v779: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:21:32.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:32 smithi142 ceph-mon[21799]: pgmap v779: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:21:33.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:32 smithi138 ceph-mon[18076]: pgmap v779: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 126 op/s 2024-08-20T01:21:33.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 525840 3.62 MB/sec execute 585 sec latency 1161.772 ms 2024-08-20T01:21:34.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527015 3.62 MB/sec execute 586 sec latency 12.092 ms 2024-08-20T01:21:34.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:34 smithi167 ceph-mon[21965]: pgmap v780: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-08-20T01:21:34.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:34 smithi142 ceph-mon[21799]: pgmap v780: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-08-20T01:21:35.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:34 smithi138 ceph-mon[18076]: pgmap v780: 129 pgs: 129 active+clean; 454 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.1 MiB/s wr, 102 op/s 2024-08-20T01:21:35.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.61 MB/sec execute 587 sec latency 981.566 ms 2024-08-20T01:21:36.214 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.61 MB/sec execute 588 sec latency 1981.703 ms 2024-08-20T01:21:36.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:36 smithi167 ceph-mon[21965]: pgmap v781: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-08-20T01:21:36.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:36 smithi142 ceph-mon[21799]: pgmap v781: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-08-20T01:21:37.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:36 smithi138 ceph-mon[18076]: pgmap v781: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 143 op/s 2024-08-20T01:21:37.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.60 MB/sec execute 589 sec latency 2981.826 ms 2024-08-20T01:21:37.972 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.4 2024-08-20T01:21:37.972 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:21:38.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.59 MB/sec execute 590 sec latency 3981.877 ms 2024-08-20T01:21:38.492 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:21:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 -- 172.21.15.138:0/191921136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 msgr2=0x7f7100108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 --2- 172.21.15.138:0/191921136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f7100108640 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009900 tx=0x7f70ec02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 -- 172.21.15.138:0/191921136 shutdown_connections 2024-08-20T01:21:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 --2- 172.21.15.138:0/191921136 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7100108b80 0x7f710010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 --2- 172.21.15.138:0/191921136 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f7100108640 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 --2- 172.21.15.138:0/191921136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f7100102be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 -- 172.21.15.138:0/191921136 >> 172.21.15.138:0/191921136 conn(0x7f71000fe280 msgr2=0x7f71001006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 -- 172.21.15.138:0/191921136 shutdown_connections 2024-08-20T01:21:38.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.623+0000 7f70ffe00640 1 -- 172.21.15.138:0/191921136 wait complete. 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.624+0000 7f70ffe00640 1 Processor -- start 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.624+0000 7f70ffe00640 1 -- start start 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.624+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.624+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f710019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70ffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7100108b80 0x7f71001a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70ffe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7100112890 con 0x7f7100102800 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70ffe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7100112710 con 0x7f7100108b80 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70ffe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7100112a10 con 0x7f7100108200 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33242/0 (socket says 172.21.15.138:33242) 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 -- 172.21.15.138:0/2774776825 learned_addr learned my addr 172.21.15.138:0/2774776825 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fe000640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f710019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 -- 172.21.15.138:0/2774776825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 msgr2=0x7f710019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f710019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 -- 172.21.15.138:0/2774776825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7100108b80 msgr2=0x7f71001a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7100108b80 0x7f71001a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 -- 172.21.15.138:0/2774776825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71001a1010 con 0x7f7100102800 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fe000640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f710019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.625+0000 7f70fea00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.626+0000 7f70f7400640 1 -- 172.21.15.138:0/2774776825 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f000cd80 con 0x7f7100102800 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.626+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71001a1300 con 0x7f7100102800 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.626+0000 7f70f7400640 1 -- 172.21.15.138:0/2774776825 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f00108e0 con 0x7f7100102800 2024-08-20T01:21:38.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.626+0000 7f70f7400640 1 -- 172.21.15.138:0/2774776825 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f0018980 con 0x7f7100102800 2024-08-20T01:21:38.627 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.626+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71001a79c0 con 0x7f7100102800 2024-08-20T01:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.629+0000 7f70f7400640 1 -- 172.21.15.138:0/2774776825 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f70f0010430 con 0x7f7100102800 2024-08-20T01:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 Processor -- start 2024-08-20T01:21:38.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 -- start start 2024-08-20T01:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c794ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603c78b8210 con 0x5603c794f3c0 2024-08-20T01:21:38.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5603c78a7710 con 0x5603c7943430 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f7108402740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603c78ab750 con 0x5603c7942b10 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.630+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70ff400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c794ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70ff400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38428/0 (socket says 172.21.15.138:38428) 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33248/0 (socket says 172.21.15.138:33248) 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70ff400640 1 -- 172.21.15.138:0/1387703549 learned_addr learned my addr 172.21.15.138:0/1387703549 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70f7400640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 0x7f70e40b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 -- 172.21.15.138:0/1387703549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 msgr2=0x5603c7942ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 -- 172.21.15.138:0/1387703549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 msgr2=0x5603c794ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c794ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70f7400640 1 -- 172.21.15.138:0/2774776825 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f70f00e47c0 con 0x7f7100102800 2024-08-20T01:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 -- 172.21.15.138:0/1387703549 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c7873de0 con 0x5603c794f3c0 2024-08-20T01:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70ff400640 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70ff400640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 0x7f70e40b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.631+0000 7f70fe000640 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f70ec02a9e0 tx=0x7f70ec0043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402e86f25c822062 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.632+0000 7f70eb400640 1 -- 172.21.15.138:0/1387703549 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec035040 con 0x5603c794f3c0 2024-08-20T01:21:38.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.632+0000 7f70eb400640 1 -- 172.21.15.138:0/1387703549 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70ec004770 con 0x5603c794f3c0 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.632+0000 7f70eb400640 1 -- 172.21.15.138:0/1387703549 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec003920 con 0x5603c794f3c0 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- 172.21.15.138:0/1387703549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 msgr2=0x5603c79517b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f70ff400640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 0x7f70e40b6200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70f80049d0 tx=0x7f70f8004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f70ec02a9e0 tx=0x7f70ec0043f0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- 172.21.15.138:0/1387703549 shutdown_connections 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79517b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c794ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 --2- 172.21.15.138:0/1387703549 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7942ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- 172.21.15.138:0/1387703549 >> 172.21.15.138:0/1387703549 conn(0x5603c7940940 msgr2=0x5603c7941ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- 172.21.15.138:0/1387703549 shutdown_connections 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- 172.21.15.138:0/1387703549 wait complete. 2024-08-20T01:21:38.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.633+0000 7f7108402740 1 -- start start 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 Processor -- start 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c794eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c794c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5603c77e0ab0 con 0x5603c7943430 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5603c784e140 con 0x5603c7942b10 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5603c7952040 con 0x5603c794f3c0 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f7108402740 10 client.0 _ll_register_callbacks cb 0x5603c78615d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33250/0 (socket says 172.21.15.138:33250) 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f70fe000640 1 -- 172.21.15.138:0/4169720763 learned_addr learned my addr 172.21.15.138:0/4169720763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.634+0000 7f70ff400640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c794c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 -- 172.21.15.138:0/4169720763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 msgr2=0x5603c794c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:38.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c794c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 -- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 msgr2=0x5603c794eba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c794eba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 -- 172.21.15.138:0/4169720763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c784f170 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70fe000640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005ab0 tx=0x7f70ec005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab0ad401fba10b37 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec035040 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f70ec004b20 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.635+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70ec004770 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70ec05a050 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 10 client.17040 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603c760a790 con 0x5603c7943430 2024-08-20T01:21:38.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70ec004770 con 0x5603c7943430 2024-08-20T01:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f70ec055030 con 0x5603c7943430 2024-08-20T01:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 10 client.17040 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 10 client.17040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 20 client.17040 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:38.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c795ed80 0x5603c7961170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 4 client.17040 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:38.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.636+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603c78ba160 con 0x5603c795ed80 2024-08-20T01:21:38.654 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.655+0000 7f70fea00640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c795ed80 0x5603c7961170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.715+0000 7f70dfe00640 10 client.17040 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:38.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.715+0000 7f70fea00640 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c795ed80 0x5603c7961170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1593350a90e6dff server_cookie=9098e08a0719e5c1 in_seq=0 out_seq=0 2024-08-20T01:21:38.738 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.739+0000 7f70dfe00640 1 -- 172.21.15.138:0/4169720763 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603c78ba160 con 0x5603c795ed80 2024-08-20T01:21:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.739+0000 7f70dfe00640 10 client.17040 handle_command_reply: tid=0 2024-08-20T01:21:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.740+0000 7f7108402740 1 client.17040 shutdown 2024-08-20T01:21:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.740+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 msgr2=0x5603c794bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.740+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005ab0 tx=0x7f70ec005900 comp rx=0 tx=0).stop 2024-08-20T01:21:38.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 shutdown_connections 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c795ed80 0x5603c7961170 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c794c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c794bfe0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 --2- 172.21.15.138:0/4169720763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c794eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 >> 172.21.15.138:0/4169720763 conn(0x5603c7940940 msgr2=0x5603c79464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.741+0000 7f70dfe00640 0 client.17040 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.742+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 shutdown_connections 2024-08-20T01:21:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.742+0000 7f7108402740 1 -- 172.21.15.138:0/4169720763 wait complete. 2024-08-20T01:21:38.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.743+0000 7f7108402740 20 client.17040 trim_cache size 0 max 16384 2024-08-20T01:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.771+0000 7f7108402740 1 Processor -- start 2024-08-20T01:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.771+0000 7f7108402740 1 -- start start 2024-08-20T01:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c7966cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79491d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b43f0 con 0x5603c794f3c0 2024-08-20T01:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b1cc0 con 0x5603c7943430 2024-08-20T01:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f7108402740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71001afb30 con 0x5603c7942b10 2024-08-20T01:21:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f70fea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f70fea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38442/0 (socket says 172.21.15.138:38442) 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.772+0000 7f70fea00640 1 -- 172.21.15.138:0/3318060252 learned_addr learned my addr 172.21.15.138:0/3318060252 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70ff400640 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70fea00640 1 -- 172.21.15.138:0/3318060252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 msgr2=0x5603c7966cb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70fea00640 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c7966cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70fea00640 1 -- 172.21.15.138:0/3318060252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 msgr2=0x5603c79491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70fea00640 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.773+0000 7f70fea00640 1 -- 172.21.15.138:0/3318060252 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603c7683620 con 0x5603c7942b10 2024-08-20T01:21:38.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.774+0000 7f70fea00640 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70f00aaf60 tx=0x7f70f000cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f3ea306d5887932 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.774+0000 7f70eaa00640 1 -- 172.21.15.138:0/3318060252 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00aaae0 con 0x5603c7942b10 2024-08-20T01:21:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.774+0000 7f70eaa00640 1 -- 172.21.15.138:0/3318060252 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f00ab6d0 con 0x5603c7942b10 2024-08-20T01:21:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.774+0000 7f70eaa00640 1 -- 172.21.15.138:0/3318060252 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f00a73f0 con 0x5603c7942b10 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.775+0000 7f7108402740 1 -- 172.21.15.138:0/3318060252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 msgr2=0x5603c7966770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.775+0000 7f7108402740 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70f00aaf60 tx=0x7f70f000cf20 comp rx=0 tx=0).stop 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 -- 172.21.15.138:0/3318060252 shutdown_connections 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c794f3c0 0x5603c79491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7943430 0x5603c7966cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 --2- 172.21.15.138:0/3318060252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c7942b10 0x5603c7966770 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 -- 172.21.15.138:0/3318060252 >> 172.21.15.138:0/3318060252 conn(0x5603c7940940 msgr2=0x5603c794de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 -- 172.21.15.138:0/3318060252 shutdown_connections 2024-08-20T01:21:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 -- 172.21.15.138:0/3318060252 wait complete. 2024-08-20T01:21:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.776+0000 7f7108402740 1 -- start start 2024-08-20T01:21:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.777+0000 7f7108402740 1 Processor -- start 2024-08-20T01:21:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.778+0000 7f7108402740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c7954490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c79c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b0180 con 0x5603c7943430 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b15f0 con 0x5603c7942b10 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f71001b0850 con 0x5603c794f3c0 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f7108402740 10 client.0 _ll_register_callbacks cb 0x7f70f00a7620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f70fe000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33266/0 (socket says 172.21.15.138:33266) 2024-08-20T01:21:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.779+0000 7f70fe000640 1 -- 172.21.15.138:0/1567986409 learned_addr learned my addr 172.21.15.138:0/1567986409 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 -- 172.21.15.138:0/1567986409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 msgr2=0x5603c79c7740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c79c7740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 -- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 msgr2=0x5603c7954490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c7954490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 -- 172.21.15.138:0/1567986409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603c79c7c80 con 0x5603c7943430 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70fe000640 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005ec0 tx=0x7f70ec031940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a47e84b6730f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec035040 con 0x5603c7943430 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f70ec0035f0 con 0x5603c7943430 2024-08-20T01:21:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.780+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70f00ab6d0 con 0x5603c7943430 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f70ec005310 con 0x5603c7943430 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 10 client.17046 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5603c79c8e00 con 0x5603c7943430 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70f00ab6d0 con 0x5603c7943430 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f70ec055040 con 0x5603c7943430 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 10 client.17046 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 20 client.17046 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c79c94c0 0x5603c79cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 4 client.17046 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.781+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603c78b5c00 con 0x5603c79c94c0 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.782+0000 7f70fea00640 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c79c94c0 0x5603c79cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.782+0000 7f70fea00640 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c79c94c0 0x5603c79cb8e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c234d01529cb782 server_cookie=8ade102bc679490f in_seq=0 out_seq=0 2024-08-20T01:21:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.782+0000 7f70dfe00640 10 client.17046 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.826+0000 7f70dfe00640 1 -- 172.21.15.138:0/1567986409 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603c78b5c00 con 0x5603c79c94c0 2024-08-20T01:21:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.826+0000 7f70dfe00640 10 client.17046 handle_command_reply: tid=0 2024-08-20T01:21:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.826+0000 7f7108402740 1 client.17046 shutdown 2024-08-20T01:21:38.825 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.826+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 msgr2=0x5603c79549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.826+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f70ec005ec0 tx=0x7f70ec031940 comp rx=0 tx=0).stop 2024-08-20T01:21:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 shutdown_connections 2024-08-20T01:21:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5603c79c94c0 0x5603c79cb8e0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.826 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5603c794f3c0 0x5603c79c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5603c7943430 0x5603c79549d0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f70dfe00640 0 client.17046 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 --2- 172.21.15.138:0/1567986409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5603c7942b10 0x5603c7954490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 >> 172.21.15.138:0/1567986409 conn(0x5603c7940940 msgr2=0x5603c7945c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 shutdown_connections 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 1 -- 172.21.15.138:0/1567986409 wait complete. 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f7108402740 20 client.17046 trim_cache size 0 max 16384 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:21:38.827 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 msgr2=0x7f70e40b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 0x7f70e40b6200 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f70f80049d0 tx=0x7f70f8004a40 comp rx=0 tx=0).stop 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 msgr2=0x7f710019af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:38.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.827+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f70f000e9f0 tx=0x7f70f000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 shutdown_connections 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f70e40b3d40 0x7f70e40b6200 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7100108b80 0x7f71001a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7100108200 0x7f710019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 --2- 172.21.15.138:0/2774776825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7100102800 0x7f710019af70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 >> 172.21.15.138:0/2774776825 conn(0x7f71000fe280 msgr2=0x7f71001047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 shutdown_connections 2024-08-20T01:21:38.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:38.828+0000 7f70ffe00640 1 -- 172.21.15.138:0/2774776825 wait complete. 2024-08-20T01:21:38.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:38 smithi167 ceph-mon[21965]: pgmap v782: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-20T01:21:38.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:38 smithi142 ceph-mon[21799]: pgmap v782: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-20T01:21:39.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:38 smithi138 ceph-mon[18076]: pgmap v782: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.3 MiB/s wr, 115 op/s 2024-08-20T01:21:39.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.59 MB/sec execute 591 sec latency 4981.928 ms 2024-08-20T01:21:39.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:39 smithi167 ceph-mon[21965]: pgmap v783: 129 pgs: 129 active+clean; 464 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-20T01:21:39.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:39 smithi142 ceph-mon[21799]: pgmap v783: 129 pgs: 129 active+clean; 464 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-20T01:21:40.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:39 smithi138 ceph-mon[18076]: pgmap v783: 129 pgs: 129 active+clean; 464 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-08-20T01:21:40.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527026 3.58 MB/sec execute 592 sec latency 5981.969 ms 2024-08-20T01:21:41.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527188 3.58 MB/sec execute 593 sec latency 6944.173 ms 2024-08-20T01:21:41.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.428+0000 7f9547000640 1 -- 172.21.15.138:0/1133248892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108320 msgr2=0x7f9540108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:41.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.428+0000 7f9547000640 1 --2- 172.21.15.138:0/1133248892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108320 0x7f9540108760 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f952c009900 tx=0x7f952c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 -- 172.21.15.138:0/1133248892 shutdown_connections 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 --2- 172.21.15.138:0/1133248892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540108ca0 0x7f954010f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 --2- 172.21.15.138:0/1133248892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108320 0x7f9540108760 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 --2- 172.21.15.138:0/1133248892 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540102910 0x7f9540102cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 -- 172.21.15.138:0/1133248892 >> 172.21.15.138:0/1133248892 conn(0x7f95400fe3c0 msgr2=0x7f95401007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.429+0000 7f9547000640 1 -- 172.21.15.138:0/1133248892 shutdown_connections 2024-08-20T01:21:41.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.430+0000 7f9547000640 1 -- 172.21.15.138:0/1133248892 wait complete. 2024-08-20T01:21:41.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 Processor -- start 2024-08-20T01:21:41.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 -- start start 2024-08-20T01:21:41.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540102910 0x7f954019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:41.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 0x7f954019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f95401129b0 con 0x7f9540102910 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9540112830 con 0x7f9540108ca0 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.431+0000 7f9547000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9540112b30 con 0x7f9540108320 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9545c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540102910 0x7f954019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9545200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 0x7f954019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9545200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 0x7f954019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38486/0 (socket says 172.21.15.138:38486) 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9545200640 1 -- 172.21.15.138:0/2934271228 learned_addr learned my addr 172.21.15.138:0/2934271228 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:41.431 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38536/0 (socket says 172.21.15.138:38536) 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 -- 172.21.15.138:0/2934271228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 msgr2=0x7f954019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 0x7f954019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 -- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540102910 msgr2=0x7f954019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540102910 0x7f954019af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 -- 172.21.15.138:0/2934271228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95401a1020 con 0x7f9540108ca0 2024-08-20T01:21:41.432 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.432+0000 7f9546600640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f953800ec50 tx=0x7f953800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.433+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538003b70 con 0x7f9540108ca0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.433+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9538003d10 con 0x7f9540108ca0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.433+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95401a1310 con 0x7f9540108ca0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.433+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9538010830 con 0x7f9540108ca0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.433+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95401a79d0 con 0x7f9540108ca0 2024-08-20T01:21:41.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.434+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9540102cf0 con 0x7f9540108ca0 2024-08-20T01:21:41.435 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.436+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f95380109d0 con 0x7f9540108ca0 2024-08-20T01:21:41.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.438+0000 7f953e000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 0x7f95280b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:41.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.438+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f95380e48e0 con 0x7f9540108ca0 2024-08-20T01:21:41.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.438+0000 7f9545c00640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 0x7f95280b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:41.437 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.439+0000 7f9545c00640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 0x7f95280b6320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95340095f0 tx=0x7f9534005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:41.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.439+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f95380ab880 con 0x7f9540108ca0 2024-08-20T01:21:41.659 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.660+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f954019c0a0 con 0x7f9540108ca0 2024-08-20T01:21:41.660 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.661+0000 7f953e000640 1 -- 172.21.15.138:0/2934271228 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11933 (secure 0 0 0) 0x7f954019c0a0 con 0x7f9540108ca0 2024-08-20T01:21:41.661 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:21:41.661 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":44,"btime":"2024-08-20T01:21:27:555257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:21:27.554072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:21:41.661 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 44 2024-08-20T01:21:41.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 msgr2=0x7f95280b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:41.663 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 0x7f95280b6320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f95340095f0 tx=0x7f9534005f90 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 msgr2=0x7f95401a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f953800ec50 tx=0x7f953800c6a0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 shutdown_connections 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f95280b3e60 0x7f95280b6320 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9540108ca0 0x7f95401a0850 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9540108320 0x7f954019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 --2- 172.21.15.138:0/2934271228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9540102910 0x7f954019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.664+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 >> 172.21.15.138:0/2934271228 conn(0x7f95400fe3c0 msgr2=0x7f95401048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.665+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 shutdown_connections 2024-08-20T01:21:41.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:41.665+0000 7f9547000640 1 -- 172.21.15.138:0/2934271228 wait complete. 2024-08-20T01:21:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:41 smithi138 ceph-mon[18076]: pgmap v784: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-08-20T01:21:42.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:41 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2934271228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:42.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:41 smithi167 ceph-mon[21965]: pgmap v784: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-08-20T01:21:42.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:41 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2934271228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:42.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527456 3.58 MB/sec execute 594 sec latency 781.802 ms 2024-08-20T01:21:42.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:41 smithi142 ceph-mon[21799]: pgmap v784: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.5 MiB/s wr, 159 op/s 2024-08-20T01:21:42.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:41 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2934271228' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:21:42.509 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 141.4 --await-for 45.0' 2024-08-20T01:21:42.509 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 141.4 --await-for 45.0 2024-08-20T01:21:42.890 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:21:43.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527456 3.57 MB/sec execute 595 sec latency 1781.864 ms 2024-08-20T01:21:43.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 -- 172.21.15.138:0/455055323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810a6c0 msgr2=0x7eff7810ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 --2- 172.21.15.138:0/455055323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810a6c0 0x7eff7810ab00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7eff60009900 tx=0x7eff6002a410 comp rx=0 tx=0).stop 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 -- 172.21.15.138:0/455055323 shutdown_connections 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 --2- 172.21.15.138:0/455055323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810b040 0x7eff78111840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 --2- 172.21.15.138:0/455055323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810a6c0 0x7eff7810ab00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 --2- 172.21.15.138:0/455055323 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 0x7eff78105090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 -- 172.21.15.138:0/455055323 >> 172.21.15.138:0/455055323 conn(0x7eff78100720 msgr2=0x7eff78102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 -- 172.21.15.138:0/455055323 shutdown_connections 2024-08-20T01:21:43.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.562+0000 7eff7d600640 1 -- 172.21.15.138:0/455055323 wait complete. 2024-08-20T01:21:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.563+0000 7eff7d600640 1 Processor -- start 2024-08-20T01:21:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.563+0000 7eff7d600640 1 -- start start 2024-08-20T01:21:43.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 0x7eff7819b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114d50 con 0x7eff7810b040 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114bd0 con 0x7eff7810a6c0 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff7d600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff78114ed0 con 0x7eff78104cb0 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33312/0 (socket says 172.21.15.138:33312) 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38544/0 (socket says 172.21.15.138:38544) 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff77400640 1 -- 172.21.15.138:0/3094028449 learned_addr learned my addr 172.21.15.138:0/3094028449 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.564+0000 7eff76a00640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 0x7eff7819b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 -- 172.21.15.138:0/3094028449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 msgr2=0x7eff7819b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 0x7eff7819b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 -- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 msgr2=0x7eff781a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 -- 172.21.15.138:0/3094028449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff781a1210 con 0x7eff7810a6c0 2024-08-20T01:21:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff77400640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:43.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff76000640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff6002aa00 tx=0x7eff60003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff6f400640 1 -- 172.21.15.138:0/3094028449 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6003f070 con 0x7eff7810a6c0 2024-08-20T01:21:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff781a14a0 con 0x7eff7810a6c0 2024-08-20T01:21:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff6f400640 1 -- 172.21.15.138:0/3094028449 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff60031b50 con 0x7eff7810a6c0 2024-08-20T01:21:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff6f400640 1 -- 172.21.15.138:0/3094028449 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60035030 con 0x7eff7810a6c0 2024-08-20T01:21:43.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.565+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff781a7bc0 con 0x7eff7810a6c0 2024-08-20T01:21:43.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.568+0000 7eff8022b740 1 Processor -- start 2024-08-20T01:21:43.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 -- start start 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae47b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563fae41b670 con 0x563fae487ca0 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fae3ef7c0 con 0x563fae47bd10 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff8022b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fae3decc0 con 0x563fae47b3f0 2024-08-20T01:21:43.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff6f400640 1 -- 172.21.15.138:0/3094028449 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7eff60042070 con 0x7eff7810a6c0 2024-08-20T01:21:43.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff76a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff77400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae47b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff76a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38554/0 (socket says 172.21.15.138:38554) 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff76a00640 1 -- 172.21.15.138:0/1614220612 learned_addr learned my addr 172.21.15.138:0/1614220612 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff77400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae47b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:38522/0 (socket says 172.21.15.138:38522) 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.569+0000 7eff76000640 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 -- 172.21.15.138:0/1614220612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 msgr2=0x563fae47b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae47b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 -- 172.21.15.138:0/1614220612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 msgr2=0x563fae48a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 -- 172.21.15.138:0/1614220612 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fae372110 con 0x563fae47bd10 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.570+0000 7eff76a00640 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff680114f0 tx=0x7eff680119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840a1e49586e6f15 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff67400640 1 -- 172.21.15.138:0/1614220612 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6801e070 con 0x563fae47bd10 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff67400640 1 -- 172.21.15.138:0/1614220612 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6800fad0 con 0x563fae47bd10 2024-08-20T01:21:43.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff67400640 1 -- 172.21.15.138:0/1614220612 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68019ab0 con 0x563fae47bd10 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff6f400640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 0x7eff5c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff8022b740 1 -- 172.21.15.138:0/1614220612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 msgr2=0x563fae487760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff8022b740 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff680114f0 tx=0x7eff680119b0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff6f400640 1 -- 172.21.15.138:0/3094028449 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7eff60104530 con 0x7eff7810a6c0 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.571+0000 7eff77400640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 0x7eff5c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 -- 172.21.15.138:0/1614220612 shutdown_connections 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae487760 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 --2- 172.21.15.138:0/1614220612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae47b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 -- 172.21.15.138:0/1614220612 >> 172.21.15.138:0/1614220612 conn(0x563fae479220 msgr2=0x563fae47a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 -- 172.21.15.138:0/1614220612 shutdown_connections 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 -- 172.21.15.138:0/1614220612 wait complete. 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff77400640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 0x7eff5c0b6200 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff70009640 tx=0x7eff70005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.572+0000 7eff8022b740 1 -- start start 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 Processor -- start 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae484c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae487ca0 0x563fae4856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563fae384780 con 0x563fae47bd10 2024-08-20T01:21:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563fae3e2d00 con 0x563fae487ca0 2024-08-20T01:21:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563fae2417a0 con 0x563fae47b3f0 2024-08-20T01:21:43.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff8022b740 10 client.0 _ll_register_callbacks cb 0x563fae38c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff76000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff76000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33334/0 (socket says 172.21.15.138:33334) 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff76000640 1 -- 172.21.15.138:0/2914796206 learned_addr learned my addr 172.21.15.138:0/2914796206 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff77400640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae487ca0 0x563fae4856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.573+0000 7eff76a00640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae484c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 -- 172.21.15.138:0/2914796206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 msgr2=0x563fae484c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae484c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 -- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae487ca0 msgr2=0x563fae4856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae487ca0 0x563fae4856a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 -- 172.21.15.138:0/2914796206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fae3919c0 con 0x563fae47bd10 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff76000640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7eff60000c00 tx=0x7eff60107c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3191daf3f734dcad server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff5be00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:43.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60049030 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7eff60104130 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.574+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff6800fad0 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7eff600cac20 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 10 client.17064 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563fae14db90 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff6800fad0 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7eff600cb0b0 con 0x563fae47bd10 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 10 client.17064 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 10 client.17064 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 20 client.17064 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae499a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 4 client.17064 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.575+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fae3f1710 con 0x563fae497600 2024-08-20T01:21:43.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.576+0000 7eff76a00640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae499a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.576+0000 7eff76a00640 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae499a30 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f669198682750345 server_cookie=43a21cf981b15898 in_seq=0 out_seq=0 2024-08-20T01:21:43.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.576+0000 7eff5be00640 10 client.17064 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff5be00640 1 -- 172.21.15.138:0/2914796206 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fae3f1710 con 0x563fae497600 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff5be00640 10 client.17064 handle_command_reply: tid=0 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 client.17064 shutdown 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 msgr2=0x563fae485160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7eff60000c00 tx=0x7eff60107c70 comp rx=0 tx=0).stop 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 shutdown_connections 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae499a30 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff5be00640 0 client.17064 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae487ca0 0x563fae4856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae47bd10 0x563fae485160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 --2- 172.21.15.138:0/2914796206 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae484c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.577+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 >> 172.21.15.138:0/2914796206 conn(0x563fae479220 msgr2=0x563fae47ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.578+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 shutdown_connections 2024-08-20T01:21:43.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.578+0000 7eff8022b740 1 -- 172.21.15.138:0/2914796206 wait complete. 2024-08-20T01:21:43.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.578+0000 7eff8022b740 20 client.17064 trim_cache size 0 max 16384 2024-08-20T01:21:43.594 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.595+0000 7eff8022b740 1 Processor -- start 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.595+0000 7eff8022b740 1 -- start start 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae4834a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae49f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b4560 con 0x563fae487ca0 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b1e30 con 0x563fae47bd10 2024-08-20T01:21:43.595 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff8022b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff781afca0 con 0x563fae47b3f0 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae49f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae49f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33338/0 (socket says 172.21.15.138:33338) 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38566/0 (socket says 172.21.15.138:38566) 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae4834a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 -- 172.21.15.138:0/1532393647 learned_addr learned my addr 172.21.15.138:0/1532393647 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:43.596 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 -- 172.21.15.138:0/1532393647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 msgr2=0x563fae4834a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae4834a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 -- 172.21.15.138:0/1532393647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 msgr2=0x563fae49f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae49f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.596+0000 7eff76000640 1 -- 172.21.15.138:0/1532393647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fae47a950 con 0x563fae47bd10 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.597+0000 7eff76000640 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff600c6ed0 tx=0x7eff60005c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270040dc6744408b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.597+0000 7eff66a00640 1 -- 172.21.15.138:0/1532393647 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60049070 con 0x563fae47bd10 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.597+0000 7eff66a00640 1 -- 172.21.15.138:0/1532393647 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff600ca960 con 0x563fae47bd10 2024-08-20T01:21:43.597 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.597+0000 7eff66a00640 1 -- 172.21.15.138:0/1532393647 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600ca5c0 con 0x563fae47bd10 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- 172.21.15.138:0/1532393647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 msgr2=0x563fae49f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7eff600c6ed0 tx=0x7eff60005c90 comp rx=0 tx=0).stop 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- 172.21.15.138:0/1532393647 shutdown_connections 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae49f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae49f300 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 --2- 172.21.15.138:0/1532393647 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae4834a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.599 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- 172.21.15.138:0/1532393647 >> 172.21.15.138:0/1532393647 conn(0x563fae479220 msgr2=0x563fae486a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- 172.21.15.138:0/1532393647 shutdown_connections 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- 172.21.15.138:0/1532393647 wait complete. 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 -- start start 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.598+0000 7eff8022b740 1 Processor -- start 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae48bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b02f0 con 0x563fae487ca0 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b1760 con 0x563fae47bd10 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7eff781b09c0 con 0x563fae47b3f0 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff8022b740 10 client.0 _ll_register_callbacks cb 0x7eff60107d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff77400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.600 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff76000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38582/0 (socket says 172.21.15.138:38582) 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff76000640 1 -- 172.21.15.138:0/127557253 learned_addr learned my addr 172.21.15.138:0/127557253 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.599+0000 7eff76a00640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae48bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 -- 172.21.15.138:0/127557253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 msgr2=0x563fae48bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae48bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 -- 172.21.15.138:0/127557253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 msgr2=0x563fae48c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 -- 172.21.15.138:0/127557253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fae483b80 con 0x563fae47bd10 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff76000640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff6002a930 tx=0x7eff601019c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=772a70e52a0bd552 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff5be00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600ea060 con 0x563fae47bd10 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7eff600c7440 con 0x563fae47bd10 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff600ca960 con 0x563fae47bd10 2024-08-20T01:21:43.601 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7eff60049070 con 0x563fae47bd10 2024-08-20T01:21:43.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff8022b740 10 client.27101 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:43.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.600+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x563fae483b80 con 0x563fae47bd10 2024-08-20T01:21:43.602 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff600ca960 con 0x563fae47bd10 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7eff60104130 con 0x563fae47bd10 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 10 client.27101 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 10 client.27101 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 20 client.27101 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae50d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 4 client.27101 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.4, "await_for": 45.0}] 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 141.4, "await_for": 45.0}) -- 0x563fae3ed1b0 con 0x563fae497600 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.601+0000 7eff76a00640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae50d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.602+0000 7eff76a00640 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae50d180 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921a06b58cf7325f server_cookie=bac6ae00a87b9c11 in_seq=0 out_seq=0 2024-08-20T01:21:43.603 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.602+0000 7eff5be00640 10 client.27101 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.675+0000 7eff5be00640 1 -- 172.21.15.138:0/127557253 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563fae3ed1b0 con 0x563fae497600 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.675+0000 7eff5be00640 10 client.27101 handle_command_reply: tid=0 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.675+0000 7eff8022b740 1 client.27101 shutdown 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 msgr2=0x563fae48c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff6002a930 tx=0x7eff601019c0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 shutdown_connections 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563fae497600 0x563fae50d180 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.675 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563fae487ca0 0x563fae48c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563fae47bd10 0x563fae48c070 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 --2- 172.21.15.138:0/127557253 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563fae47b3f0 0x563fae48bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff5be00640 0 client.27101 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 >> 172.21.15.138:0/127557253 conn(0x563fae479220 msgr2=0x563fae486a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 shutdown_connections 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 1 -- 172.21.15.138:0/127557253 wait complete. 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.676+0000 7eff8022b740 20 client.27101 trim_cache size 0 max 16384 2024-08-20T01:21:43.676 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 44, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 50, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "6ebd2b70": { 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 50, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:21:43.677 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 141.4, 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:21:43.678 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 msgr2=0x7eff5c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 0x7eff5c0b6200 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff70009640 tx=0x7eff70005f90 comp rx=0 tx=0).stop 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 msgr2=0x7eff7819b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7eff6002aa00 tx=0x7eff60003b90 comp rx=0 tx=0).stop 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 shutdown_connections 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7eff5c0b3d40 0x7eff5c0b6200 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff7810b040 0x7eff781a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.679 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff7810a6c0 0x7eff7819b6a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 --2- 172.21.15.138:0/3094028449 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7eff78104cb0 0x7eff7819b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:43.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 >> 172.21.15.138:0/3094028449 conn(0x7eff78100720 msgr2=0x7eff78102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:43.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 shutdown_connections 2024-08-20T01:21:43.680 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:21:43.677+0000 7eff7d600640 1 -- 172.21.15.138:0/3094028449 wait complete. 2024-08-20T01:21:44.167 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:43 smithi138 ceph-mon[18076]: pgmap v785: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-08-20T01:21:44.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:43 smithi167 ceph-mon[21965]: pgmap v785: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-08-20T01:21:44.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527459 3.56 MB/sec execute 596 sec latency 2211.879 ms 2024-08-20T01:21:44.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:43 smithi142 ceph-mon[21799]: pgmap v785: 129 pgs: 129 active+clean; 463 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 141 op/s 2024-08-20T01:21:44.613 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6ebd2b70', quiesce took 6.6 seconds. Will release after: 14.799999999999999 2024-08-20T01:21:45.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527459 3.56 MB/sec execute 597 sec latency 1563.068 ms 2024-08-20T01:21:45.943 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:45 smithi138 ceph-mon[18076]: pgmap v786: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-08-20T01:21:46.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:45 smithi167 ceph-mon[21965]: pgmap v786: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-08-20T01:21:46.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527459 3.55 MB/sec execute 598 sec latency 2563.185 ms 2024-08-20T01:21:46.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:45 smithi142 ceph-mon[21799]: pgmap v786: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 147 op/s 2024-08-20T01:21:47.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 527459 3.55 MB/sec execute 599 sec latency 3563.253 ms 2024-08-20T01:21:48.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:47 smithi167 ceph-mon[21965]: pgmap v787: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.0 MiB/s wr, 107 op/s 2024-08-20T01:21:48.215 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 600 sec 2024-08-20T01:21:48.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:47 smithi142 ceph-mon[21799]: pgmap v787: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.0 MiB/s wr, 107 op/s 2024-08-20T01:21:48.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:47 smithi138 ceph-mon[18076]: pgmap v787: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 2.0 MiB/s wr, 107 op/s 2024-08-20T01:21:49.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 601 sec 2024-08-20T01:21:50.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:49 smithi167 ceph-mon[21965]: pgmap v788: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-08-20T01:21:50.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 602 sec 2024-08-20T01:21:50.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:49 smithi142 ceph-mon[21799]: pgmap v788: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-08-20T01:21:50.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:49 smithi138 ceph-mon[18076]: pgmap v788: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 2.0 MiB/s wr, 102 op/s 2024-08-20T01:21:51.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 603 sec 2024-08-20T01:21:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 -- 172.21.15.138:0/1022220248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25810f3c0 msgr2=0x7fe258115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 --2- 172.21.15.138:0/1022220248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25810f3c0 0x7fe258115bc0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe250009ca0 tx=0x7fe25002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 -- 172.21.15.138:0/1022220248 shutdown_connections 2024-08-20T01:21:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 --2- 172.21.15.138:0/1022220248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe25810f3c0 0x7fe258115bc0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 --2- 172.21.15.138:0/1022220248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810ea40 0x7fe25810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 --2- 172.21.15.138:0/1022220248 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe258109040 0x7fe258109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 -- 172.21.15.138:0/1022220248 >> 172.21.15.138:0/1022220248 conn(0x7fe258078670 msgr2=0x7fe25807aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 -- 172.21.15.138:0/1022220248 shutdown_connections 2024-08-20T01:21:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.952+0000 7fe25ec00640 1 -- 172.21.15.138:0/1022220248 wait complete. 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.953+0000 7fe25ec00640 1 Processor -- start 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.953+0000 7fe25ec00640 1 -- start start 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe25810ea40 0x7fe25819fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 0x7fe2581a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe258119040 con 0x7fe258109040 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe258118ec0 con 0x7fe25810f3c0 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581191c0 con 0x7fe25810ea40 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe257e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.968 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe257e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50330/0 (socket says 172.21.15.138:50330) 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe257e00640 1 -- 172.21.15.138:0/667212094 learned_addr learned my addr 172.21.15.138:0/667212094 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 0x7fe2581a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe257400640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe25810ea40 0x7fe25819fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 -- 172.21.15.138:0/667212094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe25810ea40 msgr2=0x7fe25819fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe25810ea40 0x7fe25819fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 -- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 msgr2=0x7fe25819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe25cc00640 1 -- 172.21.15.138:0/667212094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2581a55a0 con 0x7fe25810f3c0 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.954+0000 7fe257e00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:51.969 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe25cc00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 0x7fe2581a4dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe250005e30 tx=0x7fe2500317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe254c00640 1 -- 172.21.15.138:0/667212094 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe250003b70 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe254c00640 1 -- 172.21.15.138:0/667212094 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250003d10 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2581a5830 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe254c00640 1 -- 172.21.15.138:0/667212094 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25002f970 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.955+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2581abf50 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.958+0000 7fe261a54740 1 Processor -- start 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.958+0000 7fe254c00640 1 -- 172.21.15.138:0/667212094 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe25002fb10 con 0x7fe25810f3c0 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.958+0000 7fe261a54740 1 -- start start 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bc5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bd1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.970 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bd45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561a11b3b210 con 0x561a11bc5930 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a11b2a710 con 0x561a11bd21e0 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe261a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a11b2e750 con 0x561a11bc6250 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bc5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bc5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50336/0 (socket says 172.21.15.138:50336) 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257400640 1 -- 172.21.15.138:0/2191134152 learned_addr learned my addr 172.21.15.138:0/2191134152 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bd1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257400640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bd45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 -- 172.21.15.138:0/2191134152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 msgr2=0x561a11bd45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bd45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 -- 172.21.15.138:0/2191134152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 msgr2=0x561a11bc5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bc5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257e00640 1 -- 172.21.15.138:0/2191134152 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a11af6de0 con 0x561a11bc6250 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.959+0000 7fe257400640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bd45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe257e00640 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bd1ca0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe248011510 tx=0x7fe2480119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55594eef5f6c567 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe254c00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 0x7fe23c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe24d200640 1 -- 172.21.15.138:0/2191134152 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248005020 con 0x561a11bc6250 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe24d200640 1 -- 172.21.15.138:0/2191134152 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2480047a0 con 0x561a11bc6250 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe24d200640 1 -- 172.21.15.138:0/2191134152 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe24800f8e0 con 0x561a11bc6250 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe254c00640 1 -- 172.21.15.138:0/667212094 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7fe250105520 con 0x7fe25810f3c0 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.960+0000 7fe257400640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 0x7fe23c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- 172.21.15.138:0/2191134152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 msgr2=0x561a11bd1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bd1ca0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe248011510 tx=0x7fe2480119d0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- 172.21.15.138:0/2191134152 shutdown_connections 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bd45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bd1ca0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 --2- 172.21.15.138:0/2191134152 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bc5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- 172.21.15.138:0/2191134152 >> 172.21.15.138:0/2191134152 conn(0x561a11bc3760 msgr2=0x561a11bc4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe257400640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 0x7fe23c0b6200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe244009fd0 tx=0x7fe244015040 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- 172.21.15.138:0/2191134152 shutdown_connections 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- 172.21.15.138:0/2191134152 wait complete. 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.961+0000 7fe261a54740 1 -- start start 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 Processor -- start 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561a11a63ab0 con 0x561a11bc5930 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561a11ad1140 con 0x561a11bd21e0 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561a11bd4ec0 con 0x561a11bc6250 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe261a54740 10 client.0 _ll_register_callbacks cb 0x561a11ae45d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe257e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.962+0000 7fe257e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50352/0 (socket says 172.21.15.138:50352) 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 -- 172.21.15.138:0/2874294531 learned_addr learned my addr 172.21.15.138:0/2874294531 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257400640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe25cc00640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 -- 172.21.15.138:0/2874294531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 msgr2=0x561a11bcee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bcee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 -- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 msgr2=0x561a11bcf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bcf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 -- 172.21.15.138:0/2874294531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a11ad2170 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe257e00640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe2480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28224bdaa6bb6920 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe241600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.963+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe248027070 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2480047a0 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7fe2480050d0 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe248033050 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 10 client.17082 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561a1188d790 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2480047a0 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7fe2480035f0 con 0x561a11bc5930 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 10 client.17082 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:51.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 10 client.17082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 20 client.17082 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11be1d30 0x561a11be4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 4 client.17082 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.964+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a11b3d160 con 0x561a11be1d30 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.965+0000 7fe257400640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11be1d30 0x561a11be4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.965+0000 7fe257400640 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11be1d30 0x561a11be4120 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6349c0cf711f73 server_cookie=990c068de4d6f794 in_seq=0 out_seq=0 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.965+0000 7fe241600640 10 client.17082 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe241600640 1 -- 172.21.15.138:0/2874294531 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a11b3d160 con 0x561a11be1d30 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe241600640 10 client.17082 handle_command_reply: tid=0 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 client.17082 shutdown 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 msgr2=0x561a11bd19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe248000c00 tx=0x7fe2480042f0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 shutdown_connections 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11be1d30 0x561a11be4120 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bd21e0 0x561a11bcf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bcee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe241600640 0 client.17082 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 --2- 172.21.15.138:0/2874294531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bc5930 0x561a11bd19d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 >> 172.21.15.138:0/2874294531 conn(0x561a11bc3760 msgr2=0x561a11bc9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:51.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 shutdown_connections 2024-08-20T01:21:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.966+0000 7fe261a54740 1 -- 172.21.15.138:0/2874294531 wait complete. 2024-08-20T01:21:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.967+0000 7fe261a54740 20 client.17082 trim_cache size 0 max 16384 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.981+0000 7fe261a54740 1 Processor -- start 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 -- start start 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 0x561a11bd15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b8a10 con 0x561a11bd21e0 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b62e0 con 0x561a11bc5930 2024-08-20T01:21:51.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe261a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b4150 con 0x561a11bc6250 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe25cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe257e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 0x561a11bd15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe25cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50362/0 (socket says 172.21.15.138:50362) 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe25cc00640 1 -- 172.21.15.138:0/4011538930 learned_addr learned my addr 172.21.15.138:0/4011538930 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.982+0000 7fe257400640 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 -- 172.21.15.138:0/4011538930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 msgr2=0x561a11bce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 -- 172.21.15.138:0/4011538930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 msgr2=0x561a11bd15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 0x561a11bd15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 -- 172.21.15.138:0/4011538930 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a11bd49f0 con 0x561a11bd21e0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe257e00640 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 0x561a11bd15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe25cc00640 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe250102a90 tx=0x7fe2500095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ebe8d9365c0b65 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe243e00640 1 -- 172.21.15.138:0/4011538930 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500e6070 con 0x561a11bd21e0 2024-08-20T01:21:51.984 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe243e00640 1 -- 172.21.15.138:0/4011538930 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250036040 con 0x561a11bd21e0 2024-08-20T01:21:51.985 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.983+0000 7fe243e00640 1 -- 172.21.15.138:0/4011538930 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500d4f50 con 0x561a11bd21e0 2024-08-20T01:21:51.987 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- 172.21.15.138:0/4011538930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 msgr2=0x561a11bce840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fe250102a90 tx=0x7fe2500095c0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- 172.21.15.138:0/4011538930 shutdown_connections 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bce840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc6250 0x561a11bce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 --2- 172.21.15.138:0/4011538930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc5930 0x561a11bd15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- 172.21.15.138:0/4011538930 >> 172.21.15.138:0/4011538930 conn(0x561a11bc3760 msgr2=0x561a11bca400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- 172.21.15.138:0/4011538930 shutdown_connections 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- 172.21.15.138:0/4011538930 wait complete. 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 -- start start 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.984+0000 7fe261a54740 1 Processor -- start 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc5930 0x561a11bda4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc6250 0x561a11bdaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b47a0 con 0x561a11bd21e0 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b5c10 con 0x561a11bc6250 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe2581b4e70 con 0x561a11bc5930 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe261a54740 10 client.0 _ll_register_callbacks cb 0x7fe244009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50368/0 (socket says 172.21.15.138:50368) 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 -- 172.21.15.138:0/787029158 learned_addr learned my addr 172.21.15.138:0/787029158 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe257e00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc5930 0x561a11bda4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe257400640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc6250 0x561a11bdaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 -- 172.21.15.138:0/787029158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc5930 msgr2=0x561a11bda4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc5930 0x561a11bda4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 -- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc6250 msgr2=0x561a11bdaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:21:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc6250 0x561a11bdaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.985+0000 7fe25cc00640 1 -- 172.21.15.138:0/787029158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a11906620 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe25cc00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c9110 tx=0x7fe250102d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df89ad47117b22f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2500e6040 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7fe2500c8490 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe250036040 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fe2500cb9b0 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 10 client.17088 fetch_fsmap learned FSMap version 44 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x561a11c4c440 con 0x561a11bd21e0 2024-08-20T01:21:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe250036040 con 0x561a11bd21e0 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7fe250091080 con 0x561a11bd21e0 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 10 client.17088 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 20 client.17088 populate_metadata read hostname 'smithi138' 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11c4cab0 0x561a11c4eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 4 client.17088 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.986+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a11b38c00 con 0x561a11c4cab0 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.987+0000 7fe257e00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11c4cab0 0x561a11c4eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.987+0000 7fe257e00640 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11c4cab0 0x561a11c4eed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89289516ea0835e2 server_cookie=4a30ade35b0f00f6 in_seq=0 out_seq=0 2024-08-20T01:21:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:51.987+0000 7fe241600640 10 client.17088 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe241600640 1 -- 172.21.15.138:0/787029158 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a11b38c00 con 0x561a11c4cab0 2024-08-20T01:21:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe241600640 10 client.17088 handle_command_reply: tid=0 2024-08-20T01:21:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 client.17088 shutdown 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 msgr2=0x561a11bd5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe2500c9110 tx=0x7fe250102d80 comp rx=0 tx=0).stop 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 shutdown_connections 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561a11c4cab0 0x561a11c4eed0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561a11bd21e0 0x561a11bd5c20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a11bc6250 0x561a11bdaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe241600640 0 client.17088 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 --2- 172.21.15.138:0/787029158 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561a11bc5930 0x561a11bda4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 >> 172.21.15.138:0/787029158 conn(0x561a11bc3760 msgr2=0x561a11bc1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 shutdown_connections 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.007+0000 7fe261a54740 1 -- 172.21.15.138:0/787029158 wait complete. 2024-08-20T01:21:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe261a54740 20 client.17088 trim_cache size 0 max 16384 2024-08-20T01:21:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:21:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:21:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:21:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:21:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 msgr2=0x7fe23c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 0x7fe23c0b6200 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe244009fd0 tx=0x7fe244015040 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 msgr2=0x7fe2581a4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 0x7fe2581a4dd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe250005e30 tx=0x7fe2500317c0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 shutdown_connections 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe23c0b3d40 0x7fe23c0b6200 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe25810f3c0 0x7fe2581a4dd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe25810ea40 0x7fe25819fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 --2- 172.21.15.138:0/667212094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe258109040 0x7fe25819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 >> 172.21.15.138:0/667212094 conn(0x7fe258078670 msgr2=0x7fe25807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.008+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 shutdown_connections 2024-08-20T01:21:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:21:52.009+0000 7fe25ec00640 1 -- 172.21.15.138:0/667212094 wait complete. 2024-08-20T01:21:52.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:51 smithi167 ceph-mon[21965]: pgmap v789: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-08-20T01:21:52.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 604 sec 2024-08-20T01:21:52.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:51 smithi142 ceph-mon[21799]: pgmap v789: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-08-20T01:21:52.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:51 smithi138 ceph-mon[18076]: pgmap v789: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.8 MiB/s wr, 94 op/s 2024-08-20T01:21:53.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 605 sec 2024-08-20T01:21:54.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:53 smithi167 ceph-mon[21965]: pgmap v790: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 579 KiB/s wr, 39 op/s 2024-08-20T01:21:54.216 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 606 sec 2024-08-20T01:21:54.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:53 smithi142 ceph-mon[21799]: pgmap v790: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 579 KiB/s wr, 39 op/s 2024-08-20T01:21:54.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:53 smithi138 ceph-mon[18076]: pgmap v790: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 380 KiB/s rd, 579 KiB/s wr, 39 op/s 2024-08-20T01:21:55.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 607 sec 2024-08-20T01:21:56.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:55 smithi167 ceph-mon[21965]: pgmap v791: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-08-20T01:21:56.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 608 sec 2024-08-20T01:21:56.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:55 smithi142 ceph-mon[21799]: pgmap v791: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-08-20T01:21:56.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:55 smithi138 ceph-mon[18076]: pgmap v791: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 101 KiB/s wr, 24 op/s 2024-08-20T01:21:57.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 609 sec 2024-08-20T01:21:58.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:57 smithi167 ceph-mon[21965]: pgmap v792: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-08-20T01:21:58.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:57 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:21:58.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 610 sec 2024-08-20T01:21:58.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:57 smithi142 ceph-mon[21799]: pgmap v792: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-08-20T01:21:58.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:57 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:21:58.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:57 smithi138 ceph-mon[18076]: pgmap v792: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 66 KiB/s wr, 19 op/s 2024-08-20T01:21:58.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:57 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:21:59.213 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:21:59.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:21:59.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:21:58 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:21:59.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 611 sec 2024-08-20T01:21:59.241 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:21:59.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:21:59.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:21:58 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:21:59.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:21:59.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:21:59.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:21:58 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:21:59.414 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:21:59.786 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:22:00.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 612 sec 2024-08-20T01:22:00.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:00 smithi138 ceph-mon[18076]: pgmap v793: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-08-20T01:22:00.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:00 smithi167 ceph-mon[21965]: pgmap v793: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-08-20T01:22:00.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:00 smithi142 ceph-mon[21799]: pgmap v793: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 -- 172.21.15.138:0/3369720766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc104ca0 msgr2=0x7f6fcc105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 --2- 172.21.15.138:0/3369720766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc105080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009900 tx=0x7f6fbc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 -- 172.21.15.138:0/3369720766 shutdown_connections 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 --2- 172.21.15.138:0/3369720766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 --2- 172.21.15.138:0/3369720766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 --2- 172.21.15.138:0/3369720766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc105080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 -- 172.21.15.138:0/3369720766 >> 172.21.15.138:0/3369720766 conn(0x7f6fcc100730 msgr2=0x7f6fcc102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 -- 172.21.15.138:0/3369720766 shutdown_connections 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.790+0000 7f6fd3600640 1 -- 172.21.15.138:0/3369720766 wait complete. 2024-08-20T01:22:00.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 Processor -- start 2024-08-20T01:22:00.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 -- start start 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc19b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc1a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc114cf0 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc114b70 con 0x7f6fcc10b030 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.791+0000 7f6fd3600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc114e70 con 0x7f6fcc104ca0 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fd1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc1a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc19b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fd0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc19b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43868/0 (socket says 172.21.15.138:43868) 2024-08-20T01:22:00.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fd1600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc1a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43916/0 (socket says 172.21.15.138:43916) 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fd0c00640 1 -- 172.21.15.138:0/1169671960 learned_addr learned my addr 172.21.15.138:0/1169671960 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 -- 172.21.15.138:0/1169671960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 msgr2=0x7f6fcc19b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc19b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 -- 172.21.15.138:0/1169671960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 msgr2=0x7f6fcc1a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc1a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 -- 172.21.15.138:0/1169671960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc1a1300 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.792+0000 7f6fc3e00640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc19b790 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9f0 tx=0x7f6fb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.793+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb400ccc0 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.793+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fb4005020 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.793+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4004a00 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.793+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc1a15f0 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.793+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1a7cb0 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.794+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc105080 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.796+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f6fb4004ba0 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.797+0000 7f6fca000640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 0x7f6fb00b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.798+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f6fb40e4370 con 0x7f6fcc10a6b0 2024-08-20T01:22:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.798+0000 7f6fd0c00640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 0x7f6fb00b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:00.798 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.799+0000 7f6fd0c00640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 0x7f6fb00b6280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc031440 tx=0x7f6fbc0058d0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:00.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:00.801+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f6fb40ab280 con 0x7f6fcc10a6b0 2024-08-20T01:22:01.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.049+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fcc10ce70 con 0x7f6fcc10a6b0 2024-08-20T01:22:01.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.050+0000 7f6fca000640 1 -- 172.21.15.138:0/1169671960 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) ==== 94+0+11933 (secure 0 0 0) 0x7f6fb40aab80 con 0x7f6fcc10a6b0 2024-08-20T01:22:01.051 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:22:01.051 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":44,"btime":"2024-08-20T01:21:27:555257+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":44,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:21:27.554072+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:22:01.051 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 44 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 msgr2=0x7f6fb00b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 0x7f6fb00b6280 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc031440 tx=0x7f6fbc0058d0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 msgr2=0x7f6fcc19b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc19b790 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400e9f0 tx=0x7f6fb400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 shutdown_connections 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f6fb00b3dc0 0x7f6fb00b6280 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6fcc10b030 0x7f6fcc1a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fcc10a6b0 0x7f6fcc19b790 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 --2- 172.21.15.138:0/1169671960 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6fcc104ca0 0x7f6fcc19b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:01.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 >> 172.21.15.138:0/1169671960 conn(0x7f6fcc100730 msgr2=0x7f6fcc102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:01.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 shutdown_connections 2024-08-20T01:22:01.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:01.053+0000 7f6fd3600640 1 -- 172.21.15.138:0/1169671960 wait complete. 2024-08-20T01:22:01.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 613 sec 2024-08-20T01:22:01.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:01 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1169671960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:01.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:01 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1169671960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:01.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:01 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1169671960' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:01.937 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id 6ebd2b70 --release --await' 2024-08-20T01:22:01.937 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id 6ebd2b70 --release --await 2024-08-20T01:22:02.217 INFO:tasks.workunit.client.0.smithi138.stdout: 1 cleanup 614 sec 2024-08-20T01:22:02.315 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:22:02.645 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:02 smithi138 ceph-mon[18076]: pgmap v794: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-08-20T01:22:02.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:02 smithi167 ceph-mon[21965]: pgmap v794: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-08-20T01:22:02.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:02 smithi142 ceph-mon[21799]: pgmap v794: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 14 op/s 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 -- 172.21.15.138:0/526437156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 msgr2=0x7f72c8111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 --2- 172.21.15.138:0/526437156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c8111830 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009ca0 tx=0x7f72c002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 -- 172.21.15.138:0/526437156 shutdown_connections 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 --2- 172.21.15.138:0/526437156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c8111830 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 --2- 172.21.15.138:0/526437156 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c810a6b0 0x7f72c810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 --2- 172.21.15.138:0/526437156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c8104ca0 0x7f72c8105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 -- 172.21.15.138:0/526437156 >> 172.21.15.138:0/526437156 conn(0x7f72c81006f0 msgr2=0x7f72c8102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 -- 172.21.15.138:0/526437156 shutdown_connections 2024-08-20T01:22:02.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.922+0000 7f72cd200640 1 -- 172.21.15.138:0/526437156 wait complete. 2024-08-20T01:22:02.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.923+0000 7f72cd200640 1 Processor -- start 2024-08-20T01:22:02.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.923+0000 7f72cd200640 1 -- start start 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c8104ca0 0x7f72c819b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 0x7f72c819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8114d00 con 0x7f72c810b030 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8114b80 con 0x7f72c810a6b0 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72cd200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72c8114e80 con 0x7f72c8104ca0 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c6800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40244/0 (socket says 172.21.15.138:40244) 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c6800640 1 -- 172.21.15.138:0/3714808451 learned_addr learned my addr 172.21.15.138:0/3714808451 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72bd400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 0x7f72c819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c5e00640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c8104ca0 0x7f72c819b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c6800640 1 -- 172.21.15.138:0/3714808451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c8104ca0 msgr2=0x7f72c819b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.924+0000 7f72c6800640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c8104ca0 0x7f72c819b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.925 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72c6800640 1 -- 172.21.15.138:0/3714808451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 msgr2=0x7f72c819b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72c6800640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 0x7f72c819b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72c6800640 1 -- 172.21.15.138:0/3714808451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c81a1200 con 0x7f72c810b030 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72bd400640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 0x7f72c819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72c6800640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f72c002f540 tx=0x7f72c00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72bf400640 1 -- 172.21.15.138:0/3714808451 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003b70 con 0x7f72c810b030 2024-08-20T01:22:02.926 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c81a1490 con 0x7f72c810b030 2024-08-20T01:22:02.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72bf400640 1 -- 172.21.15.138:0/3714808451 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72c0003d10 con 0x7f72c810b030 2024-08-20T01:22:02.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72bf400640 1 -- 172.21.15.138:0/3714808451 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c002f990 con 0x7f72c810b030 2024-08-20T01:22:02.927 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.925+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c81a7bb0 con 0x7f72c810b030 2024-08-20T01:22:02.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.928+0000 7f72cfc23740 1 Processor -- start 2024-08-20T01:22:02.928 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.928+0000 7f72bf400640 1 -- 172.21.15.138:0/3714808451 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f72c002fb30 con 0x7f72c810b030 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 -- start start 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x56245299f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56245299f5a0 0x5624529a1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562452934670 con 0x562452992cf0 2024-08-20T01:22:02.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562452908d70 con 0x56245299f5a0 2024-08-20T01:22:02.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72cfc23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5624528f8270 con 0x562452993610 2024-08-20T01:22:02.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40258/0 (socket says 172.21.15.138:40258) 2024-08-20T01:22:02.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72bd400640 1 -- 172.21.15.138:0/329056471 learned_addr learned my addr 172.21.15.138:0/329056471 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:02.930 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.929+0000 7f72c5e00640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x56245299f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72bd400640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56245299f5a0 0x5624529a1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 -- 172.21.15.138:0/329056471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56245299f5a0 msgr2=0x5624529a1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56245299f5a0 0x5624529a1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 -- 172.21.15.138:0/329056471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 msgr2=0x5624529930d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.931 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 -- 172.21.15.138:0/329056471 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562452894870 con 0x562452993610 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72bd400640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72c5e00640 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x56245299f060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009c20 tx=0x7f72b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17537090f0b9167e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72bf400640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 0x7f72ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.930+0000 7f72b7400640 1 -- 172.21.15.138:0/329056471 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8009dd0 con 0x562452993610 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72bf400640 1 -- 172.21.15.138:0/3714808451 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f72c0105880 con 0x7f72c810b030 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72bd400640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 0x7f72ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72b7400640 1 -- 172.21.15.138:0/329056471 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8005020 con 0x562452993610 2024-08-20T01:22:02.932 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72b7400640 1 -- 172.21.15.138:0/329056471 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8004b20 con 0x562452993610 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72cfc23740 1 -- 172.21.15.138:0/329056471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 msgr2=0x56245299f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72bd400640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 0x7f72ac0b6200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009710 tx=0x7f72b0009c00 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.931+0000 7f72cfc23740 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x56245299f060 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009c20 tx=0x7f72b8011810 comp rx=0 tx=0).stop 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 -- 172.21.15.138:0/329056471 shutdown_connections 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56245299f5a0 0x5624529a1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x56245299f060 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.933 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 --2- 172.21.15.138:0/329056471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452992cf0 0x5624529930d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 -- 172.21.15.138:0/329056471 >> 172.21.15.138:0/329056471 conn(0x562452990b20 msgr2=0x562452991cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 -- 172.21.15.138:0/329056471 shutdown_connections 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 -- 172.21.15.138:0/329056471 wait complete. 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 -- start start 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.932+0000 7f72cfc23740 1 Processor -- start 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x56245299ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56245299f5a0 0x56245299c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56245287eff0 con 0x562452993610 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5624528fc2b0 con 0x562452992cf0 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56245275abb0 con 0x56245299f5a0 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72cfc23740 10 client.0 _ll_register_callbacks cb 0x562452603d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.934 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72bd400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40268/0 (socket says 172.21.15.138:40268) 2024-08-20T01:22:02.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72bd400640 1 -- 172.21.15.138:0/551223998 learned_addr learned my addr 172.21.15.138:0/551223998 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:02.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x56245299ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.935 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.933+0000 7f72c6800640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56245299f5a0 0x56245299c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 -- 172.21.15.138:0/551223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56245299f5a0 msgr2=0x56245299c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56245299f5a0 0x56245299c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 -- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 msgr2=0x56245299ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x56245299ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 -- 172.21.15.138:0/551223998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562452657b00 con 0x562452993610 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72bd400640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72b0010f40 tx=0x7f72b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fc6c2cae53aee9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.936 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72abe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b0003d70 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8005020 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.934+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f72b0005020 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f72b0005220 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 10 client.17106 fetch_fsmap learned FSMap version 44 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x5624526ce960 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8005020 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f72b0023420 con 0x562452993610 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 10 client.17106 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 10 client.17106 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 20 client.17106 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5624529aec20 0x5624529b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 4 client.17106 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.935+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56245290acc0 con 0x5624529aec20 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.936+0000 7f72c5e00640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5624529aec20 0x5624529b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.937 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.936+0000 7f72c5e00640 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5624529aec20 0x5624529b1070 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76bc1a9513fbb01b server_cookie=af3711b853ff49c0 in_seq=0 out_seq=0 2024-08-20T01:22:02.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.936+0000 7f72abe00640 10 client.17106 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:02.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72abe00640 1 -- 172.21.15.138:0/551223998 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56245290acc0 con 0x5624529aec20 2024-08-20T01:22:02.938 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72abe00640 10 client.17106 handle_command_reply: tid=0 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 client.17106 shutdown 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 msgr2=0x56245299c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f72b0010f40 tx=0x7f72b0004730 comp rx=0 tx=0).stop 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 shutdown_connections 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5624529aec20 0x5624529b1070 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56245299f5a0 0x56245299c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452993610 0x56245299c230 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 --2- 172.21.15.138:0/551223998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x56245299ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72abe00640 0 client.17106 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 >> 172.21.15.138:0/551223998 conn(0x562452990b20 msgr2=0x562452996690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.937+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 shutdown_connections 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.938+0000 7f72cfc23740 1 -- 172.21.15.138:0/551223998 wait complete. 2024-08-20T01:22:02.939 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.938+0000 7f72cfc23740 20 client.17106 trim_cache size 0 max 16384 2024-08-20T01:22:02.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 Processor -- start 2024-08-20T01:22:02.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 -- start start 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x5624529ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x5624529b6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b4640 con 0x562452990b20 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b1f10 con 0x562452992cf0 2024-08-20T01:22:02.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.955+0000 7f72cfc23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81afd80 con 0x562452993610 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40274/0 (socket says 172.21.15.138:40274) 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 -- 172.21.15.138:0/1787560538 learned_addr learned my addr 172.21.15.138:0/1787560538 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72bd400640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x5624529ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c6800640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x5624529b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 -- 172.21.15.138:0/1787560538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 msgr2=0x5624529b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x5624529b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 -- 172.21.15.138:0/1787560538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 msgr2=0x5624529ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x5624529ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 -- 172.21.15.138:0/1787560538 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562452990950 con 0x562452990b20 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72bd400640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x5624529ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:02.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.956+0000 7f72c5e00640 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009c20 tx=0x7f72b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d264aecc56ab820 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72b6a00640 1 -- 172.21.15.138:0/1787560538 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8030070 con 0x562452990b20 2024-08-20T01:22:02.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72b6a00640 1 -- 172.21.15.138:0/1787560538 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8005590 con 0x562452990b20 2024-08-20T01:22:02.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72b6a00640 1 -- 172.21.15.138:0/1787560538 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b802b5c0 con 0x562452990b20 2024-08-20T01:22:02.958 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 -- 172.21.15.138:0/1787560538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 msgr2=0x56245299ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009c20 tx=0x7f72b8011810 comp rx=0 tx=0).stop 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 -- 172.21.15.138:0/1787560538 shutdown_connections 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452993610 0x5624529b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452992cf0 0x5624529ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 --2- 172.21.15.138:0/1787560538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x56245299ded0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.957+0000 7f72cfc23740 1 -- 172.21.15.138:0/1787560538 >> 172.21.15.138:0/1787560538 conn(0x562452997bd0 msgr2=0x56245299b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.958+0000 7f72cfc23740 1 -- 172.21.15.138:0/1787560538 shutdown_connections 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.958+0000 7f72cfc23740 1 -- 172.21.15.138:0/1787560538 wait complete. 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.958+0000 7f72cfc23740 1 -- start start 2024-08-20T01:22:02.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.958+0000 7f72cfc23740 1 Processor -- start 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452992cf0 0x5624529a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 0x562452a18910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b03d0 con 0x562452990b20 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b1840 con 0x562452993610 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f72c81b0aa0 con 0x562452992cf0 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72cfc23740 10 client.0 _ll_register_callbacks cb 0x7f72b0015890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40286/0 (socket says 172.21.15.138:40286) 2024-08-20T01:22:02.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 -- 172.21.15.138:0/2129876366 learned_addr learned my addr 172.21.15.138:0/2129876366 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72bd400640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452992cf0 0x5624529a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c6800640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 0x562452a18910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 -- 172.21.15.138:0/2129876366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452992cf0 msgr2=0x5624529a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452992cf0 0x5624529a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 -- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 msgr2=0x562452a18910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 0x562452a18910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c5e00640 1 -- 172.21.15.138:0/2129876366 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562452a18e50 con 0x562452990b20 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.959+0000 7f72c6800640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 0x562452a18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72c5e00640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009380 tx=0x7f72b8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0be922f9be752b7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72abe00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8030070 con 0x562452990b20 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9453+0+0 (secure 0 0 0) 0x7f72b8003800 con 0x562452990b20 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72b8005590 con 0x562452990b20 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f72b8003a00 con 0x562452990b20 2024-08-20T01:22:02.962 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72cfc23740 10 client.17112 fetch_fsmap learned FSMap version 44 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x562452a1a140 con 0x562452990b20 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.960+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72b8005590 con 0x562452990b20 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 44) ==== 8358+0+0 (secure 0 0 0) 0x7f72b8024410 con 0x562452990b20 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 10 client.17112 fetch_fsmap finished waiting for FSMap version 44 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 10 client.17112 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 20 client.17112 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56245299f5a0 0x562452a1c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 4 client.17112 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6ebd2b70", "release": true, "await": true}] 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6ebd2b70", "release": true, "await": true}) -- 0x562452906760 con 0x56245299f5a0 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.961+0000 7f72bd400640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56245299f5a0 0x562452a1c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:02.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.962+0000 7f72abe00640 10 client.17112 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:02.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.962+0000 7f72bd400640 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56245299f5a0 0x562452a1c670 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f5f21735f9e79c7 server_cookie=a7bce968280e694f in_seq=0 out_seq=0 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.966+0000 7f72abe00640 1 -- 172.21.15.138:0/2129876366 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562452906760 con 0x56245299f5a0 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.966+0000 7f72abe00640 10 client.17112 handle_command_reply: tid=0 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 client.17112 shutdown 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 msgr2=0x5624529a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009380 tx=0x7f72b8009660 comp rx=0 tx=0).stop 2024-08-20T01:22:02.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 shutdown_connections 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56245299f5a0 0x562452a1c670 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562452993610 0x562452a18910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562452992cf0 0x5624529a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 --2- 172.21.15.138:0/2129876366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562452990b20 0x5624529a36d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72abe00640 0 client.17112 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 >> 172.21.15.138:0/2129876366 conn(0x562452997bd0 msgr2=0x562452995d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 shutdown_connections 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 1 -- 172.21.15.138:0/2129876366 wait complete. 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.967+0000 7f72cfc23740 20 client.17112 trim_cache size 0 max 16384 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:22:02.968 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 44, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 52, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "6ebd2b70": { 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 52, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 141.4, 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:22:02.969 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:22:02.970 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 msgr2=0x7f72ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 0x7f72ac0b6200 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009710 tx=0x7f72b0009c00 comp rx=0 tx=0).stop 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 msgr2=0x7f72c81a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f72c002f540 tx=0x7f72c00316f0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 shutdown_connections 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f72ac0b3d40 0x7f72ac0b6200 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c810b030 0x7f72c81a0a30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f72c810a6b0 0x7f72c819b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 --2- 172.21.15.138:0/3714808451 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f72c8104ca0 0x7f72c819b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 >> 172.21.15.138:0/3714808451 conn(0x7f72c81006f0 msgr2=0x7f72c8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 shutdown_connections 2024-08-20T01:22:02.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:02.968+0000 7f72cd200640 1 -- 172.21.15.138:0/3714808451 wait complete. 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: 0 cleanup 615 sec 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Operation Count AvgLat MaxLat 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: ---------------------------------------- 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: NTCreateX 67503 3.324 29461.473 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Close 49580 0.005 0.790 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Rename 2861 51.751 4804.825 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Unlink 13639 2.735 6944.165 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Deltree 2 82.352 116.159 2024-08-20T01:22:03.139 INFO:tasks.workunit.client.0.smithi138.stdout: Mkdir 1 0.015 0.015 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Qpathinfo 61166 1.057 1868.761 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Qfileinfo 10730 0.002 0.034 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Qfsinfo 11230 0.299 572.322 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Sfileinfo 5504 3.521 19171.240 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Find 23665 0.301 353.312 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: WriteX 33727 2.438 28724.233 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: ReadX 105861 0.065 692.921 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: LockX 220 0.544 1.126 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: UnlockX 220 0.496 1.269 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: Flush 4733 4.306 842.459 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-08-20T01:22:03.140 INFO:tasks.workunit.client.0.smithi138.stdout:Throughput 3.54611 MB/sec 1 clients 1 procs max_latency=29461.480 ms 2024-08-20T01:22:03.141 INFO:tasks.workunit.client.0.smithi138.stderr:+ dbench 10 2024-08-20T01:22:03.141 INFO:tasks.workunit.client.0.smithi138.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-08-20T01:22:03.141 INFO:tasks.workunit.client.0.smithi138.stdout: 2024-08-20T01:22:03.141 INFO:tasks.workunit.client.0.smithi138.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-08-20T01:22:03.143 INFO:tasks.workunit.client.0.smithi138.stdout:9 of 10 processes prepared for launch 0 sec 2024-08-20T01:22:03.244 INFO:tasks.workunit.client.0.smithi138.stdout:10 of 10 processes prepared for launch 0 sec 2024-08-20T01:22:03.244 INFO:tasks.workunit.client.0.smithi138.stdout:releasing clients 2024-08-20T01:22:03.927 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6ebd2b70', total seconds elapsed: 26.0 2024-08-20T01:22:04.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 476 166.49 MB/sec warmup 1 sec latency 53.593 ms 2024-08-20T01:22:04.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:04 smithi167 ceph-mon[21965]: pgmap v795: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-08-20T01:22:04.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:04 smithi142 ceph-mon[21799]: pgmap v795: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-08-20T01:22:04.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:04 smithi138 ceph-mon[18076]: pgmap v795: 129 pgs: 129 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 13 op/s 2024-08-20T01:22:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 -- 172.21.15.138:0/1387252169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 msgr2=0x7f3bac10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/1387252169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac10f560 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009ca0 tx=0x7f3ba402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 -- 172.21.15.138:0/1387252169 shutdown_connections 2024-08-20T01:22:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/1387252169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac10f560 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/1387252169 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1083e0 0x7f3bac108820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/1387252169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1029e0 0x7f3bac102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 -- 172.21.15.138:0/1387252169 >> 172.21.15.138:0/1387252169 conn(0x7f3bac0fe420 msgr2=0x7f3bac100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 -- 172.21.15.138:0/1387252169 shutdown_connections 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.162+0000 7f3bb2c00640 1 -- 172.21.15.138:0/1387252169 wait complete. 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.163+0000 7f3bb2c00640 1 Processor -- start 2024-08-20T01:22:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.163+0000 7f3bb2c00640 1 -- start start 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 0x7f3bac074520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 0x7f3bac074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac112a70 con 0x7f3bac1083e0 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1128f0 con 0x7f3bac108d60 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb2c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac112bf0 con 0x7f3bac1029e0 2024-08-20T01:22:05.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3babe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 0x7f3bac074520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:43988/0 (socket says 172.21.15.138:43988) 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bb0c00640 1 -- 172.21.15.138:0/3889834074 learned_addr learned my addr 172.21.15.138:0/3889834074 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.164+0000 7f3bab400640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 0x7f3bac074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 -- 172.21.15.138:0/3889834074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 msgr2=0x7f3bac06d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 -- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 msgr2=0x7f3bac074a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 0x7f3bac074a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 -- 172.21.15.138:0/3889834074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac06e1a0 con 0x7f3bac1029e0 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3bab400640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 0x7f3bac074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.165+0000 7f3babe00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 0x7f3bac074520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009bc0 tx=0x7f3b9c00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.166+0000 7f3ba8c00640 1 -- 172.21.15.138:0/3889834074 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c01e070 con 0x7f3bac1029e0 2024-08-20T01:22:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.166+0000 7f3ba8c00640 1 -- 172.21.15.138:0/3889834074 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b9c0198c0 con 0x7f3bac1029e0 2024-08-20T01:22:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.166+0000 7f3ba8c00640 1 -- 172.21.15.138:0/3889834074 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0187d0 con 0x7f3bac1029e0 2024-08-20T01:22:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.166+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac06e490 con 0x7f3bac1029e0 2024-08-20T01:22:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.166+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac0733c0 con 0x7f3bac1029e0 2024-08-20T01:22:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 Processor -- start 2024-08-20T01:22:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 -- start start 2024-08-20T01:22:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61557250 0x559f61562ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f615655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f614cc210 con 0x559f615631e0 2024-08-20T01:22:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559f614bb710 con 0x559f61557250 2024-08-20T01:22:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f614bf750 con 0x559f61556930 2024-08-20T01:22:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43956/0 (socket says 172.21.15.138:43956) 2024-08-20T01:22:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.171+0000 7f3bb0c00640 1 -- 172.21.15.138:0/1910917724 learned_addr learned my addr 172.21.15.138:0/1910917724 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3ba8c00640 1 -- 172.21.15.138:0/3889834074 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3b9c021070 con 0x7f3bac1029e0 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bab400640 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f615655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 -- 172.21.15.138:0/1910917724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61557250 msgr2=0x559f61562ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61557250 0x559f61562ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 -- 172.21.15.138:0/1910917724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 msgr2=0x559f615655d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f615655d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 -- 172.21.15.138:0/1910917724 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f61487de0 con 0x559f61556930 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.172+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009930 tx=0x7f3ba4009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f40177b748baff7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3ba1200640 1 -- 172.21.15.138:0/1910917724 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4031650 con 0x559f61556930 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3ba1200640 1 -- 172.21.15.138:0/1910917724 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4036040 con 0x559f61556930 2024-08-20T01:22:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3ba1200640 1 -- 172.21.15.138:0/1910917724 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4004630 con 0x559f61556930 2024-08-20T01:22:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/1910917724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 msgr2=0x559f61556d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009930 tx=0x7f3ba4009960 comp rx=0 tx=0).stop 2024-08-20T01:22:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.173+0000 7f3ba8c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 0x7f3b900b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3ba8c00640 1 -- 172.21.15.138:0/3889834074 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9541+0+0 (secure 0 0 0) 0x7f3b9c0e7af0 con 0x7f3bac1029e0 2024-08-20T01:22:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3babe00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 0x7f3b900b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/1910917724 shutdown_connections 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f615655d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61557250 0x559f61562ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/1910917724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61556930 0x559f61556d10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.174+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/1910917724 >> 172.21.15.138:0/1910917724 conn(0x559f61554760 msgr2=0x559f61555900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.175+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/1910917724 shutdown_connections 2024-08-20T01:22:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.175+0000 7f3babe00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 0x7f3b900b6280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x559f61559ab0 tx=0x7f3b9c0ae200 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.176+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/1910917724 wait complete. 2024-08-20T01:22:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.177+0000 7f3bb5bf1740 1 -- start start 2024-08-20T01:22:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.177+0000 7f3bb5bf1740 1 Processor -- start 2024-08-20T01:22:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f615629a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f61560320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559f613f4ab0 con 0x559f615631e0 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559f61462140 con 0x559f61556930 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559f61565ce0 con 0x559f61557250 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb5bf1740 10 client.0 _ll_register_callbacks cb 0x559f614755d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bab400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f615629a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bab400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43962/0 (socket says 172.21.15.138:43962) 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.178+0000 7f3bab400640 1 -- 172.21.15.138:0/3764953238 learned_addr learned my addr 172.21.15.138:0/3764953238 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f61560320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bab400640 1 -- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 msgr2=0x559f615629a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bab400640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f615629a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bab400640 1 -- 172.21.15.138:0/3764953238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 msgr2=0x559f61560320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bab400640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f61560320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.179+0000 7f3bab400640 1 -- 172.21.15.138:0/3764953238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f61463170 con 0x559f61557250 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3bab400640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800bf00 tx=0x7f3b980117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42a8c8e4a3984a86 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3b95600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b98012650 con 0x559f61557250 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9541+0+0 (secure 0 0 0) 0x7f3b980127f0 con 0x559f61557250 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4036040 con 0x559f61557250 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.180+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3b980129f0 con 0x559f61557250 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 10 client.27121 fetch_fsmap learned FSMap version 45 2024-08-20T01:22:05.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559f6121e790 con 0x559f61557250 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4036040 con 0x559f61557250 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f3b9801b430 con 0x559f61557250 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 10 client.27121 fetch_fsmap finished waiting for FSMap version 45 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 10 client.27121 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 20 client.27121 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f61572c60 0x559f61575050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 4 client.27121 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.181+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f614ce160 con 0x559f61572c60 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.182+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f61572c60 0x559f61575050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.182+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f61572c60 0x559f61575050 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57dcca266145835a server_cookie=f6e8ce2e80d13458 in_seq=0 out_seq=0 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.182+0000 7f3b95600640 10 client.27121 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.183+0000 7f3b95600640 1 -- 172.21.15.138:0/3764953238 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559f614ce160 con 0x559f61572c60 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.183+0000 7f3b95600640 10 client.27121 handle_command_reply: tid=0 2024-08-20T01:22:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 client.27121 shutdown 2024-08-20T01:22:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 msgr2=0x559f6155fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800bf00 tx=0x7f3b980117b0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 shutdown_connections 2024-08-20T01:22:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f61572c60 0x559f61575050 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f615631e0 0x559f61560320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f61557250 0x559f6155fde0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/3764953238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f615629a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3b95600640 0 client.27121 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 >> 172.21.15.138:0/3764953238 conn(0x559f61554760 msgr2=0x559f6155a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 shutdown_connections 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/3764953238 wait complete. 2024-08-20T01:22:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.184+0000 7f3bb5bf1740 20 client.27121 trim_cache size 0 max 16384 2024-08-20T01:22:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 Processor -- start 2024-08-20T01:22:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 -- start start 2024-08-20T01:22:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6157a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f6157aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b8bc0 con 0x559f61557250 2024-08-20T01:22:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b6460 con 0x559f61556930 2024-08-20T01:22:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.204+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b42d0 con 0x559f615631e0 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:44024/0 (socket says 172.21.15.138:44024) 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 -- 172.21.15.138:0/2722126388 learned_addr learned my addr 172.21.15.138:0/2722126388 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bab400640 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6157a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f6157aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 -- 172.21.15.138:0/2722126388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 msgr2=0x559f6157aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f6157aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 -- 172.21.15.138:0/2722126388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 msgr2=0x559f6157a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6157a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.205+0000 7f3bb0c00640 1 -- 172.21.15.138:0/2722126388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f6155fbc0 con 0x559f61556930 2024-08-20T01:22:05.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.206+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4005e30 tx=0x7f3ba402a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66d46d606d61b11e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.207+0000 7f3b97e00640 1 -- 172.21.15.138:0/2722126388 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4031790 con 0x559f61556930 2024-08-20T01:22:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.207+0000 7f3b97e00640 1 -- 172.21.15.138:0/2722126388 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4036070 con 0x559f61556930 2024-08-20T01:22:05.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.207+0000 7f3b97e00640 1 -- 172.21.15.138:0/2722126388 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba402f610 con 0x559f61556930 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/2722126388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 msgr2=0x559f61553650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4005e30 tx=0x7f3ba402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/2722126388 shutdown_connections 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f6157aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6157a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/2722126388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f61553650 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.208+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/2722126388 >> 172.21.15.138:0/2722126388 conn(0x559f61554760 msgr2=0x559f61552f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.210+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/2722126388 shutdown_connections 2024-08-20T01:22:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.210+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/2722126388 wait complete. 2024-08-20T01:22:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.210+0000 7f3bb5bf1740 1 -- start start 2024-08-20T01:22:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.211+0000 7f3bb5bf1740 1 Processor -- start 2024-08-20T01:22:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f6156b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f615d62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b4920 con 0x559f61557250 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b5d90 con 0x559f61556930 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac1b4ff0 con 0x559f615631e0 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.212+0000 7f3bb5bf1740 10 client.0 _ll_register_callbacks cb 0x7f3b9800b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bb0c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f6156b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.213 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bab400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40354/0 (socket says 172.21.15.138:40354) 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bab400640 1 -- 172.21.15.138:0/319295368 learned_addr learned my addr 172.21.15.138:0/319295368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bab400640 1 -- 172.21.15.138:0/319295368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 msgr2=0x559f615d62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.213+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f615d62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.214+0000 7f3bab400640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f615d62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.214 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.214+0000 7f3bab400640 1 -- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 msgr2=0x559f6156b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.214+0000 7f3bab400640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f6156b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.214+0000 7f3bab400640 1 -- 172.21.15.138:0/319295368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f61565970 con 0x559f61557250 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.214+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f615d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.215+0000 7f3b95600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.215+0000 7f3bab400640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x559f615564f0 tx=0x7f3b98011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2271d31636f1dc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:05.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.215+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801b5f0 con 0x559f61557250 2024-08-20T01:22:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.215+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9541+0+0 (secure 0 0 0) 0x7f3b98012da0 con 0x559f61557250 2024-08-20T01:22:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.215+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba4036070 con 0x559f61557250 2024-08-20T01:22:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.216+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3b98012450 con 0x559f61557250 2024-08-20T01:22:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.216+0000 7f3bb5bf1740 10 client.17130 fetch_fsmap learned FSMap version 45 2024-08-20T01:22:05.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.216+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559f61297620 con 0x559f61557250 2024-08-20T01:22:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba4036070 con 0x559f61557250 2024-08-20T01:22:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f3b98005360 con 0x559f61557250 2024-08-20T01:22:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 10 client.17130 fetch_fsmap finished waiting for FSMap version 45 2024-08-20T01:22:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:05.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 20 client.17130 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f615e6290 0x559f615e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 4 client.17130 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:22:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.217+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f614c9c00 con 0x559f615e6290 2024-08-20T01:22:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.218+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f615e6290 0x559f615e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:05.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.218+0000 7f3b95600640 10 client.17130 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:05.219 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.219+0000 7f3bb0c00640 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f615e6290 0x559f615e86b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83071d81842a1de1 server_cookie=d79ea2f801437159 in_seq=0 out_seq=0 2024-08-20T01:22:05.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 916 107.49 MB/sec warmup 2 sec latency 101.489 ms 2024-08-20T01:22:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.249+0000 7f3b95600640 1 -- 172.21.15.138:0/319295368 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f614c9c00 con 0x559f615e6290 2024-08-20T01:22:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.249+0000 7f3b95600640 10 client.17130 handle_command_reply: tid=0 2024-08-20T01:22:05.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.249+0000 7f3bb5bf1740 1 client.17130 shutdown 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.250+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 msgr2=0x559f6156b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.250+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x559f615564f0 tx=0x7f3b98011df0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.250+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 shutdown_connections 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559f615e6290 0x559f615e86b0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x559f615631e0 0x559f615d62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559f61557250 0x559f6156b620 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 --2- 172.21.15.138:0/319295368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f61556930 0x559f6156b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 >> 172.21.15.138:0/319295368 conn(0x559f61554760 msgr2=0x559f61553760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3b95600640 0 client.17130 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 shutdown_connections 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 1 -- 172.21.15.138:0/319295368 wait complete. 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.251+0000 7f3bb5bf1740 20 client.17130 trim_cache size 0 max 16384 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:22:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:22:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.252+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 msgr2=0x7f3b900b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.252+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 0x7f3b900b6280 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x559f61559ab0 tx=0x7f3b9c0ae200 comp rx=0 tx=0).stop 2024-08-20T01:22:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.252+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 msgr2=0x7f3bac074520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.252+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 0x7f3bac074520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009bc0 tx=0x7f3b9c00b690 comp rx=0 tx=0).stop 2024-08-20T01:22:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 shutdown_connections 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f3b900b3dc0 0x7f3b900b6280 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bac108d60 0x7f3bac06d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3bac1083e0 0x7f3bac074a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 --2- 172.21.15.138:0/3889834074 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3bac1029e0 0x7f3bac074520 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 >> 172.21.15.138:0/3889834074 conn(0x7f3bac0fe420 msgr2=0x7f3bac100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.253+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 shutdown_connections 2024-08-20T01:22:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:05.254+0000 7f3bb2c00640 1 -- 172.21.15.138:0/3889834074 wait complete. 2024-08-20T01:22:05.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:05 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:05.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:05 smithi167 ceph-mon[21965]: osdmap e104: 12 total, 12 up, 12 in 2024-08-20T01:22:05.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:05 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:05.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:05 smithi142 ceph-mon[21799]: osdmap e104: 12 total, 12 up, 12 in 2024-08-20T01:22:05.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:05 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:05.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:05 smithi138 ceph-mon[18076]: osdmap e104: 12 total, 12 up, 12 in 2024-08-20T01:22:06.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 1721 86.57 MB/sec warmup 3 sec latency 65.154 ms 2024-08-20T01:22:06.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:06 smithi167 ceph-mon[21965]: pgmap v797: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-08-20T01:22:06.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:06 smithi142 ceph-mon[21799]: pgmap v797: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-08-20T01:22:06.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:06 smithi138 ceph-mon[18076]: pgmap v797: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 462 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 102 KiB/s wr, 39 op/s 2024-08-20T01:22:07.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 2575 76.17 MB/sec warmup 4 sec latency 57.487 ms 2024-08-20T01:22:08.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3374 74.45 MB/sec warmup 5 sec latency 378.973 ms 2024-08-20T01:22:08.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:08 smithi167 ceph-mon[21965]: pgmap v798: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 465 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 735 KiB/s wr, 93 op/s 2024-08-20T01:22:08.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:08 smithi142 ceph-mon[21799]: pgmap v798: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 465 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 735 KiB/s wr, 93 op/s 2024-08-20T01:22:08.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:08 smithi138 ceph-mon[18076]: pgmap v798: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 465 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 735 KiB/s wr, 93 op/s 2024-08-20T01:22:09.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3676 64.26 MB/sec warmup 6 sec latency 1373.956 ms 2024-08-20T01:22:09.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:09 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:09.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:09 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:09.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:09 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:10.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 3939 55.51 MB/sec warmup 7 sec latency 111.755 ms 2024-08-20T01:22:10.541 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:10 smithi138 ceph-mon[18076]: pgmap v799: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 467 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 888 KiB/s wr, 103 op/s 2024-08-20T01:22:10.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:10 smithi167 ceph-mon[21965]: pgmap v799: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 467 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 888 KiB/s wr, 103 op/s 2024-08-20T01:22:10.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:10 smithi142 ceph-mon[21799]: pgmap v799: 129 pgs: 5 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 105 active+clean; 467 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 888 KiB/s wr, 103 op/s 2024-08-20T01:22:11.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4200 49.29 MB/sec warmup 8 sec latency 173.817 ms 2024-08-20T01:22:12.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 4820 48.95 MB/sec warmup 9 sec latency 94.176 ms 2024-08-20T01:22:12.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:12 smithi167 ceph-mon[21965]: pgmap v800: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-08-20T01:22:12.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:12 smithi142 ceph-mon[21799]: pgmap v800: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-08-20T01:22:12.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:12 smithi138 ceph-mon[18076]: pgmap v800: 129 pgs: 129 active+clean; 487 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 227 op/s 2024-08-20T01:22:13.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 5705 46.13 MB/sec warmup 10 sec latency 42.000 ms 2024-08-20T01:22:14.244 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6323 46.86 MB/sec warmup 11 sec latency 53.870 ms 2024-08-20T01:22:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:14 smithi167 ceph-mon[21965]: pgmap v801: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-20T01:22:14.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:14 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:14.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:14 smithi142 ceph-mon[21799]: pgmap v801: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-20T01:22:14.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:14 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:14.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:14 smithi138 ceph-mon[18076]: pgmap v801: 129 pgs: 129 active+clean; 488 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.3 MiB/s wr, 239 op/s 2024-08-20T01:22:14.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:14 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:15.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 6967 47.25 MB/sec warmup 12 sec latency 54.188 ms 2024-08-20T01:22:15.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:15 smithi167 ceph-mon[21965]: osdmap e105: 12 total, 12 up, 12 in 2024-08-20T01:22:15.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:15 smithi142 ceph-mon[21799]: osdmap e105: 12 total, 12 up, 12 in 2024-08-20T01:22:15.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:15 smithi138 ceph-mon[18076]: osdmap e105: 12 total, 12 up, 12 in 2024-08-20T01:22:16.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7174 44.07 MB/sec warmup 13 sec latency 72.932 ms 2024-08-20T01:22:16.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:16 smithi167 ceph-mon[21965]: pgmap v803: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-08-20T01:22:16.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:16 smithi142 ceph-mon[21799]: pgmap v803: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-08-20T01:22:16.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:16 smithi138 ceph-mon[18076]: pgmap v803: 129 pgs: 129 active+clean; 499 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-08-20T01:22:17.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7381 41.07 MB/sec warmup 14 sec latency 67.288 ms 2024-08-20T01:22:18.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 7698 38.79 MB/sec warmup 15 sec latency 150.226 ms 2024-08-20T01:22:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.406+0000 7f2df5000640 1 -- 172.21.15.138:0/1002595527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077d20 msgr2=0x7f2df01130a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.406+0000 7f2df5000640 1 --2- 172.21.15.138:0/1002595527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077d20 0x7f2df01130a0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009900 tx=0x7f2dd802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 -- 172.21.15.138:0/1002595527 shutdown_connections 2024-08-20T01:22:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 --2- 172.21.15.138:0/1002595527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df01135e0 0x7f2df01159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 --2- 172.21.15.138:0/1002595527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077d20 0x7f2df01130a0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 --2- 172.21.15.138:0/1002595527 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df0077400 0x7f2df00777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 -- 172.21.15.138:0/1002595527 >> 172.21.15.138:0/1002595527 conn(0x7f2df0100640 msgr2=0x7f2df0102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 -- 172.21.15.138:0/1002595527 shutdown_connections 2024-08-20T01:22:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.407+0000 7f2df5000640 1 -- 172.21.15.138:0/1002595527 wait complete. 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 Processor -- start 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 -- start start 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077400 0x7f2df019f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01135e0 0x7f2df01a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0118fe0 con 0x7f2df0077400 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0118e60 con 0x7f2df0077d20 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.408+0000 7f2df5000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0119160 con 0x7f2df01135e0 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45452/0 (socket says 172.21.15.138:45452) 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 -- 172.21.15.138:0/1452777422 learned_addr learned my addr 172.21.15.138:0/1452777422 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2dede00640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077400 0x7f2df019f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2dee800640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01135e0 0x7f2df01a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 -- 172.21.15.138:0/1452777422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01135e0 msgr2=0x7f2df01a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01135e0 0x7f2df01a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 -- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077400 msgr2=0x7f2df019f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077400 0x7f2df019f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 -- 172.21.15.138:0/1452777422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a5600 con 0x7f2df0077d20 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de5400640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8006010 tx=0x7f2dd80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.409+0000 7f2de7400640 1 -- 172.21.15.138:0/1452777422 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd803f030 con 0x7f2df0077d20 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.410+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a5890 con 0x7f2df0077d20 2024-08-20T01:22:18.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.410+0000 7f2de7400640 1 -- 172.21.15.138:0/1452777422 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80044c0 con 0x7f2df0077d20 2024-08-20T01:22:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.410+0000 7f2de7400640 1 -- 172.21.15.138:0/1452777422 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8035070 con 0x7f2df0077d20 2024-08-20T01:22:18.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.410+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01abfb0 con 0x7f2df0077d20 2024-08-20T01:22:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 Processor -- start 2024-08-20T01:22:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 -- start start 2024-08-20T01:22:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2de7400640 1 -- 172.21.15.138:0/1452777422 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2dd8042030 con 0x7f2df0077d20 2024-08-20T01:22:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2310ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d23135d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587d227a210 con 0x5587d23111e0 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5587d2269710 con 0x5587d2304930 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2df7e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5587d226d750 con 0x5587d2305250 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2dee800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2dee800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45464/0 (socket says 172.21.15.138:45464) 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2dee800640 1 -- 172.21.15.138:0/534985709 learned_addr learned my addr 172.21.15.138:0/534985709 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.413+0000 7f2de5400640 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d23135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dede00640 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2310ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 -- 172.21.15.138:0/534985709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 msgr2=0x5587d2310ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2310ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 -- 172.21.15.138:0/534985709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 msgr2=0x5587d23135d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d23135d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 -- 172.21.15.138:0/534985709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d2235de0 con 0x5587d2304930 2024-08-20T01:22:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2dee800640 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f90 tx=0x7f2de80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8da1effdcea0b63f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2ddf400640 1 -- 172.21.15.138:0/534985709 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8004d20 con 0x5587d2304930 2024-08-20T01:22:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2ddf400640 1 -- 172.21.15.138:0/534985709 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8004050 con 0x5587d2304930 2024-08-20T01:22:18.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.414+0000 7f2ddf400640 1 -- 172.21.15.138:0/534985709 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8005220 con 0x5587d2304930 2024-08-20T01:22:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 -- 172.21.15.138:0/534985709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 msgr2=0x5587d2304d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009f90 tx=0x7f2de80118d0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2de7400640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 0x7f2dd40b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 -- 172.21.15.138:0/534985709 shutdown_connections 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d23135d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2310ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 --2- 172.21.15.138:0/534985709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2304d10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 -- 172.21.15.138:0/534985709 >> 172.21.15.138:0/534985709 conn(0x5587d2302760 msgr2=0x5587d2303900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2de7400640 1 -- 172.21.15.138:0/1452777422 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2dd8104630 con 0x7f2df0077d20 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 -- 172.21.15.138:0/534985709 shutdown_connections 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2dede00640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 0x7f2dd40b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.415+0000 7f2df7e14740 1 -- 172.21.15.138:0/534985709 wait complete. 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 -- start start 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 Processor -- start 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2dede00640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 0x7f2dd40b6280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5587d2307ab0 tx=0x7f2de0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2310910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d230e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5587d21a2ab0 con 0x5587d23111e0 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5587d2210140 con 0x5587d2304930 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5587d2313ec0 con 0x5587d2305250 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.416+0000 7f2df7e14740 10 client.0 _ll_register_callbacks cb 0x5587d22235d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:18.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2dede00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2310910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:43914/0 (socket says 172.21.15.138:43914) 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 -- 172.21.15.138:0/2819531345 learned_addr learned my addr 172.21.15.138:0/2819531345 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2dee800640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d230e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 -- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 msgr2=0x5587d2310910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2310910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 -- 172.21.15.138:0/2819531345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 msgr2=0x5587d230e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d230e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 -- 172.21.15.138:0/2819531345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d2211170 con 0x5587d2305250 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2de5400640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80c6f70 tx=0x7f2dd802a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9272278d3f3f392a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2dd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8049070 con 0x5587d2305250 2024-08-20T01:22:18.423 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.417+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2de8004050 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2dd81041b0 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2dd80cd070 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 10 client.27136 fetch_fsmap learned FSMap version 47 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5587d1fcc790 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8004050 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f2dd80d3a60 con 0x5587d2305250 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 10 client.27136 fetch_fsmap finished waiting for FSMap version 47 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 10 client.27136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 20 client.27136 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:18.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d2320c90 0x5587d2323080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 4 client.27136 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.418+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587d227c160 con 0x5587d2320c90 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.419+0000 7f2dede00640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d2320c90 0x5587d2323080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.419+0000 7f2dede00640 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d2320c90 0x5587d2323080 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=987a8e4fca427b13 server_cookie=60c186a87fdad1b3 in_seq=0 out_seq=0 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.419+0000 7f2dd3e00640 10 client.27136 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2819531345 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587d227c160 con 0x5587d2320c90 2024-08-20T01:22:18.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2dd3e00640 10 client.27136 handle_command_reply: tid=0 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 client.27136 shutdown 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 msgr2=0x5587d230dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80c6f70 tx=0x7f2dd802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 shutdown_connections 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d2320c90 0x5587d2323080 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d230e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d230dd50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2819531345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2310910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2dd3e00640 0 client.27136 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 >> 172.21.15.138:0/2819531345 conn(0x5587d2302760 msgr2=0x5587d2308320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 shutdown_connections 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.420+0000 7f2df7e14740 1 -- 172.21.15.138:0/2819531345 wait complete. 2024-08-20T01:22:18.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.421+0000 7f2df7e14740 20 client.27136 trim_cache size 0 max 16384 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.438+0000 7f2df7e14740 1 Processor -- start 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.438+0000 7f2df7e14740 1 -- start start 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d2304930 0x5587d2303ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 0x5587d2328e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b8ac0 con 0x5587d2304930 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b6300 con 0x5587d2305250 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2df7e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b4170 con 0x5587d23111e0 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45494/0 (socket says 172.21.15.138:45494) 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 -- 172.21.15.138:0/3180765252 learned_addr learned my addr 172.21.15.138:0/3180765252 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2dede00640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d2304930 0x5587d2303ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2dee800640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 0x5587d2328e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 -- 172.21.15.138:0/3180765252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 msgr2=0x5587d2328e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 0x5587d2328e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 -- 172.21.15.138:0/3180765252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d2304930 msgr2=0x5587d2303ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d2304930 0x5587d2303ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2de5400640 1 -- 172.21.15.138:0/3180765252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587d23233f0 con 0x5587d2305250 2024-08-20T01:22:18.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.439+0000 7f2dee800640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 0x5587d2328e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.440+0000 7f2de5400640 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80cb0b0 tx=0x7f2dd803b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88573394889c3dd2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.440+0000 7f2ddea00640 1 -- 172.21.15.138:0/3180765252 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80e5070 con 0x5587d2305250 2024-08-20T01:22:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.440+0000 7f2ddea00640 1 -- 172.21.15.138:0/3180765252 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80caa00 con 0x5587d2305250 2024-08-20T01:22:18.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.440+0000 7f2ddea00640 1 -- 172.21.15.138:0/3180765252 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8049040 con 0x5587d2305250 2024-08-20T01:22:18.443 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- 172.21.15.138:0/3180765252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 msgr2=0x5587d2328950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80cb0b0 tx=0x7f2dd803b4f0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- 172.21.15.138:0/3180765252 shutdown_connections 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d23111e0 0x5587d2328e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2305250 0x5587d2328950 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 --2- 172.21.15.138:0/3180765252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d2304930 0x5587d2303ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- 172.21.15.138:0/3180765252 >> 172.21.15.138:0/3180765252 conn(0x5587d2302760 msgr2=0x5587d23037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- 172.21.15.138:0/3180765252 shutdown_connections 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- 172.21.15.138:0/3180765252 wait complete. 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 -- start start 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.441+0000 7f2df7e14740 1 Processor -- start 2024-08-20T01:22:18.444 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2319970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2316e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d2317520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b47c0 con 0x5587d23111e0 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b5c30 con 0x5587d2304930 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01b4e90 con 0x5587d2305250 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2dede00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2319970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2dede00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2319970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45498/0 (socket says 172.21.15.138:45498) 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2dede00640 1 -- 172.21.15.138:0/2219624744 learned_addr learned my addr 172.21.15.138:0/2219624744 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2de5400640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2316e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2df7e14740 10 client.0 _ll_register_callbacks cb 0x7f2dd8005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.442+0000 7f2dee800640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d2317520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 -- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 msgr2=0x5587d2319970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2319970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 -- 172.21.15.138:0/2219624744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 msgr2=0x5587d2317520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d2317520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 -- 172.21.15.138:0/2219624744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587d2045620 con 0x5587d2305250 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2dee800640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d2317520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2de5400640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2316e50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5587d2304400 tx=0x7f2dd8005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b97a1f239b3f37da server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2dd3e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd80e9020 con 0x5587d2305250 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2dd80ca150 con 0x5587d2305250 2024-08-20T01:22:18.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd80caa00 con 0x5587d2305250 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.443+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2dd80e5050 con 0x5587d2305250 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 10 client.27142 fetch_fsmap learned FSMap version 47 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5587d2045620 con 0x5587d2305250 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dd80a7030 con 0x5587d2305250 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 47) ==== 8370+0+0 (secure 0 0 0) 0x7f2dd80c9bc0 con 0x5587d2305250 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 10 client.27142 fetch_fsmap finished waiting for FSMap version 47 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 10 client.27142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 20 client.27142 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d239cd00 0x5587d239f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 4 client.27142 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:22:18.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5587d2277c00 con 0x5587d239cd00 2024-08-20T01:22:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.444+0000 7f2dede00640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d239cd00 0x5587d239f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.445+0000 7f2dede00640 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d239cd00 0x5587d239f120 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60b1ceccf066b6e server_cookie=5fd2042aaad89e51 in_seq=0 out_seq=0 2024-08-20T01:22:18.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.445+0000 7f2dd3e00640 10 client.27142 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.481+0000 7f2dd3e00640 1 -- 172.21.15.138:0/2219624744 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5587d2277c00 con 0x5587d239cd00 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.481+0000 7f2dd3e00640 10 client.27142 handle_command_reply: tid=0 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.481+0000 7f2df7e14740 1 client.27142 shutdown 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 msgr2=0x5587d2316e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2316e50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5587d2304400 tx=0x7f2dd8005bc0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 shutdown_connections 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5587d239cd00 0x5587d239f120 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5587d23111e0 0x5587d2317520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5587d2305250 0x5587d2316e50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 --2- 172.21.15.138:0/2219624744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587d2304930 0x5587d2319970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2dd3e00640 0 client.27142 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 >> 172.21.15.138:0/2219624744 conn(0x5587d2302760 msgr2=0x5587d230f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 shutdown_connections 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 1 -- 172.21.15.138:0/2219624744 wait complete. 2024-08-20T01:22:18.482 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.482+0000 7f2df7e14740 20 client.27142 trim_cache size 0 max 16384 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 msgr2=0x7f2dd40b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 0x7f2dd40b6280 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x5587d2307ab0 tx=0x7f2de0005f90 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 msgr2=0x7f2df019faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8006010 tx=0x7f2dd80040c0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 shutdown_connections 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2dd40b3dc0 0x7f2dd40b6280 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2df01135e0 0x7f2df01a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2df0077d20 0x7f2df019faa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 --2- 172.21.15.138:0/1452777422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2df0077400 0x7f2df019f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 >> 172.21.15.138:0/1452777422 conn(0x7f2df0100640 msgr2=0x7f2df0102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 shutdown_connections 2024-08-20T01:22:18.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:18.483+0000 7f2df5000640 1 -- 172.21.15.138:0/1452777422 wait complete. 2024-08-20T01:22:18.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:18 smithi167 ceph-mon[21965]: pgmap v804: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 299 op/s 2024-08-20T01:22:18.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:18 smithi142 ceph-mon[21799]: pgmap v804: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 299 op/s 2024-08-20T01:22:18.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:18 smithi138 ceph-mon[18076]: pgmap v804: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 5.2 MiB/s wr, 299 op/s 2024-08-20T01:22:19.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 8317 39.03 MB/sec warmup 16 sec latency 103.236 ms 2024-08-20T01:22:20.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 9414 39.18 MB/sec warmup 17 sec latency 181.043 ms 2024-08-20T01:22:20.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:20 smithi167 ceph-mon[21965]: pgmap v805: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-08-20T01:22:20.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:20 smithi142 ceph-mon[21799]: pgmap v805: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-08-20T01:22:20.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:20 smithi138 ceph-mon[18076]: pgmap v805: 129 pgs: 129 active+clean; 512 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.1 MiB/s wr, 291 op/s 2024-08-20T01:22:21.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 10319 40.79 MB/sec warmup 18 sec latency 68.310 ms 2024-08-20T01:22:22.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 11038 40.95 MB/sec warmup 19 sec latency 106.897 ms 2024-08-20T01:22:22.713 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:22 smithi167 ceph-mon[21965]: pgmap v806: 129 pgs: 129 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.9 MiB/s wr, 267 op/s 2024-08-20T01:22:22.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:22 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:22.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:22 smithi142 ceph-mon[21799]: pgmap v806: 129 pgs: 129 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.9 MiB/s wr, 267 op/s 2024-08-20T01:22:22.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:22 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:22.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:22 smithi138 ceph-mon[18076]: pgmap v806: 129 pgs: 129 active+clean; 559 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 7.9 MiB/s wr, 267 op/s 2024-08-20T01:22:22.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:22 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:22:23.246 INFO:tasks.workunit.client.0.smithi138.stdout: 10 11980 41.00 MB/sec warmup 20 sec latency 89.109 ms 2024-08-20T01:22:24.245 INFO:tasks.workunit.client.0.smithi138.stdout: 10 12802 40.88 MB/sec warmup 21 sec latency 225.033 ms 2024-08-20T01:22:24.741 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:24 smithi138 ceph-mon[18076]: pgmap v807: 129 pgs: 129 active+clean; 576 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.5 MiB/s wr, 333 op/s 2024-08-20T01:22:24.741 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:24 smithi142 ceph-mon[21799]: pgmap v807: 129 pgs: 129 active+clean; 576 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.5 MiB/s wr, 333 op/s 2024-08-20T01:22:24.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:24 smithi167 ceph-mon[21965]: pgmap v807: 129 pgs: 129 active+clean; 576 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 9.5 MiB/s wr, 333 op/s 2024-08-20T01:22:25.252 INFO:tasks.workunit.client.0.smithi138.stdout: 10 12840 39.08 MB/sec warmup 22 sec latency 726.085 ms 2024-08-20T01:22:26.252 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13213 38.40 MB/sec warmup 23 sec latency 1661.430 ms 2024-08-20T01:22:26.642 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:26 smithi138 ceph-mon[18076]: pgmap v808: 129 pgs: 129 active+clean; 625 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 382 op/s 2024-08-20T01:22:26.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:26 smithi142 ceph-mon[21799]: pgmap v808: 129 pgs: 129 active+clean; 625 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 382 op/s 2024-08-20T01:22:26.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:26 smithi167 ceph-mon[21965]: pgmap v808: 129 pgs: 129 active+clean; 625 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 14 MiB/s wr, 382 op/s 2024-08-20T01:22:27.252 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13567 37.86 MB/sec warmup 24 sec latency 2661.480 ms 2024-08-20T01:22:28.252 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 36.72 MB/sec warmup 25 sec latency 3488.330 ms 2024-08-20T01:22:28.588 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:28 smithi138 ceph-mon[18076]: pgmap v809: 129 pgs: 129 active+clean; 775 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 454 op/s 2024-08-20T01:22:28.742 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:28 smithi142 ceph-mon[21799]: pgmap v809: 129 pgs: 129 active+clean; 775 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 454 op/s 2024-08-20T01:22:28.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:28 smithi167 ceph-mon[21965]: pgmap v809: 129 pgs: 129 active+clean; 775 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 24 MiB/s wr, 454 op/s 2024-08-20T01:22:29.252 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 35.31 MB/sec warmup 26 sec latency 4488.396 ms 2024-08-20T01:22:30.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 34.00 MB/sec warmup 27 sec latency 5488.446 ms 2024-08-20T01:22:30.761 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:30 smithi138 ceph-mon[18076]: pgmap v810: 129 pgs: 129 active+clean; 788 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 407 op/s 2024-08-20T01:22:30.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:30 smithi167 ceph-mon[21965]: pgmap v810: 129 pgs: 129 active+clean; 788 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 407 op/s 2024-08-20T01:22:30.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:30 smithi142 ceph-mon[21799]: pgmap v810: 129 pgs: 129 active+clean; 788 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 24 MiB/s wr, 407 op/s 2024-08-20T01:22:31.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 32.78 MB/sec warmup 28 sec latency 6488.534 ms 2024-08-20T01:22:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1717046403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410a610 msgr2=0x7f2a6410aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- 172.21.15.138:0/1717046403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410a610 0x7f2a6410aa50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009900 tx=0x7f2a5002a400 comp rx=0 tx=0).stop 2024-08-20T01:22:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1717046403 shutdown_connections 2024-08-20T01:22:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- 172.21.15.138:0/1717046403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a6410af90 0x7f2a64111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- 172.21.15.138:0/1717046403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410a610 0x7f2a6410aa50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- 172.21.15.138:0/1717046403 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 0x7f2a64104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1717046403 >> 172.21.15.138:0/1717046403 conn(0x7f2a64100610 msgr2=0x7f2a64102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1717046403 shutdown_connections 2024-08-20T01:22:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1717046403 wait complete. 2024-08-20T01:22:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 Processor -- start 2024-08-20T01:22:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- start start 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 0x7f2a6419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a6410a610 0x7f2a6419b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64114c70 con 0x7f2a6410af90 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64114af0 con 0x7f2a6410a610 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a64114df0 con 0x7f2a64104c10 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58214/0 (socket says 172.21.15.138:58214) 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 -- 172.21.15.138:0/1588641142 learned_addr learned my addr 172.21.15.138:0/1588641142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a62a00640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 0x7f2a6419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 -- 172.21.15.138:0/1588641142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 msgr2=0x7f2a6419b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 0x7f2a6419b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 -- 172.21.15.138:0/1588641142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a6410a610 msgr2=0x7f2a6419b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a6410a610 0x7f2a6419b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 -- 172.21.15.138:0/1588641142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a641a1210 con 0x7f2a6410af90 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a63400640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00efd0 tx=0x7f2a5c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a5b400640 1 -- 172.21.15.138:0/1588641142 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c003b70 con 0x7f2a6410af90 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a641a1500 con 0x7f2a6410af90 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a5b400640 1 -- 172.21.15.138:0/1588641142 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c003d10 con 0x7f2a6410af90 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a5b400640 1 -- 172.21.15.138:0/1588641142 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0108a0 con 0x7f2a6410af90 2024-08-20T01:22:31.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.599+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a641a7bc0 con 0x7f2a6410af90 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 Processor -- start 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a5b400640 1 -- 172.21.15.138:0/1588641142 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2a5c004020 con 0x7f2a6410af90 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 -- start start 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce091ef0 0x5577ce09d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce09de80 0x5577ce0a0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577ce007210 con 0x5577ce091ef0 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5577cdff6710 con 0x5577ce0915d0 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577cdffa750 con 0x5577ce09de80 2024-08-20T01:22:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce091ef0 0x5577ce09d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35684/0 (socket says 172.21.15.138:35684) 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 -- 172.21.15.138:0/189627128 learned_addr learned my addr 172.21.15.138:0/189627128 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce09de80 0x5577ce0a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 -- 172.21.15.138:0/189627128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce09de80 msgr2=0x5577ce0a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce09de80 0x5577ce0a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 -- 172.21.15.138:0/189627128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce091ef0 msgr2=0x5577ce09d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce091ef0 0x5577ce09d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 -- 172.21.15.138:0/189627128 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cdfc2de0 con 0x5577ce0915d0 2024-08-20T01:22:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a5b400640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 0x7f2a480b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a62000640 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005c20 tx=0x7f2a50031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c80aa2fac47ca997 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.605+0000 7f2a5b400640 1 -- 172.21.15.138:0/1588641142 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2a5c0e48e0 con 0x7f2a6410af90 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a62a00640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 0x7f2a480b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a4f400640 1 -- 172.21.15.138:0/189627128 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a50035070 con 0x5577ce0915d0 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a4f400640 1 -- 172.21.15.138:0/189627128 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a50031bd0 con 0x5577ce0915d0 2024-08-20T01:22:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a4f400640 1 -- 172.21.15.138:0/189627128 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a500039c0 con 0x5577ce0915d0 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/189627128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 msgr2=0x5577ce0919b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a62a00640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 0x7f2a480b6200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5577ce094750 tx=0x7f2a54005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2a50005c20 tx=0x7f2a50031400 comp rx=0 tx=0).stop 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/189627128 shutdown_connections 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce09de80 0x5577ce0a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce091ef0 0x5577ce09d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/189627128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0919b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/189627128 >> 172.21.15.138:0/189627128 conn(0x5577ce08f3b0 msgr2=0x5577ce0905a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/189627128 shutdown_connections 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/189627128 wait complete. 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- start start 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 Processor -- start 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce09ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce09b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5577cdf2fab0 con 0x5577ce09de80 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5577cdf9d140 con 0x5577ce0915d0 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5577ce0a0b00 con 0x5577ce091ef0 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 10 client.0 _ll_register_callbacks cb 0x5577cdfb05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58232/0 (socket says 172.21.15.138:58232) 2024-08-20T01:22:31.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 -- 172.21.15.138:0/123328546 learned_addr learned my addr 172.21.15.138:0/123328546 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a62000640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce09b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 -- 172.21.15.138:0/123328546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 msgr2=0x5577ce09b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce09b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 -- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 msgr2=0x5577ce09ade0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce09ade0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 -- 172.21.15.138:0/123328546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cdf9e170 con 0x5577ce09de80 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a63400640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ab080 tx=0x7f2a5c0aacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eca9ba9345e1a9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0af6f0 con 0x5577ce09de80 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2a5c0a7380 con 0x5577ce09de80 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5577cdfcda30 con 0x5577ce09de80 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c0b41a0 con 0x5577ce09de80 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 10 client.17160 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:31.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5577cdd59790 con 0x5577ce09de80 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5577cdfcda30 con 0x5577ce09de80 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f2a5c01f020 con 0x5577ce09de80 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 10 client.17160 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 20 client.17160 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce0ade00 0x5577ce0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 4 client.17160 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577ce009160 con 0x5577ce0ade00 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a62a00640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce0ade00 0x5577ce0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a62a00640 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce0ade00 0x5577ce0b0230 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17eae87d9edfc6f6 server_cookie=b865c8e8633dd70b in_seq=0 out_seq=0 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.610+0000 7f2a43e00640 10 client.17160 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a43e00640 1 -- 172.21.15.138:0/123328546 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577ce009160 con 0x5577ce0ade00 2024-08-20T01:22:31.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a43e00640 10 client.17160 handle_command_reply: tid=0 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 client.17160 shutdown 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 msgr2=0x5577ce09b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ab080 tx=0x7f2a5c0aacc0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 shutdown_connections 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce0ade00 0x5577ce0b0230 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b860 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce09b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/123328546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce09ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a43e00640 0 client.17160 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 >> 172.21.15.138:0/123328546 conn(0x5577ce08f3b0 msgr2=0x5577ce094fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 shutdown_connections 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 1 -- 172.21.15.138:0/123328546 wait complete. 2024-08-20T01:22:31.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.615+0000 7f2a6c5de740 20 client.17160 trim_cache size 0 max 16384 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 Processor -- start 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 -- start start 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce0915d0 0x5577ce095790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce091ef0 0x5577ce099cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b4580 con 0x5577ce09de80 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b1dc0 con 0x5577ce091ef0 2024-08-20T01:22:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641afc30 con 0x5577ce0915d0 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58236/0 (socket says 172.21.15.138:58236) 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 -- 172.21.15.138:0/2582441523 learned_addr learned my addr 172.21.15.138:0/2582441523 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a62000640 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce091ef0 0x5577ce099cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a62a00640 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce0915d0 0x5577ce095790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 -- 172.21.15.138:0/2582441523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce0915d0 msgr2=0x5577ce095790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce0915d0 0x5577ce095790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 -- 172.21.15.138:0/2582441523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce091ef0 msgr2=0x5577ce099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce091ef0 0x5577ce099cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 -- 172.21.15.138:0/2582441523 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cddd2620 con 0x5577ce09de80 2024-08-20T01:22:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a63400640 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0a8360 tx=0x7f2a5c0ab270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74100bc6c2abfbd2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a4ea00640 1 -- 172.21.15.138:0/2582441523 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0af6f0 con 0x5577ce09de80 2024-08-20T01:22:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a4ea00640 1 -- 172.21.15.138:0/2582441523 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c070120 con 0x5577ce09de80 2024-08-20T01:22:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a4ea00640 1 -- 172.21.15.138:0/2582441523 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0aa350 con 0x5577ce09de80 2024-08-20T01:22:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 -- 172.21.15.138:0/2582441523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 msgr2=0x5577ce09b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.628+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0a8360 tx=0x7f2a5c0ab270 comp rx=0 tx=0).stop 2024-08-20T01:22:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- 172.21.15.138:0/2582441523 shutdown_connections 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce09b4c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce091ef0 0x5577ce099cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/2582441523 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce0915d0 0x5577ce095790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- 172.21.15.138:0/2582441523 >> 172.21.15.138:0/2582441523 conn(0x5577ce08f3b0 msgr2=0x5577ce09adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- 172.21.15.138:0/2582441523 shutdown_connections 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- 172.21.15.138:0/2582441523 wait complete. 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- start start 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 Processor -- start 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0a6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce0a66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b0280 con 0x5577ce09de80 2024-08-20T01:22:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b16f0 con 0x5577ce0915d0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2a641b0950 con 0x5577ce091ef0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a6c5de740 10 client.0 _ll_register_callbacks cb 0x7f2a5c004830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a63400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:58240/0 (socket says 172.21.15.138:58240) 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a63400640 1 -- 172.21.15.138:0/3799824887 learned_addr learned my addr 172.21.15.138:0/3799824887 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a62a00640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.633+0000 7f2a62000640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce0a66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 -- 172.21.15.138:0/3799824887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 msgr2=0x5577ce0a66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce0a66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 -- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 msgr2=0x5577ce0a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 -- 172.21.15.138:0/3799824887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577ce0a20d0 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a62000640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce0a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a63400640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ab080 tx=0x7f2a5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b5861b181beaf7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0af6f0 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f2a5c004b40 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a5c070120 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2a5c0a6d90 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 10 client.17166 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5577ce118830 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a5c070120 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f2a5c01f070 con 0x5577ce09de80 2024-08-20T01:22:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 10 client.17166 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 20 client.17166 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce118c90 0x5577ce11b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 4 client.17166 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5577ce004c00 con 0x5577ce118c90 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a62a00640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce118c90 0x5577ce11b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a62a00640 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce118c90 0x5577ce11b0b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64f1108b092798f7 server_cookie=bb463ca3935840b2 in_seq=0 out_seq=0 2024-08-20T01:22:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.634+0000 7f2a43e00640 10 client.17166 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a43e00640 1 -- 172.21.15.138:0/3799824887 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5577ce004c00 con 0x5577ce118c90 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a43e00640 10 client.17166 handle_command_reply: tid=0 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 client.17166 shutdown 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 msgr2=0x5577ce0a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c0ab080 tx=0x7f2a5c009f90 comp rx=0 tx=0).stop 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 shutdown_connections 2024-08-20T01:22:31.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5577ce118c90 0x5577ce11b0b0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5577ce09de80 0x5577ce0a18a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a43e00640 0 client.17166 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5577ce091ef0 0x5577ce0a66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 --2- 172.21.15.138:0/3799824887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5577ce0915d0 0x5577ce0a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 >> 172.21.15.138:0/3799824887 conn(0x5577ce08f3b0 msgr2=0x5577ce09cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.979+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 shutdown_connections 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.980+0000 7f2a6c5de740 1 -- 172.21.15.138:0/3799824887 wait complete. 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.980+0000 7f2a6c5de740 20 client.17166 trim_cache size 0 max 16384 2024-08-20T01:22:31.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:22:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:22:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:22:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 msgr2=0x7f2a480b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 0x7f2a480b6200 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x5577ce094750 tx=0x7f2a54005f90 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 msgr2=0x7f2a641a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00efd0 tx=0x7f2a5c00c6a0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 shutdown_connections 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2a480b3d40 0x7f2a480b6200 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a6410af90 0x7f2a641a0a40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a6410a610 0x7f2a6419b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 --2- 172.21.15.138:0/1588641142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2a64104c10 0x7f2a6419b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 >> 172.21.15.138:0/1588641142 conn(0x7f2a64100610 msgr2=0x7f2a64102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 shutdown_connections 2024-08-20T01:22:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:31.981+0000 7f2a69600640 1 -- 172.21.15.138:0/1588641142 wait complete. 2024-08-20T01:22:32.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 31.65 MB/sec warmup 29 sec latency 7488.603 ms 2024-08-20T01:22:32.765 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:32 smithi138 ceph-mon[18076]: pgmap v811: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 540 op/s 2024-08-20T01:22:32.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:32 smithi167 ceph-mon[21965]: pgmap v811: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 540 op/s 2024-08-20T01:22:32.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:32 smithi142 ceph-mon[21799]: pgmap v811: 129 pgs: 129 active+clean; 767 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 26 MiB/s wr, 540 op/s 2024-08-20T01:22:33.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 30.60 MB/sec warmup 30 sec latency 8488.644 ms 2024-08-20T01:22:34.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 29.61 MB/sec warmup 31 sec latency 9488.719 ms 2024-08-20T01:22:34.791 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:34 smithi138 ceph-mon[18076]: pgmap v812: 129 pgs: 129 active+clean; 756 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 22 MiB/s wr, 483 op/s 2024-08-20T01:22:34.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:34 smithi167 ceph-mon[21965]: pgmap v812: 129 pgs: 129 active+clean; 756 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 22 MiB/s wr, 483 op/s 2024-08-20T01:22:34.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:34 smithi142 ceph-mon[21799]: pgmap v812: 129 pgs: 129 active+clean; 756 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 22 MiB/s wr, 483 op/s 2024-08-20T01:22:35.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 28.69 MB/sec warmup 32 sec latency 10488.808 ms 2024-08-20T01:22:36.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 27.82 MB/sec warmup 33 sec latency 11488.875 ms 2024-08-20T01:22:36.803 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:36 smithi138 ceph-mon[18076]: pgmap v813: 129 pgs: 129 active+clean; 748 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 487 op/s 2024-08-20T01:22:36.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:36 smithi167 ceph-mon[21965]: pgmap v813: 129 pgs: 129 active+clean; 748 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 487 op/s 2024-08-20T01:22:36.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:36 smithi142 ceph-mon[21799]: pgmap v813: 129 pgs: 129 active+clean; 748 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 21 MiB/s wr, 487 op/s 2024-08-20T01:22:37.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 13700 27.00 MB/sec warmup 34 sec latency 12488.916 ms 2024-08-20T01:22:38.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 14363 26.63 MB/sec warmup 35 sec latency 12995.197 ms 2024-08-20T01:22:38.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:38 smithi167 ceph-mon[21965]: pgmap v814: 129 pgs: 129 active+clean; 744 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 505 op/s 2024-08-20T01:22:38.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:38 smithi142 ceph-mon[21799]: pgmap v814: 129 pgs: 129 active+clean; 744 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 505 op/s 2024-08-20T01:22:39.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:38 smithi138 ceph-mon[18076]: pgmap v814: 129 pgs: 129 active+clean; 744 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 505 op/s 2024-08-20T01:22:39.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 15164 27.47 MB/sec warmup 36 sec latency 352.678 ms 2024-08-20T01:22:40.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 16016 28.17 MB/sec warmup 37 sec latency 133.801 ms 2024-08-20T01:22:40.687 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:40 smithi138 ceph-mon[18076]: pgmap v815: 129 pgs: 129 active+clean; 745 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 327 op/s 2024-08-20T01:22:40.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:40 smithi167 ceph-mon[21965]: pgmap v815: 129 pgs: 129 active+clean; 745 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 327 op/s 2024-08-20T01:22:40.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:40 smithi142 ceph-mon[21799]: pgmap v815: 129 pgs: 129 active+clean; 745 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 327 op/s 2024-08-20T01:22:41.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 16614 27.99 MB/sec warmup 38 sec latency 205.241 ms 2024-08-20T01:22:42.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 17795 28.77 MB/sec warmup 39 sec latency 120.445 ms 2024-08-20T01:22:42.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:42 smithi167 ceph-mon[21965]: pgmap v816: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-08-20T01:22:42.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:42 smithi142 ceph-mon[21799]: pgmap v816: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-08-20T01:22:43.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:42 smithi138 ceph-mon[18076]: pgmap v816: 129 pgs: 129 active+clean; 751 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 367 op/s 2024-08-20T01:22:43.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 18920 29.95 MB/sec warmup 40 sec latency 49.481 ms 2024-08-20T01:22:43.672 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.0 2024-08-20T01:22:43.672 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:22:44.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 19672 30.16 MB/sec warmup 41 sec latency 125.506 ms 2024-08-20T01:22:44.281 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:22:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:44 smithi167 ceph-mon[21965]: pgmap v817: 129 pgs: 129 active+clean; 755 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 257 op/s 2024-08-20T01:22:44.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:44 smithi142 ceph-mon[21799]: pgmap v817: 129 pgs: 129 active+clean; 755 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 257 op/s 2024-08-20T01:22:45.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:44 smithi138 ceph-mon[18076]: pgmap v817: 129 pgs: 129 active+clean; 755 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 257 op/s 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.163+0000 7f59f7e00640 1 -- 172.21.15.138:0/2155283623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 msgr2=0x7f59f0101910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2155283623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0101910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009900 tx=0x7f59e403b490 comp rx=0 tx=0).stop 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 -- 172.21.15.138:0/2155283623 shutdown_connections 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2155283623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f010f0e0 0x7f59f01114d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2155283623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f0101e50 0x7f59f010eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2155283623 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0101910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.164+0000 7f59f7e00640 1 -- 172.21.15.138:0/2155283623 >> 172.21.15.138:0/2155283623 conn(0x7f59f0078620 msgr2=0x7f59f00ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.165+0000 7f59f7e00640 1 -- 172.21.15.138:0/2155283623 shutdown_connections 2024-08-20T01:22:45.165 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.165+0000 7f59f7e00640 1 -- 172.21.15.138:0/2155283623 wait complete. 2024-08-20T01:22:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.167+0000 7f59f7e00640 1 Processor -- start 2024-08-20T01:22:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.168+0000 7f59f7e00640 1 -- start start 2024-08-20T01:22:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.168+0000 7f59f7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.168+0000 7f59f7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f0101e50 0x7f59f0197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 0x7f59f019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f7e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0105fd0 con 0x7f59f0101e50 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f7e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0105e50 con 0x7f59f010f0e0 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f7e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0106150 con 0x7f59f0101530 2024-08-20T01:22:45.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f5400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:35640/0 (socket says 172.21.15.138:35640) 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 0x7f59f019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f5400640 1 -- 172.21.15.138:0/2339374784 learned_addr learned my addr 172.21.15.138:0/2339374784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.169+0000 7f59f5e00640 1 -- 172.21.15.138:0/2339374784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 msgr2=0x7f59f0197250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.170+0000 7f59f5e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0197250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.170+0000 7f59f5e00640 1 -- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f0101e50 msgr2=0x7f59f0197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.170+0000 7f59f5e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f0101e50 0x7f59f0197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.170+0000 7f59f5e00640 1 -- 172.21.15.138:0/2339374784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f019d000 con 0x7f59f010f0e0 2024-08-20T01:22:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.171+0000 7f59f5e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 0x7f59f019c830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59e800da10 tx=0x7f59e800ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.172+0000 7f59ee000640 1 -- 172.21.15.138:0/2339374784 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e800bd70 con 0x7f59f010f0e0 2024-08-20T01:22:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.172+0000 7f59ee000640 1 -- 172.21.15.138:0/2339374784 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e8003e20 con 0x7f59f010f0e0 2024-08-20T01:22:45.172 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.172+0000 7f59ee000640 1 -- 172.21.15.138:0/2339374784 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80106c0 con 0x7f59f010f0e0 2024-08-20T01:22:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.172+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f019d2f0 con 0x7f59f010f0e0 2024-08-20T01:22:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.173+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f01a39b0 con 0x7f59f010f0e0 2024-08-20T01:22:45.178 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.178+0000 7f59fade0740 1 Processor -- start 2024-08-20T01:22:45.180 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.180+0000 7f59ee000640 1 -- 172.21.15.138:0/2339374784 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f59e8010860 con 0x7f59f010f0e0 2024-08-20T01:22:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.183+0000 7f59ee000640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 0x7f59d80b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.183+0000 7f59ee000640 1 -- 172.21.15.138:0/2339374784 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f59e80e48e0 con 0x7f59f010f0e0 2024-08-20T01:22:45.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 -- start start 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a88940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b9579f2210 con 0x55b957a7c5d0 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b9579e1710 con 0x55b957a88e80 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.184+0000 7f59fade0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b9579e5750 con 0x55b957a7cef0 2024-08-20T01:22:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.185+0000 7f59f5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.185+0000 7f59f5400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53970/0 (socket says 172.21.15.138:53970) 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.185+0000 7f59f5400640 1 -- 172.21.15.138:0/128406850 learned_addr learned my addr 172.21.15.138:0/128406850 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.185+0000 7f59f4a00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 0x7f59d80b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5e00640 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f4a00640 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a88940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5400640 1 -- 172.21.15.138:0/128406850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 msgr2=0x55b957a88940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5400640 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a88940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5400640 1 -- 172.21.15.138:0/128406850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 msgr2=0x55b957a8b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5400640 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f5400640 1 -- 172.21.15.138:0/128406850 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9579adde0 con 0x55b957a7c5d0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.186+0000 7f59f4a00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 0x7f59d80b6300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59dc011fd0 tx=0x7f59dc00b8f0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.187+0000 7f59f5400640 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f59e4004150 tx=0x7f59e4004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dfd68bdd7e008f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.188+0000 7f59e2600640 1 -- 172.21.15.138:0/128406850 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e4046030 con 0x55b957a7c5d0 2024-08-20T01:22:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.188+0000 7f59e2600640 1 -- 172.21.15.138:0/128406850 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e40054f0 con 0x55b957a7c5d0 2024-08-20T01:22:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.188+0000 7f59e2600640 1 -- 172.21.15.138:0/128406850 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e40035f0 con 0x55b957a7c5d0 2024-08-20T01:22:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 -- 172.21.15.138:0/128406850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 msgr2=0x55b957a7c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f59e4004150 tx=0x7f59e4004180 comp rx=0 tx=0).stop 2024-08-20T01:22:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 -- 172.21.15.138:0/128406850 shutdown_connections 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a88940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 --2- 172.21.15.138:0/128406850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a7c9b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 -- 172.21.15.138:0/128406850 >> 172.21.15.138:0/128406850 conn(0x55b957a7a3b0 msgr2=0x55b957a7b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 -- 172.21.15.138:0/128406850 shutdown_connections 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.189+0000 7f59fade0740 1 -- 172.21.15.138:0/128406850 wait complete. 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.190+0000 7f59fade0740 1 -- start start 2024-08-20T01:22:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.190+0000 7f59fade0740 1 Processor -- start 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a7c5d0 0x55b957a85de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 0x55b957a86860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b95791aab0 con 0x55b957a88e80 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b957988140 con 0x55b957a7c5d0 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b957a8bb00 con 0x55b957a7cef0 2024-08-20T01:22:45.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59fade0740 10 client.0 _ll_register_callbacks cb 0x55b95799b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:35650/0 (socket says 172.21.15.138:35650) 2024-08-20T01:22:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.191+0000 7f59f4a00640 1 -- 172.21.15.138:0/3312913106 learned_addr learned my addr 172.21.15.138:0/3312913106 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.192+0000 7f59f5e00640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 0x55b957a86860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.192+0000 7f59f5400640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a7c5d0 0x55b957a85de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f4a00640 1 -- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a7c5d0 msgr2=0x55b957a85de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a7c5d0 0x55b957a85de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f4a00640 1 -- 172.21.15.138:0/3312913106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 msgr2=0x55b957a86860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 0x55b957a86860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f4a00640 1 -- 172.21.15.138:0/3312913106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b957989170 con 0x55b957a7cef0 2024-08-20T01:22:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.193+0000 7f59f5e00640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 0x55b957a86860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.195+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59dc010a80 tx=0x7f59dc005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b981f4a31ff092 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.195+0000 7f59d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.195+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc01aaf0 con 0x55b957a7cef0 2024-08-20T01:22:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.195+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f59dc01ac90 con 0x55b957a7cef0 2024-08-20T01:22:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.195+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b9579b8a30 con 0x55b957a7cef0 2024-08-20T01:22:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.196+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f59dc026400 con 0x55b957a7cef0 2024-08-20T01:22:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.196+0000 7f59fade0740 10 client.27172 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.196+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b957744790 con 0x55b957a7cef0 2024-08-20T01:22:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.196+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b9579b8a30 con 0x55b957a7cef0 2024-08-20T01:22:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f59dc00f040 con 0x55b957a7cef0 2024-08-20T01:22:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 10 client.27172 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 10 client.27172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 20 client.27172 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957a98dc0 0x55b957a9b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 4 client.27172 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.197+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9579f4160 con 0x55b957a98dc0 2024-08-20T01:22:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.198+0000 7f59f5400640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957a98dc0 0x55b957a9b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.199+0000 7f59f5400640 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957a98dc0 0x55b957a9b1d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdda078a12e52f88 server_cookie=18e3b334b6864781 in_seq=0 out_seq=0 2024-08-20T01:22:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.199+0000 7f59d6a00640 10 client.27172 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.203+0000 7f59d6a00640 1 -- 172.21.15.138:0/3312913106 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9579f4160 con 0x55b957a98dc0 2024-08-20T01:22:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.203+0000 7f59d6a00640 10 client.27172 handle_command_reply: tid=0 2024-08-20T01:22:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.203+0000 7f59fade0740 1 client.27172 shutdown 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 msgr2=0x55b957a86320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59dc010a80 tx=0x7f59dc005f90 comp rx=0 tx=0).stop 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 shutdown_connections 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957a98dc0 0x55b957a9b1d0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59d6a00640 0 client.27172 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a88e80 0x55b957a86860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7cef0 0x55b957a86320 secure :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f59dc010a80 tx=0x7f59dc005f90 comp rx=0 tx=0).stop 2024-08-20T01:22:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 --2- 172.21.15.138:0/3312913106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a7c5d0 0x55b957a85de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.205+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 >> 172.21.15.138:0/3312913106 conn(0x55b957a7a3b0 msgr2=0x55b957a7ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.206+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 shutdown_connections 2024-08-20T01:22:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.206+0000 7f59fade0740 1 -- 172.21.15.138:0/3312913106 wait complete. 2024-08-20T01:22:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.206+0000 7f59fade0740 20 client.27172 trim_cache size 0 max 16384 2024-08-20T01:22:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 Processor -- start 2024-08-20T01:22:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 -- start start 2024-08-20T01:22:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 0x55b957a80720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 0x55b957a84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01b0320 con 0x55b957a81730 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01adb60 con 0x55b957a88e80 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.233+0000 7f59fade0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ab9d0 con 0x55b957a7c5d0 2024-08-20T01:22:45.234 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f5e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 0x55b957a84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42488/0 (socket says 172.21.15.138:42488) 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 -- 172.21.15.138:0/4030788347 learned_addr learned my addr 172.21.15.138:0/4030788347 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 -- 172.21.15.138:0/4030788347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 msgr2=0x55b957a80720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f5400640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 0x55b957a80720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 0x55b957a80720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 -- 172.21.15.138:0/4030788347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 msgr2=0x55b957a84b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 0x55b957a84b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.235 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f4a00640 1 -- 172.21.15.138:0/4030788347 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9577bd620 con 0x55b957a88e80 2024-08-20T01:22:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f5400640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 0x55b957a80720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:22:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.234+0000 7f59f5e00640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 0x55b957a84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.235+0000 7f59f4a00640 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f59dc010a50 tx=0x7f59dc0119e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17ec7bd8edf6eb7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.235+0000 7f59e1c00640 1 -- 172.21.15.138:0/4030788347 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc010680 con 0x55b957a88e80 2024-08-20T01:22:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.235+0000 7f59e1c00640 1 -- 172.21.15.138:0/4030788347 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59dc030420 con 0x55b957a88e80 2024-08-20T01:22:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.235+0000 7f59e1c00640 1 -- 172.21.15.138:0/4030788347 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc026b00 con 0x55b957a88e80 2024-08-20T01:22:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.236+0000 7f59fade0740 1 -- 172.21.15.138:0/4030788347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 msgr2=0x55b957a84640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.236+0000 7f59fade0740 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f59dc010a50 tx=0x7f59dc0119e0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 -- 172.21.15.138:0/4030788347 shutdown_connections 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a81730 0x55b957a84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a84640 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 --2- 172.21.15.138:0/4030788347 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957a7c5d0 0x55b957a80720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 -- 172.21.15.138:0/4030788347 >> 172.21.15.138:0/4030788347 conn(0x55b957a7a3b0 msgr2=0x55b957a81020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 -- 172.21.15.138:0/4030788347 shutdown_connections 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.237+0000 7f59fade0740 1 -- 172.21.15.138:0/4030788347 wait complete. 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 -- start start 2024-08-20T01:22:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 Processor -- start 2024-08-20T01:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a8c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957affa90 0x55b957b01e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ac020 con 0x55b957a7c5d0 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ad490 con 0x55b957a88e80 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f59f01ac6f0 con 0x55b957affa90 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.238+0000 7f59fade0740 10 client.0 _ll_register_callbacks cb 0x7f59dc0118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42502/0 (socket says 172.21.15.138:42502) 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 -- 172.21.15.138:0/3366516763 learned_addr learned my addr 172.21.15.138:0/3366516763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f5400640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a8c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 -- 172.21.15.138:0/3366516763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957affa90 msgr2=0x55b957b01e80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957affa90 0x55b957b01e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 -- 172.21.15.138:0/3366516763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 msgr2=0x55b957a8c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.239+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a8c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.240+0000 7f59f4a00640 1 -- 172.21.15.138:0/3366516763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b957b023c0 con 0x55b957a88e80 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.240+0000 7f59f5400640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a8c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.240+0000 7f59f4a00640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59dc011c00 tx=0x7f59dc01ad20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9519ed63ddfb2f64 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.240+0000 7f59d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:22:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59dc03e070 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f59dc0267b0 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59dc030420 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f59dc0269b0 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59fade0740 10 client.27206 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55b957b03610 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.241+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59dc030420 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f59dc038030 con 0x55b957a88e80 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 10 client.27206 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 20 client.27206 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957b03d00 0x55b957b06120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 4 client.27206 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:22:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9579efc00 con 0x55b957b03d00 2024-08-20T01:22:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59f5400640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957b03d00 0x55b957b06120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.242+0000 7f59f5400640 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957b03d00 0x55b957b06120 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1da865933ea83736 server_cookie=d377d7e174203516 in_seq=0 out_seq=0 2024-08-20T01:22:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.243+0000 7f59d6a00640 10 client.27206 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:45.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 20552 30.29 MB/sec warmup 42 sec latency 59.889 ms 2024-08-20T01:22:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.614+0000 7f59d6a00640 1 -- 172.21.15.138:0/3366516763 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9579efc00 con 0x55b957b03d00 2024-08-20T01:22:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.614+0000 7f59d6a00640 10 client.27206 handle_command_reply: tid=0 2024-08-20T01:22:45.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.614+0000 7f59fade0740 1 client.27206 shutdown 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 msgr2=0x55b957a8c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59dc011c00 tx=0x7f59dc01ad20 comp rx=0 tx=0).stop 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 shutdown_connections 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b957b03d00 0x55b957b06120 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b957affa90 0x55b957b01e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b957a88e80 0x55b957a8c720 secure :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f59dc011c00 tx=0x7f59dc01ad20 comp rx=0 tx=0).stop 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 --2- 172.21.15.138:0/3366516763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b957a7c5d0 0x55b957a8c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 >> 172.21.15.138:0/3366516763 conn(0x55b957a7a3b0 msgr2=0x55b957a845c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59d6a00640 0 client.27206 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 shutdown_connections 2024-08-20T01:22:45.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.615+0000 7f59fade0740 1 -- 172.21.15.138:0/3366516763 wait complete. 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.616+0000 7f59fade0740 20 client.27206 trim_cache size 0 max 16384 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:22:45.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.617+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 msgr2=0x7f59d80b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.617+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 0x7f59d80b6300 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59dc011fd0 tx=0x7f59dc00b8f0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.617+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 msgr2=0x7f59f019c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.617+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 0x7f59f019c830 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f59e800da10 tx=0x7f59e800ded0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 shutdown_connections 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f59d80b3e40 0x7f59d80b6300 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59f010f0e0 0x7f59f019c830 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59f0101e50 0x7f59f0197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 --2- 172.21.15.138:0/2339374784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f59f0101530 0x7f59f0197250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:45.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 >> 172.21.15.138:0/2339374784 conn(0x7f59f0078620 msgr2=0x7f59f0111320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.618+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 shutdown_connections 2024-08-20T01:22:45.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:45.619+0000 7f59f7e00640 1 -- 172.21.15.138:0/2339374784 wait complete. 2024-08-20T01:22:46.253 INFO:tasks.workunit.client.0.smithi138.stdout: 10 21376 30.84 MB/sec warmup 43 sec latency 180.366 ms 2024-08-20T01:22:46.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:46 smithi138 ceph-mon[18076]: pgmap v818: 129 pgs: 129 active+clean; 776 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-08-20T01:22:46.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:46 smithi167 ceph-mon[21965]: pgmap v818: 129 pgs: 129 active+clean; 776 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-08-20T01:22:46.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:46 smithi142 ceph-mon[21799]: pgmap v818: 129 pgs: 129 active+clean; 776 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 387 op/s 2024-08-20T01:22:47.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.046+0000 7f4dbfe00640 1 -- 172.21.15.138:0/1742658770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db8111620 msgr2=0x7f4db8117e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:47.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.046+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/1742658770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db8111620 0x7f4db8117e20 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009ca0 tx=0x7f4db002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 -- 172.21.15.138:0/1742658770 shutdown_connections 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/1742658770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db8111620 0x7f4db8117e20 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/1742658770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8110ca0 0x7f4db81110e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/1742658770 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db810b2c0 0x7f4db810b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 -- 172.21.15.138:0/1742658770 >> 172.21.15.138:0/1742658770 conn(0x7f4db8072270 msgr2=0x7f4db8074690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 -- 172.21.15.138:0/1742658770 shutdown_connections 2024-08-20T01:22:47.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.048+0000 7f4dbfe00640 1 -- 172.21.15.138:0/1742658770 wait complete. 2024-08-20T01:22:47.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.049+0000 7f4dbfe00640 1 Processor -- start 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.049+0000 7f4dbfe00640 1 -- start start 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 0x7f4db806e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db8110ca0 0x7f4db806c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811b2a0 con 0x7f4db810b2c0 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811b120 con 0x7f4db8111620 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbfe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4db811b420 con 0x7f4db8110ca0 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbde00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:42522/0 (socket says 172.21.15.138:42522) 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbde00640 1 -- 172.21.15.138:0/3872720040 learned_addr learned my addr 172.21.15.138:0/3872720040 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbd400640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 0x7f4db806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:47.051 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.050+0000 7f4dbca00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db8110ca0 0x7f4db806c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:47.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.051+0000 7f4dbde00640 1 -- 172.21.15.138:0/3872720040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db8110ca0 msgr2=0x7f4db806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:47.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.051+0000 7f4dbde00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db8110ca0 0x7f4db806c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.051+0000 7f4dbde00640 1 -- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 msgr2=0x7f4db806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:47.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.051+0000 7f4dbde00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 0x7f4db806e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.052+0000 7f4dbde00640 1 -- 172.21.15.138:0/3872720040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db806d4b0 con 0x7f4db8111620 2024-08-20T01:22:47.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.052+0000 7f4dbd400640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 0x7f4db806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:22:47.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.053+0000 7f4dbde00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009430 tx=0x7f4db00317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:47.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.053+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db0003b70 con 0x7f4db8111620 2024-08-20T01:22:47.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.053+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db81a7a30 con 0x7f4db8111620 2024-08-20T01:22:47.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.054+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db81a7fc0 con 0x7f4db8111620 2024-08-20T01:22:47.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.054+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db0003d10 con 0x7f4db8111620 2024-08-20T01:22:47.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.055+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db002fa00 con 0x7f4db8111620 2024-08-20T01:22:47.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.057+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4db810b6a0 con 0x7f4db8111620 2024-08-20T01:22:47.058 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.058+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4db002fc40 con 0x7f4db8111620 2024-08-20T01:22:47.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.060+0000 7f4db6000640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 0x7f4da00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:47.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.061+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f4db01057a0 con 0x7f4db8111620 2024-08-20T01:22:47.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.061+0000 7f4dbd400640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 0x7f4da00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:47.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.062+0000 7f4dbd400640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 0x7f4da00b6300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009c30 tx=0x7f4dac005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:47.067 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.067+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4db00cd5f0 con 0x7f4db8111620 2024-08-20T01:22:47.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 22378 31.34 MB/sec warmup 44 sec latency 133.405 ms 2024-08-20T01:22:47.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.461+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4db8113400 con 0x7f4db8111620 2024-08-20T01:22:47.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.462+0000 7f4db6000640 1 -- 172.21.15.138:0/3872720040 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11939 (secure 0 0 0) 0x7f4db00ccef0 con 0x7f4db8111620 2024-08-20T01:22:47.463 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:22:47.463 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":48,"btime":"2024-08-20T01:22:21:451368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:22:21.168170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:22:47.463 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 48 2024-08-20T01:22:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 msgr2=0x7f4da00b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 0x7f4da00b6300 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009c30 tx=0x7f4dac005f90 comp rx=0 tx=0).stop 2024-08-20T01:22:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 msgr2=0x7f4db806cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4db0009430 tx=0x7f4db00317c0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 shutdown_connections 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4da00b3e40 0x7f4da00b6300 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4db8111620 0x7f4db806cbf0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4db8110ca0 0x7f4db806c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 --2- 172.21.15.138:0/3872720040 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4db810b2c0 0x7f4db806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 >> 172.21.15.138:0/3872720040 conn(0x7f4db8072270 msgr2=0x7f4db8074660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.465+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 shutdown_connections 2024-08-20T01:22:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:47.466+0000 7f4dbfe00640 1 -- 172.21.15.138:0/3872720040 wait complete. 2024-08-20T01:22:47.677 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3872720040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:47.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3872720040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:47.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3872720040' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:22:48.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 23140 31.57 MB/sec warmup 45 sec latency 211.224 ms 2024-08-20T01:22:48.469 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0' 2024-08-20T01:22:48.469 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 140.0 --await-for 45.0 2024-08-20T01:22:48.766 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:48 smithi138 ceph-mon[18076]: pgmap v819: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-08-20T01:22:48.864 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:48 smithi167 ceph-mon[21965]: pgmap v819: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-08-20T01:22:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:48 smithi142 ceph-mon[21799]: pgmap v819: 129 pgs: 129 active+clean; 788 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.2 MiB/s wr, 464 op/s 2024-08-20T01:22:49.125 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:22:49.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 24019 31.78 MB/sec warmup 46 sec latency 60.161 ms 2024-08-20T01:22:50.254 INFO:tasks.workunit.client.0.smithi138.stdout: 10 24959 32.45 MB/sec warmup 47 sec latency 54.665 ms 2024-08-20T01:22:50.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 -- 172.21.15.138:0/1409802653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d40751f0 msgr2=0x7f02d40755d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 --2- 172.21.15.138:0/1409802653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d40751f0 0x7f02d40755d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009ca0 tx=0x7f02cc02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.483 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 -- 172.21.15.138:0/1409802653 shutdown_connections 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 --2- 172.21.15.138:0/1409802653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d410f0c0 0x7f02d41114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 --2- 172.21.15.138:0/1409802653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 --2- 172.21.15.138:0/1409802653 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d40751f0 0x7f02d40755d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 -- 172.21.15.138:0/1409802653 >> 172.21.15.138:0/1409802653 conn(0x7f02d40fe500 msgr2=0x7f02d4100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 -- 172.21.15.138:0/1409802653 shutdown_connections 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.482+0000 7f02d9a00640 1 -- 172.21.15.138:0/1409802653 wait complete. 2024-08-20T01:22:50.484 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.483+0000 7f02d9a00640 1 Processor -- start 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.483+0000 7f02d9a00640 1 -- start start 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40751f0 0x7f02d419b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d419b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 0x7f02d41a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4114cf0 con 0x7f02d4075b10 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4114b70 con 0x7f02d40751f0 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d9a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4114e70 con 0x7f02d410f0c0 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d419b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 0x7f02d41a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.485 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40751f0 0x7f02d419b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 0x7f02d41a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44504/0 (socket says 172.21.15.138:44504) 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.484+0000 7f02d3400640 1 -- 172.21.15.138:0/730717791 learned_addr learned my addr 172.21.15.138:0/730717791 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 -- 172.21.15.138:0/730717791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 msgr2=0x7f02d41a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 0x7f02d41a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 -- 172.21.15.138:0/730717791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40751f0 msgr2=0x7f02d419b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40751f0 0x7f02d419b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 -- 172.21.15.138:0/730717791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d41a1230 con 0x7f02d4075b10 2024-08-20T01:22:50.486 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.485+0000 7f02d2000640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d419b6c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c970 tx=0x7f02bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.486+0000 7f02cb400640 1 -- 172.21.15.138:0/730717791 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc007bb0 con 0x7f02d4075b10 2024-08-20T01:22:50.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.486+0000 7f02cb400640 1 -- 172.21.15.138:0/730717791 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02bc007d50 con 0x7f02d4075b10 2024-08-20T01:22:50.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.486+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d41a1520 con 0x7f02d4075b10 2024-08-20T01:22:50.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.486+0000 7f02cb400640 1 -- 172.21.15.138:0/730717791 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00e710 con 0x7f02d4075b10 2024-08-20T01:22:50.487 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.486+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d41a7be0 con 0x7f02d4075b10 2024-08-20T01:22:50.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.490+0000 7f02dc491740 1 Processor -- start 2024-08-20T01:22:50.491 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 -- start start 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x5565604037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 0x55656040f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 0x556560412090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5565603a3670 con 0x556560403d10 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5565603777c0 con 0x55656040fca0 2024-08-20T01:22:50.492 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.491+0000 7f02dc491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556560366cc0 con 0x5565604033f0 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.492+0000 7f02d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x5565604037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.492+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 0x55656040f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.492+0000 7f02d3400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x5565604037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44518/0 (socket says 172.21.15.138:44518) 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.492+0000 7f02d3400640 1 -- 172.21.15.138:0/703818392 learned_addr learned my addr 172.21.15.138:0/703818392 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.493+0000 7f02d2000640 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 0x556560412090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.493+0000 7f02cb400640 1 -- 172.21.15.138:0/730717791 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f02bc00fa90 con 0x7f02d4075b10 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2a00640 1 -- 172.21.15.138:0/703818392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 msgr2=0x5565604037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.497 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2a00640 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x5565604037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2a00640 1 -- 172.21.15.138:0/703818392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 msgr2=0x556560412090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2a00640 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 0x556560412090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2a00640 1 -- 172.21.15.138:0/703818392 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565602fa110 con 0x556560403d10 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.495+0000 7f02d2000640 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 0x556560412090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.496+0000 7f02d2a00640 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 0x55656040f760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006010 tx=0x7f02cc0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9581bfba9e8883 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.496+0000 7f02cb400640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 0x7f02b80b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.497+0000 7f02d3400640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 0x7f02b80b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.497+0000 7f02c3e00640 1 -- 172.21.15.138:0/703818392 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc031450 con 0x556560403d10 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.497+0000 7f02cb400640 1 -- 172.21.15.138:0/730717791 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f02bc0e45a0 con 0x7f02d4075b10 2024-08-20T01:22:50.498 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.497+0000 7f02c3e00640 1 -- 172.21.15.138:0/703818392 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc036040 con 0x556560403d10 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 -- 172.21.15.138:0/703818392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 msgr2=0x55656040f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 0x55656040f760 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006010 tx=0x7f02cc0099c0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02d3400640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 0x7f02b80b6280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c400b580 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 -- 172.21.15.138:0/703818392 shutdown_connections 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040fca0 0x556560412090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.501 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556560403d10 0x55656040f760 secure :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006010 tx=0x7f02cc0099c0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 --2- 172.21.15.138:0/703818392 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x5565604037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 -- 172.21.15.138:0/703818392 >> 172.21.15.138:0/703818392 conn(0x556560401220 msgr2=0x5565604023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 -- 172.21.15.138:0/703818392 shutdown_connections 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.498+0000 7f02dc491740 1 -- 172.21.15.138:0/703818392 wait complete. 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.499+0000 7f02dc491740 1 -- start start 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.499+0000 7f02dc491740 1 Processor -- start 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.499+0000 7f02dc491740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x55656040cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x55656040d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55656030c780 con 0x55656040d7b0 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55656036ad00 con 0x5565604033f0 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5565601c97a0 con 0x55656040fca0 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02dc491740 10 client.0 _ll_register_callbacks cb 0x556560314940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:50.502 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02d2000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x55656040d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02d3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x55656040cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02d3400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57204/0 (socket says 172.21.15.138:57204) 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.500+0000 7f02d3400640 1 -- 172.21.15.138:0/1260861171 learned_addr learned my addr 172.21.15.138:0/1260861171 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 -- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 msgr2=0x55656040cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x55656040cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 -- 172.21.15.138:0/1260861171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 msgr2=0x55656041bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.503 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 -- 172.21.15.138:0/1260861171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565603199c0 con 0x55656040fca0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d3400640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02d2000640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x55656040d270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0e18f0 tx=0x7f02bc0aac90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7da8a1719d5f95b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0ab4b0 con 0x55656040fca0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.501+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc036040 con 0x55656040fca0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f02bc0a73a0 con 0x55656040fca0 2024-08-20T01:22:50.504 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02bc0aadb0 con 0x55656040fca0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 10 client.27190 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5565600d5b90 con 0x55656040fca0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc036040 con 0x55656040fca0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f02bc020070 con 0x55656040fca0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 10 client.27190 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 10 client.27190 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 20 client.27190 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5565604282f0 0x55656042a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 4 client.27190 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.502+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556560379710 con 0x5565604282f0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.503+0000 7f02d2a00640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5565604282f0 0x55656042a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.503+0000 7f02d2a00640 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5565604282f0 0x55656042a6e0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88eb538fffcb2f9d server_cookie=4bf9b64abe782590 in_seq=0 out_seq=0 2024-08-20T01:22:50.505 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.503+0000 7f02c0c00640 10 client.27190 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02c0c00640 1 -- 172.21.15.138:0/1260861171 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556560379710 con 0x5565604282f0 2024-08-20T01:22:50.506 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02c0c00640 10 client.27190 handle_command_reply: tid=0 2024-08-20T01:22:50.508 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 client.27190 shutdown 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 msgr2=0x55656040d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x55656040d270 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0e18f0 tx=0x7f02bc0aac90 comp rx=0 tx=0).stop 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 shutdown_connections 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5565604282f0 0x55656042a6e0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x55656041bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x55656040d270 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 --2- 172.21.15.138:0/1260861171 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x55656040cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 >> 172.21.15.138:0/1260861171 conn(0x556560401220 msgr2=0x556560406dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.506+0000 7f02c0c00640 0 client.27190 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.507+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 shutdown_connections 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.507+0000 7f02dc491740 1 -- 172.21.15.138:0/1260861171 wait complete. 2024-08-20T01:22:50.509 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.507+0000 7f02dc491740 20 client.27190 trim_cache size 0 max 16384 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.537+0000 7f02dc491740 1 Processor -- start 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 -- start start 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040d7b0 0x55656040cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040fca0 0x55656041b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41b0340 con 0x55656040fca0 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41adb80 con 0x55656040d7b0 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02dc491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ab9f0 con 0x5565604033f0 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:44542/0 (socket says 172.21.15.138:44542) 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02d2a00640 1 -- 172.21.15.138:0/1671144672 learned_addr learned my addr 172.21.15.138:0/1671144672 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.538+0000 7f02d2000640 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040d7b0 0x55656040cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.539 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d3400640 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040fca0 0x55656041b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 -- 172.21.15.138:0/1671144672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040d7b0 msgr2=0x55656040cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040d7b0 0x55656040cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 -- 172.21.15.138:0/1671144672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040fca0 msgr2=0x55656041b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040fca0 0x55656041b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 -- 172.21.15.138:0/1671144672 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556560403b40 con 0x5565604033f0 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.539+0000 7f02d2a00640 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02ade0 tx=0x7f02cc005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc51b7ade0a0845e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.540+0000 7f02c3400640 1 -- 172.21.15.138:0/1671144672 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc04a040 con 0x5565604033f0 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.540+0000 7f02c3400640 1 -- 172.21.15.138:0/1671144672 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc055070 con 0x5565604033f0 2024-08-20T01:22:50.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.540+0000 7f02c3400640 1 -- 172.21.15.138:0/1671144672 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc02f610 con 0x5565604033f0 2024-08-20T01:22:50.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.541+0000 7f02dc491740 1 -- 172.21.15.138:0/1671144672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 msgr2=0x55656040e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:50.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.541+0000 7f02dc491740 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02ade0 tx=0x7f02cc005f80 comp rx=0 tx=0).stop 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.542+0000 7f02dc491740 1 -- 172.21.15.138:0/1671144672 shutdown_connections 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.542+0000 7f02dc491740 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040fca0 0x55656041b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.542+0000 7f02dc491740 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55656040d7b0 0x55656040cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.542+0000 7f02dc491740 1 --2- 172.21.15.138:0/1671144672 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5565604033f0 0x55656040e9d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.542+0000 7f02dc491740 1 -- 172.21.15.138:0/1671144672 >> 172.21.15.138:0/1671144672 conn(0x556560401220 msgr2=0x5565604003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.543+0000 7f02dc491740 1 -- 172.21.15.138:0/1671144672 shutdown_connections 2024-08-20T01:22:50.543 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.543+0000 7f02dc491740 1 -- 172.21.15.138:0/1671144672 wait complete. 2024-08-20T01:22:50.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.543+0000 7f02dc491740 1 -- start start 2024-08-20T01:22:50.544 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.543+0000 7f02dc491740 1 Processor -- start 2024-08-20T01:22:50.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.546+0000 7f02dc491740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x556560418070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.547 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.546+0000 7f02dc491740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02dc491740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x556560413a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02dc491740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ac040 con 0x55656040d7b0 2024-08-20T01:22:50.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02dc491740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ad4b0 con 0x5565604033f0 2024-08-20T01:22:50.549 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02dc491740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ac710 con 0x55656040fca0 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02dc491740 10 client.0 _ll_register_callbacks cb 0x7f02bc0e1da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02d2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57224/0 (socket says 172.21.15.138:57224) 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02d2000640 1 -- 172.21.15.138:0/20217654 learned_addr learned my addr 172.21.15.138:0/20217654 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02d3400640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x556560413a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.548+0000 7f02d3400640 1 -- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 msgr2=0x556560418070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.547+0000 7f02d2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x556560418070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.550 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.549+0000 7f02d3400640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x556560418070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.549+0000 7f02d3400640 1 -- 172.21.15.138:0/20217654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 msgr2=0x556560413540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.549+0000 7f02d3400640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.549+0000 7f02d3400640 1 -- 172.21.15.138:0/20217654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556560414170 con 0x55656040fca0 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02d2000640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02d2a00640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x556560418070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02d3400640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x556560413a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012cb0 tx=0x7f02c40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c43d391d8deff4cf server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:50.551 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02c0c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c401dad0 con 0x55656040fca0 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.550+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f02c401dc70 con 0x55656040fca0 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.551+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02cc055070 con 0x55656040fca0 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.551+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f02c401de70 con 0x55656040fca0 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.552+0000 7f02dc491740 10 client.27196 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:50.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.552+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x556560492ee0 con 0x55656040fca0 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02cc055070 con 0x55656040fca0 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f02c40104e0 con 0x55656040fca0 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 10 client.27196 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 10 client.27196 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 20 client.27196 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556560493780 0x556560495ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 4 client.27196 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}] 2024-08-20T01:22:50.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.554+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.0, "await_for": 45.0}) -- 0x5565603751b0 con 0x556560493780 2024-08-20T01:22:50.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.555+0000 7f02d2a00640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556560493780 0x556560495ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.557+0000 7f02d2a00640 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556560493780 0x556560495ba0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdee03abd2b17771 server_cookie=673ebb29f1ea4a4a in_seq=0 out_seq=0 2024-08-20T01:22:50.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:22:50.557+0000 7f02c0c00640 10 client.27196 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:50.810 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:50 smithi138 ceph-mon[18076]: pgmap v820: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-08-20T01:22:50.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:50 smithi167 ceph-mon[21965]: pgmap v820: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-08-20T01:22:50.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:50 smithi142 ceph-mon[21799]: pgmap v820: 129 pgs: 129 active+clean; 789 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 408 op/s 2024-08-20T01:22:51.259 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25223 31.89 MB/sec warmup 48 sec latency 693.136 ms 2024-08-20T01:22:52.259 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25223 31.24 MB/sec warmup 49 sec latency 1693.207 ms 2024-08-20T01:22:52.743 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:52 smithi138 ceph-mon[18076]: pgmap v821: 129 pgs: 129 active+clean; 800 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-20T01:22:52.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:52 smithi167 ceph-mon[21965]: pgmap v821: 129 pgs: 129 active+clean; 800 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-20T01:22:52.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:52 smithi142 ceph-mon[21799]: pgmap v821: 129 pgs: 129 active+clean; 800 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 6.2 MiB/s wr, 597 op/s 2024-08-20T01:22:53.259 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25223 30.62 MB/sec warmup 50 sec latency 2693.268 ms 2024-08-20T01:22:54.259 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25225 30.02 MB/sec warmup 51 sec latency 3693.335 ms 2024-08-20T01:22:54.755 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:54 smithi138 ceph-mon[18076]: pgmap v822: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 617 op/s 2024-08-20T01:22:54.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:54 smithi167 ceph-mon[21965]: pgmap v822: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 617 op/s 2024-08-20T01:22:54.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:54 smithi142 ceph-mon[21799]: pgmap v822: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 7.1 MiB/s wr, 617 op/s 2024-08-20T01:22:55.259 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 29.44 MB/sec warmup 52 sec latency 4693.393 ms 2024-08-20T01:22:56.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 28.88 MB/sec warmup 53 sec latency 5695.392 ms 2024-08-20T01:22:56.760 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:56 smithi138 ceph-mon[18076]: pgmap v823: 129 pgs: 129 active+clean; 827 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 697 op/s 2024-08-20T01:22:56.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:56 smithi167 ceph-mon[21965]: pgmap v823: 129 pgs: 129 active+clean; 827 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 697 op/s 2024-08-20T01:22:56.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:56 smithi142 ceph-mon[21799]: pgmap v823: 129 pgs: 129 active+clean; 827 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 697 op/s 2024-08-20T01:22:57.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 28.35 MB/sec warmup 54 sec latency 6695.457 ms 2024-08-20T01:22:58.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 27.83 MB/sec warmup 55 sec latency 7695.512 ms 2024-08-20T01:22:58.733 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:58 smithi138 ceph-mon[18076]: pgmap v824: 129 pgs: 129 active+clean; 814 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.7 MiB/s wr, 647 op/s 2024-08-20T01:22:58.734 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:22:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- 172.21.15.138:0/1636437550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934006aa70 msgr2=0x7f934006aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- 172.21.15.138:0/1636437550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934006aa70 0x7f934006aeb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f932c009900 tx=0x7f932c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- 172.21.15.138:0/1636437550 shutdown_connections 2024-08-20T01:22:58.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- 172.21.15.138:0/1636437550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934010ef30 0x7f9340111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- 172.21.15.138:0/1636437550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f934006aa70 0x7f934006aeb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- 172.21.15.138:0/1636437550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93401033c0 0x7f934006a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- 172.21.15.138:0/1636437550 >> 172.21.15.138:0/1636437550 conn(0x7f9340077270 msgr2=0x7f9340077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- 172.21.15.138:0/1636437550 shutdown_connections 2024-08-20T01:22:58.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- 172.21.15.138:0/1636437550 wait complete. 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 Processor -- start 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- start start 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934006aa70 0x7f934019b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 0x7f934019b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114a40 con 0x7f934006aa70 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93401148c0 con 0x7f934010ef30 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.741+0000 7f9345200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9340114bc0 con 0x7f93401033c0 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 0x7f934019b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.746 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36868/0 (socket says 172.21.15.138:36868) 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 -- 172.21.15.138:0/60726097 learned_addr learned my addr 172.21.15.138:0/60726097 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 0x7f934019b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57436/0 (socket says 172.21.15.138:57436) 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933f400640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934006aa70 0x7f934019b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 -- 172.21.15.138:0/60726097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 msgr2=0x7f934019b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 0x7f934019b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 -- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934006aa70 msgr2=0x7f934019b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934006aa70 0x7f934019b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 -- 172.21.15.138:0/60726097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93401a1100 con 0x7f934010ef30 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933fe00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f933800d950 tx=0x7f933800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9337e00640 1 -- 172.21.15.138:0/60726097 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338017070 con 0x7f934010ef30 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9337e00640 1 -- 172.21.15.138:0/60726097 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9338009c30 con 0x7f934010ef30 2024-08-20T01:22:58.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93401a13f0 con 0x7f934010ef30 2024-08-20T01:22:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9337e00640 1 -- 172.21.15.138:0/60726097 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338003c10 con 0x7f934010ef30 2024-08-20T01:22:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93401a7ab0 con 0x7f934010ef30 2024-08-20T01:22:58.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 Processor -- start 2024-08-20T01:22:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9337e00640 1 -- 172.21.15.138:0/60726097 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9338003db0 con 0x7f934010ef30 2024-08-20T01:22:58.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 -- start start 2024-08-20T01:22:58.751 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f29aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f2a73c0 0x55c70f2a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c70f210210 con 0x55c70f2a73c0 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c70f1ff710 con 0x55c70f29ab10 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f9349872740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c70f203750 con 0x55c70f29b430 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57448/0 (socket says 172.21.15.138:57448) 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933f400640 1 -- 172.21.15.138:0/1114118349 learned_addr learned my addr 172.21.15.138:0/1114118349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.746+0000 7f933ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f29aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933ea00640 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f2a73c0 0x55c70f2a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 -- 172.21.15.138:0/1114118349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 msgr2=0x55c70f29aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f29aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 -- 172.21.15.138:0/1114118349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f2a73c0 msgr2=0x55c70f2a97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f2a73c0 0x55c70f2a97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 -- 172.21.15.138:0/1114118349 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70f1cbde0 con 0x55c70f29b430 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93300114f0 tx=0x7f93300119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae0e7d07ba469a5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9337e00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 0x7f93240b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9334c00640 1 -- 172.21.15.138:0/1114118349 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933001e070 con 0x55c70f29b430 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9334c00640 1 -- 172.21.15.138:0/1114118349 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9330003c30 con 0x55c70f29b430 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9334c00640 1 -- 172.21.15.138:0/1114118349 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933000f9b0 con 0x55c70f29b430 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9337e00640 1 -- 172.21.15.138:0/60726097 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f93380e3c00 con 0x7f934010ef30 2024-08-20T01:22:58.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933ea00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 0x7f93240b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.756 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/1114118349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 msgr2=0x55c70f2a6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f93300114f0 tx=0x7f93300119b0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/1114118349 shutdown_connections 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f2a73c0 0x55c70f2a97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29b430 0x55c70f2a6e80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- 172.21.15.138:0/1114118349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f29aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/1114118349 >> 172.21.15.138:0/1114118349 conn(0x55c70f298940 msgr2=0x55c70f299ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/1114118349 shutdown_connections 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/1114118349 wait complete. 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933ea00640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 0x7f93240b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f932c005ab0 tx=0x7f932c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- start start 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 Processor -- start 2024-08-20T01:22:58.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f2a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 0x55c70f2a44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c70f138ab0 con 0x55c70f29b430 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c70f1a6140 con 0x55c70f29ab10 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c70f2aa0a0 con 0x55c70f2a73c0 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 10 client.0 _ll_register_callbacks cb 0x55c70f1b95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36878/0 (socket says 172.21.15.138:36878) 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 -- 172.21.15.138:0/556403513 learned_addr learned my addr 172.21.15.138:0/556403513 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 0x55c70f2a44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933ea00640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f2a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 -- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 msgr2=0x55c70f2a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 -- 172.21.15.138:0/556403513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 msgr2=0x55c70f2a3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f2a3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 -- 172.21.15.138:0/556403513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70f1a7170 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933fe00640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 0x55c70f2a44a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f93380e0f50 tx=0x7f93380a9cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95cdac5a8ca886dd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9338020040 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9330003c30 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f93380a4df0 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f93380aea10 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 10 client.27208 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c70ef62790 con 0x55c70f2a73c0 2024-08-20T01:22:58.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9330003c30 con 0x55c70f2a73c0 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f93380bfab0 con 0x55c70f2a73c0 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 10 client.27208 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 10 client.27208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 20 client.27208 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2b6f80 0x55c70f2b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 4 client.27208 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c70f212160 con 0x55c70f2b6f80 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.752+0000 7f933f400640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2b6f80 0x55c70f2b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f933f400640 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2b6f80 0x55c70f2b9370 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df77b6c05b22cadb server_cookie=c4001bf9b65c1c28 in_seq=0 out_seq=0 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9329600640 10 client.27208 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9329600640 1 -- 172.21.15.138:0/556403513 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c70f212160 con 0x55c70f2b6f80 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9329600640 10 client.27208 handle_command_reply: tid=0 2024-08-20T01:22:58.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 client.27208 shutdown 2024-08-20T01:22:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 msgr2=0x55c70f2a44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 0x55c70f2a44a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f93380e0f50 tx=0x7f93380a9cf0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 shutdown_connections 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2b6f80 0x55c70f2b9370 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f2a73c0 0x55c70f2a44a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9329600640 0 client.27208 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f2a3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 --2- 172.21.15.138:0/556403513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29ab10 0x55c70f2a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 >> 172.21.15.138:0/556403513 conn(0x55c70f298940 msgr2=0x55c70f29e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 shutdown_connections 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 1 -- 172.21.15.138:0/556403513 wait complete. 2024-08-20T01:22:58.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.757+0000 7f9349872740 20 client.27208 trim_cache size 0 max 16384 2024-08-20T01:22:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 Processor -- start 2024-08-20T01:22:58.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 -- start start 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 0x55c70f2bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 0x55c70f2a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93401b4600 con 0x55c70f29ab10 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93401b1e40 con 0x55c70f29b430 2024-08-20T01:22:58.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f9349872740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93401afcb0 con 0x55c70f298940 2024-08-20T01:22:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:48850/0 (socket says 172.21.15.138:48850) 2024-08-20T01:22:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 -- 172.21.15.138:0/2576127214 learned_addr learned my addr 172.21.15.138:0/2576127214 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:58.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933f400640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 0x55c70f2bebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933fe00640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 0x55c70f2a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 -- 172.21.15.138:0/2576127214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 msgr2=0x55c70f2bebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 0x55c70f2bebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 -- 172.21.15.138:0/2576127214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 msgr2=0x55c70f2a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 0x55c70f2a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 -- 172.21.15.138:0/2576127214 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c70f2b96e0 con 0x55c70f29ab10 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933f400640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 0x55c70f2bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933fe00640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 0x55c70f2a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f933ea00640 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f932c009230 tx=0x7f932c03f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bce2d444af1896 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f932be00640 1 -- 172.21.15.138:0/2576127214 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0314a0 con 0x55c70f29ab10 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f932be00640 1 -- 172.21.15.138:0/2576127214 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c031af0 con 0x55c70f29ab10 2024-08-20T01:22:58.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.775+0000 7f932be00640 1 -- 172.21.15.138:0/2576127214 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c03da90 con 0x55c70f29ab10 2024-08-20T01:22:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- 172.21.15.138:0/2576127214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 msgr2=0x55c70f2bf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:58.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f932c009230 tx=0x7f932c03f040 comp rx=0 tx=0).stop 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- 172.21.15.138:0/2576127214 shutdown_connections 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f29b430 0x55c70f2a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29ab10 0x55c70f2bf130 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- 172.21.15.138:0/2576127214 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f298940 0x55c70f2bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- 172.21.15.138:0/2576127214 >> 172.21.15.138:0/2576127214 conn(0x55c70f2a6c80 msgr2=0x55c70f2a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- 172.21.15.138:0/2576127214 shutdown_connections 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- 172.21.15.138:0/2576127214 wait complete. 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 -- start start 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 Processor -- start 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 0x55c70f2ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.780+0000 7f9349872740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f319c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 0x55c70f2ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36886/0 (socket says 172.21.15.138:36886) 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 0x55c70f2ac750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:57460/0 (socket says 172.21.15.138:57460) 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 -- 172.21.15.138:0/2230489626 learned_addr learned my addr 172.21.15.138:0/2230489626 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f93401b0300 con 0x55c70f29b430 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f93401b1770 con 0x55c70f298940 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f93401b09d0 con 0x55c70f29ab10 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 10 client.0 _ll_register_callbacks cb 0x7f933800b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933fe00640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f319c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 -- 172.21.15.138:0/2230489626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 msgr2=0x55c70f2ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 0x55c70f2ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 -- 172.21.15.138:0/2230489626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 msgr2=0x55c70f319c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f319c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70efdb620 con 0x55c70f298940 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933f400640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f933000f610 tx=0x7f9330009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc61036618cf165 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933001e040 con 0x55c70f298940 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f933000f840 con 0x55c70f298940 2024-08-20T01:22:58.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f932c031af0 con 0x55c70f298940 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f9330039050 con 0x55c70f298940 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 10 client.27242 fetch_fsmap learned FSMap version 48 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c70f321630 con 0x55c70f298940 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f932c031af0 con 0x55c70f298940 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f93300043d0 con 0x55c70f298940 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 10 client.27242 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:22:58.785 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 10 client.27242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 20 client.27242 populate_metadata read hostname 'smithi138' 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2a73c0 0x55c70f323c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 4 client.27242 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c70f20dc00 con 0x55c70f2a73c0 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933ea00640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2a73c0 0x55c70f323c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f933ea00640 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2a73c0 0x55c70f323c90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0bbea67df6aad3 server_cookie=fded8e87769e6045 in_seq=0 out_seq=0 2024-08-20T01:22:58.786 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:58.781+0000 7f9329600640 10 client.27242 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:58.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:58 smithi167 ceph-mon[21965]: pgmap v824: 129 pgs: 129 active+clean; 814 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.7 MiB/s wr, 647 op/s 2024-08-20T01:22:58.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:22:58.991 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:58 smithi142 ceph-mon[21799]: pgmap v824: 129 pgs: 129 active+clean; 814 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 9.7 MiB/s wr, 647 op/s 2024-08-20T01:22:58.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:22:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.018+0000 7f9329600640 1 -- 172.21.15.138:0/2230489626 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c70f20dc00 con 0x55c70f2a73c0 2024-08-20T01:22:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.018+0000 7f9329600640 10 client.27242 handle_command_reply: tid=0 2024-08-20T01:22:59.018 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.018+0000 7f9349872740 1 client.27242 shutdown 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 msgr2=0x55c70f2ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f933000f610 tx=0x7f9330009b40 comp rx=0 tx=0).stop 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 shutdown_connections 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70f2a73c0 0x55c70f323c90 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c70f29b430 0x55c70f319c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c70f29ab10 0x55c70f2ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9329600640 0 client.27242 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 --2- 172.21.15.138:0/2230489626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c70f298940 0x55c70f2ac1b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 >> 172.21.15.138:0/2230489626 conn(0x55c70f2a6c80 msgr2=0x55c70f2a5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 shutdown_connections 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 1 -- 172.21.15.138:0/2230489626 wait complete. 2024-08-20T01:22:59.019 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9349872740 20 client.27242 trim_cache size 0 max 16384 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 msgr2=0x7f93240b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 0x7f93240b6200 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f932c005ab0 tx=0x7f932c005a20 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 msgr2=0x7f93401a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f933800d950 tx=0x7f933800de10 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 shutdown_connections 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f93240b3d40 0x7f93240b6200 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f934010ef30 0x7f93401a0930 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f93401033c0 0x7f934019b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 --2- 172.21.15.138:0/60726097 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f934006aa70 0x7f934019b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 >> 172.21.15.138:0/60726097 conn(0x7f9340077270 msgr2=0x7f93401008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:22:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 shutdown_connections 2024-08-20T01:22:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:22:59.019+0000 7f9345200640 1 -- 172.21.15.138:0/60726097 wait complete. 2024-08-20T01:22:59.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 27.34 MB/sec warmup 56 sec latency 8695.570 ms 2024-08-20T01:22:59.872 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:22:59.873 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:22:59.873 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:22:59 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:22:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:22:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:22:59.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:22:59 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:22:59.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:22:59.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:22:59.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:22:59 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:23:00.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 26.86 MB/sec warmup 57 sec latency 9695.623 ms 2024-08-20T01:23:00.746 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:00 smithi138 ceph-mon[18076]: pgmap v825: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.1 MiB/s wr, 524 op/s 2024-08-20T01:23:00.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:00 smithi167 ceph-mon[21965]: pgmap v825: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.1 MiB/s wr, 524 op/s 2024-08-20T01:23:00.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:00 smithi142 ceph-mon[21799]: pgmap v825: 129 pgs: 129 active+clean; 819 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 9.1 MiB/s wr, 524 op/s 2024-08-20T01:23:01.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 26.39 MB/sec warmup 58 sec latency 10695.673 ms 2024-08-20T01:23:02.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 25.95 MB/sec warmup 59 sec latency 11695.722 ms 2024-08-20T01:23:02.762 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:02 smithi138 ceph-mon[18076]: pgmap v826: 129 pgs: 129 active+clean; 874 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-08-20T01:23:02.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:02 smithi167 ceph-mon[21965]: pgmap v826: 129 pgs: 129 active+clean; 874 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-08-20T01:23:02.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:02 smithi142 ceph-mon[21799]: pgmap v826: 129 pgs: 129 active+clean; 874 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 14 MiB/s wr, 676 op/s 2024-08-20T01:23:03.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 25.51 MB/sec warmup 60 sec latency 12695.787 ms 2024-08-20T01:23:04.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 25.10 MB/sec warmup 61 sec latency 13695.836 ms 2024-08-20T01:23:04.728 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:04 smithi138 ceph-mon[18076]: pgmap v827: 129 pgs: 129 active+clean; 881 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 533 op/s 2024-08-20T01:23:04.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:04 smithi167 ceph-mon[21965]: pgmap v827: 129 pgs: 129 active+clean; 881 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 533 op/s 2024-08-20T01:23:04.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:04 smithi142 ceph-mon[21799]: pgmap v827: 129 pgs: 129 active+clean; 881 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 13 MiB/s wr, 533 op/s 2024-08-20T01:23:05.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 24.69 MB/sec warmup 62 sec latency 14695.883 ms 2024-08-20T01:23:06.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 24.30 MB/sec warmup 63 sec latency 15695.936 ms 2024-08-20T01:23:06.755 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:06 smithi138 ceph-mon[18076]: pgmap v828: 129 pgs: 129 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 12 MiB/s wr, 484 op/s 2024-08-20T01:23:06.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:06 smithi167 ceph-mon[21965]: pgmap v828: 129 pgs: 129 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 12 MiB/s wr, 484 op/s 2024-08-20T01:23:06.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:06 smithi142 ceph-mon[21799]: pgmap v828: 129 pgs: 129 active+clean; 869 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 12 MiB/s wr, 484 op/s 2024-08-20T01:23:07.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 23.92 MB/sec warmup 64 sec latency 16695.994 ms 2024-08-20T01:23:08.261 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 23.55 MB/sec warmup 65 sec latency 17696.050 ms 2024-08-20T01:23:08.777 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:08 smithi138 ceph-mon[18076]: pgmap v829: 129 pgs: 129 active+clean; 847 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 9.9 MiB/s wr, 406 op/s 2024-08-20T01:23:08.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:08 smithi167 ceph-mon[21965]: pgmap v829: 129 pgs: 129 active+clean; 847 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 9.9 MiB/s wr, 406 op/s 2024-08-20T01:23:08.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:08 smithi142 ceph-mon[21799]: pgmap v829: 129 pgs: 129 active+clean; 847 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 9.9 MiB/s wr, 406 op/s 2024-08-20T01:23:09.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 23.20 MB/sec warmup 66 sec latency 18696.098 ms 2024-08-20T01:23:10.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 22.85 MB/sec warmup 67 sec latency 19696.143 ms 2024-08-20T01:23:10.739 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:10 smithi138 ceph-mon[18076]: pgmap v830: 129 pgs: 129 active+clean; 841 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.4 MiB/s wr, 314 op/s 2024-08-20T01:23:10.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:10 smithi167 ceph-mon[21965]: pgmap v830: 129 pgs: 129 active+clean; 841 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.4 MiB/s wr, 314 op/s 2024-08-20T01:23:10.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:10 smithi142 ceph-mon[21799]: pgmap v830: 129 pgs: 129 active+clean; 841 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.4 MiB/s wr, 314 op/s 2024-08-20T01:23:11.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 22.51 MB/sec warmup 68 sec latency 20696.188 ms 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- 172.21.15.138:0/1327211880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 msgr2=0x7ff110111790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- 172.21.15.138:0/1327211880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff110111790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff108009ca0 tx=0x7ff10802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- 172.21.15.138:0/1327211880 shutdown_connections 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- 172.21.15.138:0/1327211880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff110111790 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- 172.21.15.138:0/1327211880 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff11010a610 0x7ff11010aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- 172.21.15.138:0/1327211880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff110104c10 0x7ff110104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- 172.21.15.138:0/1327211880 >> 172.21.15.138:0/1327211880 conn(0x7ff110100610 msgr2=0x7ff110102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- 172.21.15.138:0/1327211880 shutdown_connections 2024-08-20T01:23:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- 172.21.15.138:0/1327211880 wait complete. 2024-08-20T01:23:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 Processor -- start 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- start start 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110104c10 0x7ff11019b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff11010a610 0x7ff11019b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114c70 con 0x7ff11010af90 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114af0 con 0x7ff11010a610 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff117800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff110114df0 con 0x7ff110104c10 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff114e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110104c10 0x7ff11019b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40158/0 (socket says 172.21.15.138:40158) 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 -- 172.21.15.138:0/3083394355 learned_addr learned my addr 172.21.15.138:0/3083394355 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff10fe00640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff11010a610 0x7ff11019b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 -- 172.21.15.138:0/3083394355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110104c10 msgr2=0x7ff11019b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110104c10 0x7ff11019b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 -- 172.21.15.138:0/3083394355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff11010a610 msgr2=0x7ff11019b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff11010a610 0x7ff11019b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.135+0000 7ff115800640 1 -- 172.21.15.138:0/3083394355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1101a11b0 con 0x7ff11010af90 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff115800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff10802f540 tx=0x7ff1080316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10d600640 1 -- 172.21.15.138:0/3083394355 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff108003b70 con 0x7ff11010af90 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10d600640 1 -- 172.21.15.138:0/3083394355 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff108003d10 con 0x7ff11010af90 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10d600640 1 -- 172.21.15.138:0/3083394355 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10802f990 con 0x7ff11010af90 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1101a1440 con 0x7ff11010af90 2024-08-20T01:23:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1101a7b60 con 0x7ff11010af90 2024-08-20T01:23:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 Processor -- start 2024-08-20T01:23:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10d600640 1 -- 172.21.15.138:0/3083394355 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff10802fb30 con 0x7ff11010af90 2024-08-20T01:23:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 -- start start 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd50969b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 0x55bbd50a2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbd500c210 con 0x55bbd50a2e80 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bbd4ffb710 con 0x55bbd5096ef0 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bbd4fff750 con 0x55bbd50965d0 2024-08-20T01:23:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40168/0 (socket says 172.21.15.138:40168) 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10fe00640 1 -- 172.21.15.138:0/3004281617 learned_addr learned my addr 172.21.15.138:0/3004281617 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff114e00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 0x55bbd50a2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.141+0000 7ff10fe00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd50969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 -- 172.21.15.138:0/3004281617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 msgr2=0x55bbd50969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd50969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 -- 172.21.15.138:0/3004281617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 msgr2=0x55bbd50a2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 0x55bbd50a2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 -- 172.21.15.138:0/3004281617 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd4fc7de0 con 0x55bbd50a2e80 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff114e00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 0x55bbd50a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc011810 tx=0x7ff0fc0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf6b87335b57ec2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff102000640 1 -- 172.21.15.138:0/3004281617 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004030 con 0x55bbd50a2e80 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff102000640 1 -- 172.21.15.138:0/3004281617 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc011e50 con 0x55bbd50a2e80 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff102000640 1 -- 172.21.15.138:0/3004281617 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0037f0 con 0x55bbd50a2e80 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10d600640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 0x7ff0f80b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10d600640 1 -- 172.21.15.138:0/3083394355 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7ff108105680 con 0x7ff11010af90 2024-08-20T01:23:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff114e00640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 0x7ff0f80b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/3004281617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 msgr2=0x55bbd50a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc011810 tx=0x7ff0fc0097d0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/3004281617 shutdown_connections 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a5270 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd5096ef0 0x55bbd50a2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/3004281617 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd50969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/3004281617 >> 172.21.15.138:0/3004281617 conn(0x55bbd50943b0 msgr2=0x55bbd50955a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/3004281617 shutdown_connections 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff114e00640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 0x7ff0f80b6200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55bbd5099750 tx=0x7ff104005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/3004281617 wait complete. 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- start start 2024-08-20T01:23:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 Processor -- start 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd509fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd5096ef0 0x55bbd50a0350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55bbd4f34ab0 con 0x55bbd50a2e80 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55bbd4fa2140 con 0x55bbd50965d0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55bbd50a5b60 con 0x55bbd5096ef0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 10 client.0 _ll_register_callbacks cb 0x55bbd4fb55d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40180/0 (socket says 172.21.15.138:40180) 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 -- 172.21.15.138:0/1383556039 learned_addr learned my addr 172.21.15.138:0/1383556039 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff10fe00640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd5096ef0 0x55bbd50a0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff114e00640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd509fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 -- 172.21.15.138:0/1383556039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd5096ef0 msgr2=0x55bbd50a0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd5096ef0 0x55bbd50a0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 -- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 msgr2=0x55bbd509fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd509fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 -- 172.21.15.138:0/1383556039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd4fa3170 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff115800640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff108004af0 tx=0x7ff1080cb710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefebb590f84cc12 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1080e6060 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbd4fd2a30 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7ff1080c8230 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff1080ed050 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 10 client.17232 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55bbd4d5e790 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bbd4fd2a30 con 0x55bbd50a2e80 2024-08-20T01:23:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7ff108105260 con 0x55bbd50a2e80 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 10 client.17232 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 20 client.17232 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd50b2d40 0x55bbd50b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 4 client.17232 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.146+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbd500e160 con 0x55bbd50b2d40 2024-08-20T01:23:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff114e00640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd50b2d40 0x55bbd50b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff114e00640 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd50b2d40 0x55bbd50b5130 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d43778ed3aede753 server_cookie=e835a647139c25d9 in_seq=0 out_seq=0 2024-08-20T01:23:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff0f6a00640 10 client.17232 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff0f6a00640 1 -- 172.21.15.138:0/1383556039 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bbd500e160 con 0x55bbd50b2d40 2024-08-20T01:23:12.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff0f6a00640 10 client.17232 handle_command_reply: tid=0 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 client.17232 shutdown 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 msgr2=0x55bbd50a0890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff108004af0 tx=0x7ff1080cb710 comp rx=0 tx=0).stop 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 shutdown_connections 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd50b2d40 0x55bbd50b5130 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd50a2e80 0x55bbd50a0890 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd5096ef0 0x55bbd50a0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/1383556039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd509fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 >> 172.21.15.138:0/1383556039 conn(0x55bbd50943b0 msgr2=0x55bbd5099fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff0f6a00640 0 client.17232 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 shutdown_connections 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 1 -- 172.21.15.138:0/1383556039 wait complete. 2024-08-20T01:23:12.156 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.151+0000 7ff11a7ba740 20 client.17232 trim_cache size 0 max 16384 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 Processor -- start 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 -- start start 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd509a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 0x55bbd50baf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b45f0 con 0x55bbd5096ef0 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b1e30 con 0x55bbd50a2e80 2024-08-20T01:23:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101afca0 con 0x55bbd50965d0 2024-08-20T01:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff114e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd509a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40186/0 (socket says 172.21.15.138:40186) 2024-08-20T01:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 -- 172.21.15.138:0/2671925220 learned_addr learned my addr 172.21.15.138:0/2671925220 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff115800640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 0x55bbd50baf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 -- 172.21.15.138:0/2671925220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 msgr2=0x55bbd509a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd509a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 -- 172.21.15.138:0/2671925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 msgr2=0x55bbd50baf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 0x55bbd50baf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff10fe00640 1 -- 172.21.15.138:0/2671925220 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbd4dd7620 con 0x55bbd5096ef0 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff114e00640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd509a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.164+0000 7ff115800640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 0x55bbd50baf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff10fe00640 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0117b0 tx=0x7ff0fc009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3432d90a9bc3d560 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff101600640 1 -- 172.21.15.138:0/2671925220 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc02d070 con 0x55bbd5096ef0 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff101600640 1 -- 172.21.15.138:0/2671925220 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc009be0 con 0x55bbd5096ef0 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff101600640 1 -- 172.21.15.138:0/2671925220 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc012e60 con 0x55bbd5096ef0 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 -- 172.21.15.138:0/2671925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 msgr2=0x55bbd50baa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0117b0 tx=0x7ff0fc009620 comp rx=0 tx=0).stop 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 -- 172.21.15.138:0/2671925220 shutdown_connections 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50a2e80 0x55bbd50baf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50baa20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/2671925220 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50965d0 0x55bbd509a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 -- 172.21.15.138:0/2671925220 >> 172.21.15.138:0/2671925220 conn(0x55bbd50943b0 msgr2=0x55bbd509b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.169+0000 7ff11a7ba740 1 -- 172.21.15.138:0/2671925220 shutdown_connections 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- 172.21.15.138:0/2671925220 wait complete. 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- start start 2024-08-20T01:23:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 Processor -- start 2024-08-20T01:23:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd50a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 0x55bbd511b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b02f0 con 0x55bbd5096ef0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b1760 con 0x55bbd50965d0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1101b09c0 con 0x55bbd50a2e80 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 10 client.0 _ll_register_callbacks cb 0x7ff108003ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40198/0 (socket says 172.21.15.138:40198) 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 -- 172.21.15.138:0/393165213 learned_addr learned my addr 172.21.15.138:0/393165213 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff114e00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd50a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff115800640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 0x55bbd511b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 -- 172.21.15.138:0/393165213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 msgr2=0x55bbd511b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 0x55bbd511b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 -- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 msgr2=0x55bbd50a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd50a5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 -- 172.21.15.138:0/393165213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbd511bf20 con 0x55bbd5096ef0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff115800640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 0x55bbd511b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff10fe00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc02b040 tx=0x7ff0fc009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7840470d26f235ff server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc02d070 con 0x55bbd5096ef0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7ff0fc019040 con 0x55bbd5096ef0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0fc009be0 con 0x55bbd5096ef0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc03b050 con 0x55bbd5096ef0 2024-08-20T01:23:12.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 10 client.17238 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55bbd511d0b0 con 0x55bbd5096ef0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0fc009be0 con 0x55bbd5096ef0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7ff0fc028410 con 0x55bbd5096ef0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 10 client.17238 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 20 client.17238 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd511d5c0 0x55bbd511f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 4 client.17238 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bbd5009c00 con 0x55bbd511d5c0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff114e00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd511d5c0 0x55bbd511f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff114e00640 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd511d5c0 0x55bbd511f9e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1797ec9b9032a1a server_cookie=e15a4fd1daf70059 in_seq=0 out_seq=0 2024-08-20T01:23:12.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.170+0000 7ff0f6a00640 10 client.17238 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff0f6a00640 1 -- 172.21.15.138:0/393165213 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bbd5009c00 con 0x55bbd511d5c0 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff0f6a00640 10 client.17238 handle_command_reply: tid=0 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 client.17238 shutdown 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 msgr2=0x55bbd50a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc02b040 tx=0x7ff0fc009370 comp rx=0 tx=0).stop 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 shutdown_connections 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55bbd511d5c0 0x55bbd511f9e0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55bbd50a2e80 0x55bbd511b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55bbd5096ef0 0x55bbd50a6510 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 --2- 172.21.15.138:0/393165213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bbd50965d0 0x55bbd50a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff0f6a00640 0 client.17238 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 >> 172.21.15.138:0/393165213 conn(0x55bbd50943b0 msgr2=0x55bbd50a1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 shutdown_connections 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 1 -- 172.21.15.138:0/393165213 wait complete. 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff11a7ba740 20 client.17238 trim_cache size 0 max 16384 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:23:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 msgr2=0x7ff0f80b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 0x7ff0f80b6200 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55bbd5099750 tx=0x7ff104005f90 comp rx=0 tx=0).stop 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 msgr2=0x7ff1101a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7ff10802f540 tx=0x7ff1080316f0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 shutdown_connections 2024-08-20T01:23:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff0f80b3d40 0x7ff0f80b6200 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff11010af90 0x7ff1101a09e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff11010a610 0x7ff11019b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 --2- 172.21.15.138:0/3083394355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff110104c10 0x7ff11019b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 >> 172.21.15.138:0/3083394355 conn(0x7ff110100610 msgr2=0x7ff110102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 shutdown_connections 2024-08-20T01:23:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:12.214+0000 7ff117800640 1 -- 172.21.15.138:0/3083394355 wait complete. 2024-08-20T01:23:12.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 22.19 MB/sec warmup 69 sec latency 21696.232 ms 2024-08-20T01:23:12.748 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:12 smithi138 ceph-mon[18076]: pgmap v831: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 7.1 MiB/s wr, 367 op/s 2024-08-20T01:23:12.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:12 smithi167 ceph-mon[21965]: pgmap v831: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 7.1 MiB/s wr, 367 op/s 2024-08-20T01:23:12.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:12 smithi142 ceph-mon[21799]: pgmap v831: 129 pgs: 129 active+clean; 815 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 7.1 MiB/s wr, 367 op/s 2024-08-20T01:23:13.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 21.87 MB/sec warmup 70 sec latency 22696.281 ms 2024-08-20T01:23:14.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 21.56 MB/sec warmup 71 sec latency 23696.334 ms 2024-08-20T01:23:14.781 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:14 smithi138 ceph-mon[18076]: pgmap v832: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.0 MiB/s wr, 210 op/s 2024-08-20T01:23:14.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:14 smithi167 ceph-mon[21965]: pgmap v832: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.0 MiB/s wr, 210 op/s 2024-08-20T01:23:14.994 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:14 smithi142 ceph-mon[21799]: pgmap v832: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.0 MiB/s wr, 210 op/s 2024-08-20T01:23:15.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 21.26 MB/sec warmup 72 sec latency 24696.382 ms 2024-08-20T01:23:16.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 20.97 MB/sec warmup 73 sec latency 25696.431 ms 2024-08-20T01:23:16.742 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:16 smithi138 ceph-mon[18076]: pgmap v833: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 758 KiB/s wr, 172 op/s 2024-08-20T01:23:16.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:16 smithi167 ceph-mon[21965]: pgmap v833: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 758 KiB/s wr, 172 op/s 2024-08-20T01:23:16.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:16 smithi142 ceph-mon[21799]: pgmap v833: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 758 KiB/s wr, 172 op/s 2024-08-20T01:23:17.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 20.69 MB/sec warmup 74 sec latency 26696.479 ms 2024-08-20T01:23:18.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25226 20.41 MB/sec warmup 75 sec latency 27696.527 ms 2024-08-20T01:23:18.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.744+0000 7f02c0c00640 1 -- 172.21.15.138:0/20217654 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5565603751b0 con 0x556560493780 2024-08-20T01:23:18.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.744+0000 7f02c0c00640 10 client.27196 handle_command_reply: tid=0 2024-08-20T01:23:18.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.744+0000 7f02dc491740 1 client.27196 shutdown 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 msgr2=0x556560413a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x556560413a80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f02c4012cb0 tx=0x7f02c40046c0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 shutdown_connections 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556560493780 0x556560495ba0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55656040fca0 0x556560413a80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55656040d7b0 0x556560413540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 --2- 172.21.15.138:0/20217654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565604033f0 0x556560418070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02c0c00640 0 client.27196 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 >> 172.21.15.138:0/20217654 conn(0x556560401220 msgr2=0x55656040b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 shutdown_connections 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 1 -- 172.21.15.138:0/20217654 wait complete. 2024-08-20T01:23:18.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.745+0000 7f02dc491740 20 client.27196 trim_cache size 0 max 16384 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 54, 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "c9994f8d": { 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 54, 2024-08-20T01:23:18.747 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 140.0, 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:18.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 msgr2=0x7f02b80b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 0x7f02b80b6280 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f02c4005fd0 tx=0x7f02c400b580 comp rx=0 tx=0).stop 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 msgr2=0x7f02d419b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:18.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d419b6c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00c970 tx=0x7f02bc00ce30 comp rx=0 tx=0).stop 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 shutdown_connections 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f02b80b3dc0 0x7f02b80b6280 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02d410f0c0 0x7f02d41a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02d4075b10 0x7f02d419b6c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 --2- 172.21.15.138:0/730717791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f02d40751f0 0x7f02d419b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 >> 172.21.15.138:0/730717791 conn(0x7f02d40fe500 msgr2=0x7f02d40ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 shutdown_connections 2024-08-20T01:23:18.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:18.746+0000 7f02d9a00640 1 -- 172.21.15.138:0/730717791 wait complete. 2024-08-20T01:23:18.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:18 smithi138 ceph-mon[18076]: pgmap v834: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 556 KiB/s wr, 154 op/s 2024-08-20T01:23:18.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:18 smithi167 ceph-mon[21965]: pgmap v834: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 556 KiB/s wr, 154 op/s 2024-08-20T01:23:18.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:18 smithi142 ceph-mon[21799]: pgmap v834: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 549 KiB/s rd, 556 KiB/s wr, 154 op/s 2024-08-20T01:23:19.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25233 20.14 MB/sec warmup 76 sec latency 28696.605 ms 2024-08-20T01:23:19.509 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c9994f8d', quiesce took 35.8 seconds. Will release after: -15.799999999999997 2024-08-20T01:23:19.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:23:19.885 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:23:20.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25233 19.88 MB/sec warmup 77 sec latency 29696.688 ms 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 -- 172.21.15.138:0/3390351167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810b020 msgr2=0x7f97c8111820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 --2- 172.21.15.138:0/3390351167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810b020 0x7f97c8111820 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c002a8b0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 -- 172.21.15.138:0/3390351167 shutdown_connections 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 --2- 172.21.15.138:0/3390351167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810b020 0x7f97c8111820 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 --2- 172.21.15.138:0/3390351167 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810a6a0 0x7f97c810aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 --2- 172.21.15.138:0/3390351167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c8105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 -- 172.21.15.138:0/3390351167 >> 172.21.15.138:0/3390351167 conn(0x7f97c81006c0 msgr2=0x7f97c8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 -- 172.21.15.138:0/3390351167 shutdown_connections 2024-08-20T01:23:20.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.731+0000 7f97ce000640 1 -- 172.21.15.138:0/3390351167 wait complete. 2024-08-20T01:23:20.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 Processor -- start 2024-08-20T01:23:20.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 -- start start 2024-08-20T01:23:20.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:20.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810a6a0 0x7f97c819b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:20.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810b020 0x7f97c81a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8114cf0 con 0x7f97c810a6a0 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8114b70 con 0x7f97c8104c90 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.732+0000 7f97ce000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8114e70 con 0x7f97c810b020 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97bfe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810a6a0 0x7f97c819b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47278/0 (socket says 172.21.15.138:47278) 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 -- 172.21.15.138:0/1376478301 learned_addr learned my addr 172.21.15.138:0/1376478301 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7e00640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810b020 0x7f97c81a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 -- 172.21.15.138:0/1376478301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810b020 msgr2=0x7f97c81a09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810b020 0x7f97c81a09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 -- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810a6a0 msgr2=0x7f97c819b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810a6a0 0x7f97c819b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.733+0000 7f97c7400640 1 -- 172.21.15.138:0/1376478301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c81a11b0 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97c7400640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97b800e9f0 tx=0x7f97b800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b800cd80 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81a14a0 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97b80108e0 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b8018980 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.734+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81a7d30 con 0x7f97c8104c90 2024-08-20T01:23:20.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.735+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c8105070 con 0x7f97c8104c90 2024-08-20T01:23:20.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.737+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f97b8010430 con 0x7f97c8104c90 2024-08-20T01:23:20.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.739+0000 7f97c4c00640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 0x7f97ac0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:20.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.739+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f97b80e4aa0 con 0x7f97c8104c90 2024-08-20T01:23:20.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.739+0000 7f97bfe00640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 0x7f97ac0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.740+0000 7f97bfe00640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 0x7f97ac0b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f97c8106c30 tx=0x7f97b0005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:20.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.740+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f97b80ab360 con 0x7f97c8104c90 2024-08-20T01:23:20.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:20 smithi167 ceph-mon[21965]: pgmap v835: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 410 KiB/s wr, 126 op/s 2024-08-20T01:23:20.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.988+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97c819c230 con 0x7f97c8104c90 2024-08-20T01:23:20.987 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:20 smithi138 ceph-mon[18076]: pgmap v835: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 410 KiB/s wr, 126 op/s 2024-08-20T01:23:20.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.989+0000 7f97c4c00640 1 -- 172.21.15.138:0/1376478301 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11939 (secure 0 0 0) 0x7f97b80af8b0 con 0x7f97c8104c90 2024-08-20T01:23:20.988 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:23:20.988 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":48,"btime":"2024-08-20T01:22:21:451368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:22:21.168170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:23:20.989 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 48 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.991+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 msgr2=0x7f97ac0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.991+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 0x7f97ac0b6200 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f97c8106c30 tx=0x7f97b0005f90 comp rx=0 tx=0).stop 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.991+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 msgr2=0x7f97c819b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.991+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97b800e9f0 tx=0x7f97b800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 shutdown_connections 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f97ac0b3d40 0x7f97ac0b6200 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f97c810b020 0x7f97c81a09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97c810a6a0 0x7f97c819b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 --2- 172.21.15.138:0/1376478301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97c8104c90 0x7f97c819b100 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:20.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 >> 172.21.15.138:0/1376478301 conn(0x7f97c81006c0 msgr2=0x7f97c8100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:20.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 shutdown_connections 2024-08-20T01:23:20.992 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:20.992+0000 7f97ce000640 1 -- 172.21.15.138:0/1376478301 wait complete. 2024-08-20T01:23:20.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:20 smithi142 ceph-mon[21799]: pgmap v835: 129 pgs: 129 active+clean; 807 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 410 KiB/s wr, 126 op/s 2024-08-20T01:23:21.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25233 19.63 MB/sec warmup 78 sec latency 30696.770 ms 2024-08-20T01:23:21.784 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id c9994f8d --release --await' 2024-08-20T01:23:21.784 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id c9994f8d --release --await 2024-08-20T01:23:21.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:21 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1376478301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:23:21.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:21 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1376478301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:23:22.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:21 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1376478301' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:23:22.160 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:23:22.262 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25233 19.38 MB/sec warmup 79 sec latency 31696.878 ms 2024-08-20T01:23:22.803 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 -- 172.21.15.138:0/245835295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 msgr2=0x7f945c10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 --2- 172.21.15.138:0/245835295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c10f640 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ca0 tx=0x7f945402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 -- 172.21.15.138:0/245835295 shutdown_connections 2024-08-20T01:23:22.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 --2- 172.21.15.138:0/245835295 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c10f640 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 --2- 172.21.15.138:0/245835295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c1084c0 0x7f945c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 --2- 172.21.15.138:0/245835295 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c102ab0 0x7f945c102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 -- 172.21.15.138:0/245835295 >> 172.21.15.138:0/245835295 conn(0x7f945c0fe500 msgr2=0x7f945c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 -- 172.21.15.138:0/245835295 shutdown_connections 2024-08-20T01:23:22.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.778+0000 7f9463e00640 1 -- 172.21.15.138:0/245835295 wait complete. 2024-08-20T01:23:22.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.779+0000 7f9463e00640 1 Processor -- start 2024-08-20T01:23:22.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.779+0000 7f9463e00640 1 -- start start 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c1084c0 0x7f945c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f945c112b50 con 0x7f945c108e40 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1129d0 con 0x7f945c102ab0 2024-08-20T01:23:22.806 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9463e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f945c112cd0 con 0x7f945c1084c0 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47302/0 (socket says 172.21.15.138:47302) 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 -- 172.21.15.138:0/2091063879 learned_addr learned my addr 172.21.15.138:0/2091063879 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9460a00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c1084c0 0x7f945c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 -- 172.21.15.138:0/2091063879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c1084c0 msgr2=0x7f945c075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c1084c0 0x7f945c075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 -- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 msgr2=0x7f945c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.780+0000 7f9461400640 1 -- 172.21.15.138:0/2091063879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f945c1a7340 con 0x7f945c102ab0 2024-08-20T01:23:22.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f9461400640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9f0 tx=0x7f945000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f945a000640 1 -- 172.21.15.138:0/2091063879 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945000cd80 con 0x7f945c102ab0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f945a000640 1 -- 172.21.15.138:0/2091063879 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94500054d0 con 0x7f945c102ab0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f945c1a7630 con 0x7f945c102ab0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f945a000640 1 -- 172.21.15.138:0/2091063879 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450010690 con 0x7f945c102ab0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.781+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f945c1a7b20 con 0x7f945c102ab0 2024-08-20T01:23:22.809 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 Processor -- start 2024-08-20T01:23:22.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 -- start start 2024-08-20T01:23:22.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e69060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f945a000640 1 -- 172.21.15.138:0/2091063879 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9450004e90 con 0x7f945c102ab0 2024-08-20T01:23:22.810 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e6b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4dfe670 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4dd2d70 con 0x55d5b4e5d610 2024-08-20T01:23:22.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.784+0000 7f94669bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4dc2270 con 0x55d5b4e695a0 2024-08-20T01:23:22.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.811 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44054/0 (socket says 172.21.15.138:44054) 2024-08-20T01:23:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 -- 172.21.15.138:0/2852712646 learned_addr learned my addr 172.21.15.138:0/2852712646 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9460a00640 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e6b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461400640 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e69060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 -- 172.21.15.138:0/2852712646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 msgr2=0x55d5b4e6b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e6b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 -- 172.21.15.138:0/2852712646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 msgr2=0x55d5b4e69060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e69060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 -- 172.21.15.138:0/2852712646 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5b4d5e870 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.785+0000 7f9461e00640 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ca0 tx=0x7f9454005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23620d396746b37 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f944e600640 1 -- 172.21.15.138:0/2852712646 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454031cf0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f944e600640 1 -- 172.21.15.138:0/2852712646 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454036030 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f944e600640 1 -- 172.21.15.138:0/2852712646 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454004280 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 -- 172.21.15.138:0/2852712646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 msgr2=0x55d5b4e5d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f9454009ca0 tx=0x7f9454005f80 comp rx=0 tx=0).stop 2024-08-20T01:23:22.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 -- 172.21.15.138:0/2852712646 shutdown_connections 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e6b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e69060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 --2- 172.21.15.138:0/2852712646 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e5d0d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f94669bf740 1 -- 172.21.15.138:0/2852712646 >> 172.21.15.138:0/2852712646 conn(0x55d5b4e5ab20 msgr2=0x55d5b4e5bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.786+0000 7f945a000640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 0x7f94440b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f94669bf740 1 -- 172.21.15.138:0/2852712646 shutdown_connections 2024-08-20T01:23:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f94669bf740 1 -- 172.21.15.138:0/2852712646 wait complete. 2024-08-20T01:23:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f945a000640 1 -- 172.21.15.138:0/2091063879 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f94500e47e0 con 0x7f945c102ab0 2024-08-20T01:23:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f9460a00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 0x7f94440b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f94669bf740 1 -- start start 2024-08-20T01:23:22.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f94669bf740 1 Processor -- start 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.787+0000 7f9460a00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 0x7f94440b6200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f945c076cc0 tx=0x7f9448005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e66190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4d48ff0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4dc62b0 con 0x55d5b4e5d610 2024-08-20T01:23:22.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55d5b4c24bb0 con 0x55d5b4e695a0 2024-08-20T01:23:22.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f94669bf740 10 client.0 _ll_register_callbacks cb 0x55d5b4acdd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:22.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9461400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9460a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e66190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.818 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9460a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e66190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47312/0 (socket says 172.21.15.138:47312) 2024-08-20T01:23:22.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9460a00640 1 -- 172.21.15.138:0/3166775973 learned_addr learned my addr 172.21.15.138:0/3166775973 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:22.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9461400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44066/0 (socket says 172.21.15.138:44066) 2024-08-20T01:23:22.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.788+0000 7f9461e00640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 -- 172.21.15.138:0/3166775973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 msgr2=0x55d5b4e666d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.819 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e666d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 -- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 msgr2=0x55d5b4e66190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e66190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 -- 172.21.15.138:0/3166775973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5b4b21b00 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9461400640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94500e1b30 tx=0x7f94500048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d756e36fb79651 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9442a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94500aa760 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f94500e43c0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.820 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9454036030 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f94500e1fc0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f94669bf740 10 client.17256 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.789+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d5b4b98960 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9454036030 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f94500e4bd0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 10 client.17256 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 10 client.17256 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 20 client.17256 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4e81240 0x55d5b4e83690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 4 client.17256 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:23:22.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5b4dd4cc0 con 0x55d5b4e81240 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.790+0000 7f9460a00640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4e81240 0x55d5b4e83690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.791+0000 7f9460a00640 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4e81240 0x55d5b4e83690 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33b9adb877925930 server_cookie=e07a88f617853181 in_seq=0 out_seq=0 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.791+0000 7f9442a00640 10 client.17256 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.791+0000 7f9442a00640 1 -- 172.21.15.138:0/3166775973 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d5b4dd4cc0 con 0x55d5b4e81240 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.791+0000 7f9442a00640 10 client.17256 handle_command_reply: tid=0 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.791+0000 7f94669bf740 1 client.17256 shutdown 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 msgr2=0x55d5b4e68d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f94500e1b30 tx=0x7f94500048f0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 shutdown_connections 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4e81240 0x55d5b4e83690 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e666d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.822 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e66190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 --2- 172.21.15.138:0/3166775973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e68d40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f9442a00640 0 client.17256 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 >> 172.21.15.138:0/3166775973 conn(0x55d5b4e5ab20 msgr2=0x55d5b4e606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 shutdown_connections 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 1 -- 172.21.15.138:0/3166775973 wait complete. 2024-08-20T01:23:22.823 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.792+0000 7f94669bf740 20 client.17256 trim_cache size 0 max 16384 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 Processor -- start 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 -- start start 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e59b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e88f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1b01c0 con 0x55d5b4e5d610 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ada00 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f94669bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ab8d0 con 0x55d5b4e695a0 2024-08-20T01:23:22.827 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f9460a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f9460a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44074/0 (socket says 172.21.15.138:44074) 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f9460a00640 1 -- 172.21.15.138:0/3237063671 learned_addr learned my addr 172.21.15.138:0/3237063671 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f9461e00640 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e88f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.811+0000 7f9461400640 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e59b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 -- 172.21.15.138:0/3237063671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 msgr2=0x55d5b4e88f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e88f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 -- 172.21.15.138:0/3237063671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5ccf0 msgr2=0x55d5b4e59b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e59b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 -- 172.21.15.138:0/3237063671 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5b4e5c6d0 con 0x55d5b4e5d610 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f9460a00640 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9448009460 tx=0x7f944801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeac13048b7b8d29 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f944dc00640 1 -- 172.21.15.138:0/3237063671 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448005020 con 0x55d5b4e5d610 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f944dc00640 1 -- 172.21.15.138:0/3237063671 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94480047a0 con 0x55d5b4e5d610 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.812+0000 7f944dc00640 1 -- 172.21.15.138:0/3237063671 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944802b610 con 0x55d5b4e5d610 2024-08-20T01:23:22.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 -- 172.21.15.138:0/3237063671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 msgr2=0x55d5b4e5a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f9448009460 tx=0x7f944801b000 comp rx=0 tx=0).stop 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 -- 172.21.15.138:0/3237063671 shutdown_connections 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e695a0 0x55d5b4e88f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e5d610 0x55d5b4e5a070 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 --2- 172.21.15.138:0/3237063671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e59b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 -- 172.21.15.138:0/3237063671 >> 172.21.15.138:0/3237063671 conn(0x55d5b4e5ab20 msgr2=0x55d5b4e59420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 -- 172.21.15.138:0/3237063671 shutdown_connections 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.813+0000 7f94669bf740 1 -- 172.21.15.138:0/3237063671 wait complete. 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 -- start start 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 Processor -- start 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e77db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 0x55d5b4e782f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1abf20 con 0x55d5b4e695a0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ad330 con 0x55d5b4e5d610 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f945c1ac590 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.814+0000 7f94669bf740 10 client.0 _ll_register_callbacks cb 0x7f945000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 0x55d5b4e782f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56736/0 (socket says 172.21.15.138:56736) 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 -- 172.21.15.138:0/1246789047 learned_addr learned my addr 172.21.15.138:0/1246789047 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9460a00640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e77db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 -- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 msgr2=0x55d5b4e77db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.830 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e77db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 -- 172.21.15.138:0/1246789047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 msgr2=0x55d5b4e782f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 0x55d5b4e782f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 -- 172.21.15.138:0/1246789047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5b4eeafc0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461e00640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 0x55d5b4e782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9461400640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f94500b3cb0 tx=0x7f94500e45f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e32ca2b5022d627b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.815+0000 7f9442a00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450005180 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9453+0+0 (secure 0 0 0) 0x7f94500e4c70 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94480047a0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f94500aa760 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 10 client.27256 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d5b4eec310 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94480047a0 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f94500b5030 con 0x55d5b4e5ccf0 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 10 client.27256 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 10 client.27256 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 20 client.27256 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4eecb10 0x55d5b4eeef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 4 client.27256 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c9994f8d", "release": true, "await": true}] 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.816+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c9994f8d", "release": true, "await": true}) -- 0x55d5b4dd0760 con 0x55d5b4eecb10 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.817+0000 7f9460a00640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4eecb10 0x55d5b4eeef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.817+0000 7f9460a00640 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4eecb10 0x55d5b4eeef30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f394d92f142da093 server_cookie=c14d9c38176326f5 in_seq=0 out_seq=0 2024-08-20T01:23:22.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.817+0000 7f9442a00640 10 client.27256 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f9442a00640 1 -- 172.21.15.138:0/1246789047 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d5b4dd0760 con 0x55d5b4eecb10 2024-08-20T01:23:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f9442a00640 10 client.27256 handle_command_reply: tid=0 2024-08-20T01:23:22.857 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 client.27256 shutdown 2024-08-20T01:23:22.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 msgr2=0x55d5b4e77870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f94500b3cb0 tx=0x7f94500e45f0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 shutdown_connections 2024-08-20T01:23:22.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55d5b4eecb10 0x55d5b4eeef30 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.858 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55d5b4e695a0 0x55d5b4e782f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5b4e5d610 0x55d5b4e77db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 --2- 172.21.15.138:0/1246789047 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55d5b4e5ccf0 0x55d5b4e77870 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f9442a00640 0 client.27256 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 >> 172.21.15.138:0/1246789047 conn(0x55d5b4e5ab20 msgr2=0x55d5b4e59c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.858+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 shutdown_connections 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.859+0000 7f94669bf740 1 -- 172.21.15.138:0/1246789047 wait complete. 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.859+0000 7f94669bf740 20 client.27256 trim_cache size 0 max 16384 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-08-20T01:23:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 56, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "c9994f8d": { 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 56, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 140.0, 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:23:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:23:22.861 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 msgr2=0x7f94440b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 0x7f94440b6200 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f945c076cc0 tx=0x7f9448005f90 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 msgr2=0x7f945c0751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f945000e9f0 tx=0x7f945000edf0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 shutdown_connections 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f94440b3d40 0x7f94440b6200 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f945c108e40 0x7f945c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f945c1084c0 0x7f945c075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 --2- 172.21.15.138:0/2091063879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f945c102ab0 0x7f945c0751d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 >> 172.21.15.138:0/2091063879 conn(0x7f945c0fe500 msgr2=0x7f945c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.860+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 shutdown_connections 2024-08-20T01:23:22.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:23:22.861+0000 7f9463e00640 1 -- 172.21.15.138:0/2091063879 wait complete. 2024-08-20T01:23:22.963 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:22 smithi167 ceph-mon[21965]: pgmap v836: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 542 KiB/s wr, 143 op/s 2024-08-20T01:23:22.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:22 smithi142 ceph-mon[21799]: pgmap v836: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 542 KiB/s wr, 143 op/s 2024-08-20T01:23:23.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:22 smithi138 ceph-mon[18076]: pgmap v836: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 542 KiB/s wr, 143 op/s 2024-08-20T01:23:23.263 INFO:tasks.workunit.client.0.smithi138.stdout: 10 25486 19.31 MB/sec warmup 80 sec latency 32258.472 ms 2024-08-20T01:23:24.265 INFO:tasks.workunit.client.0.smithi138.stdout: 10 26326 19.70 MB/sec warmup 81 sec latency 54.500 ms 2024-08-20T01:23:24.500 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c9994f8d', total seconds elapsed: 40.8 2024-08-20T01:23:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:24 smithi138 ceph-mon[18076]: pgmap v837: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 177 KiB/s wr, 67 op/s 2024-08-20T01:23:24.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:24 smithi167 ceph-mon[21965]: pgmap v837: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 177 KiB/s wr, 67 op/s 2024-08-20T01:23:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:24 smithi142 ceph-mon[21799]: pgmap v837: 129 pgs: 129 active+clean; 808 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 177 KiB/s wr, 67 op/s 2024-08-20T01:23:25.263 INFO:tasks.workunit.client.0.smithi138.stdout: 10 26977 19.75 MB/sec warmup 82 sec latency 145.860 ms 2024-08-20T01:23:25.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 -- 172.21.15.138:0/952542725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca01135e0 msgr2=0x7f7ca01159d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/952542725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca01135e0 0x7f7ca01159d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009be0 tx=0x7f7c9802a6d0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 -- 172.21.15.138:0/952542725 shutdown_connections 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/952542725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca01135e0 0x7f7ca01159d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/952542725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca0077d20 0x7f7ca01130a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/952542725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca00777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.413+0000 7f7ca6e00640 1 -- 172.21.15.138:0/952542725 >> 172.21.15.138:0/952542725 conn(0x7f7ca0100680 msgr2=0x7f7ca0102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.414+0000 7f7ca6e00640 1 -- 172.21.15.138:0/952542725 shutdown_connections 2024-08-20T01:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.414+0000 7f7ca6e00640 1 -- 172.21.15.138:0/952542725 wait complete. 2024-08-20T01:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.415+0000 7f7ca6e00640 1 Processor -- start 2024-08-20T01:23:25.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.416+0000 7f7ca6e00640 1 -- start start 2024-08-20T01:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.417+0000 7f7ca6e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.417+0000 7f7ca6e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 0x7f7ca0112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7ca6e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca01135e0 0x7f7ca010bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7ca6e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0118fc0 con 0x7f7ca0077d20 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56746/0 (socket says 172.21.15.138:56746) 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7c9fe00640 1 -- 172.21.15.138:0/202997341 learned_addr learned my addr 172.21.15.138:0/202997341 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7c97e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 0x7f7ca0112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.418+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca01135e0 0x7f7ca010bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.419+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0118e40 con 0x7f7ca01135e0 2024-08-20T01:23:25.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.419+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0119140 con 0x7f7ca0077400 2024-08-20T01:23:25.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c9fe00640 1 -- 172.21.15.138:0/202997341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca01135e0 msgr2=0x7f7ca010bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca01135e0 0x7f7ca010bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c9fe00640 1 -- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 msgr2=0x7f7ca0112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 0x7f7ca0112ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c9fe00640 1 -- 172.21.15.138:0/202997341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca010c590 con 0x7f7ca0077400 2024-08-20T01:23:25.421 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.420+0000 7f7c97e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 0x7f7ca0112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000e9c0 tx=0x7f7c9000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7c9d600640 1 -- 172.21.15.138:0/202997341 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9000ccc0 con 0x7f7ca0077400 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7c9d600640 1 -- 172.21.15.138:0/202997341 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c900054d0 con 0x7f7ca0077400 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7c9d600640 1 -- 172.21.15.138:0/202997341 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010430 con 0x7f7ca0077400 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca010c880 con 0x7f7ca0077400 2024-08-20T01:23:25.422 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.421+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca01117f0 con 0x7f7ca0077400 2024-08-20T01:23:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.426+0000 7f7c9d600640 1 -- 172.21.15.138:0/202997341 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7c90005020 con 0x7f7ca0077400 2024-08-20T01:23:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.428+0000 7f7ca9e09740 1 Processor -- start 2024-08-20T01:23:25.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 -- start start 2024-08-20T01:23:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a81ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 0x558eb6a845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558eb69eb210 con 0x558eb6a75930 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558eb69da710 con 0x558eb6a76250 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.429+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558eb69de750 con 0x558eb6a821e0 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.430+0000 7f7c9d600640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 0x7f7c840b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.430+0000 7f7c9d600640 1 -- 172.21.15.138:0/202997341 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9541+0+0 (secure 0 0 0) 0x7f7c900e4980 con 0x7f7ca0077400 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.430+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.431+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 0x7f7c840b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.431+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:44104/0 (socket says 172.21.15.138:44104) 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.431+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3545421956 learned_addr learned my addr 172.21.15.138:0/3545421956 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.431+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a81ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.431+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 0x558eb6a845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 -- 172.21.15.138:0/3545421956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 msgr2=0x558eb6a81ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a81ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 -- 172.21.15.138:0/3545421956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 msgr2=0x558eb6a75d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 -- 172.21.15.138:0/3545421956 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eb69a6de0 con 0x558eb6a821e0 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.432+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 0x558eb6a845d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800af90 tx=0x7f7c8800f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae27589cf9bab436 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.433+0000 7f7c95600640 1 -- 172.21.15.138:0/3545421956 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88005220 con 0x558eb6a821e0 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.433+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 0x7f7c840b6300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005f00 tx=0x7f7c98005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.434+0000 7f7c95600640 1 -- 172.21.15.138:0/3545421956 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c880047a0 con 0x558eb6a821e0 2024-08-20T01:23:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.434+0000 7f7c95600640 1 -- 172.21.15.138:0/3545421956 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c88010af0 con 0x558eb6a821e0 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.435+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3545421956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 msgr2=0x558eb6a845d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.435+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 0x558eb6a845d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800af90 tx=0x7f7c8800f850 comp rx=0 tx=0).stop 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3545421956 shutdown_connections 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a821e0 0x558eb6a845d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a81ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3545421956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a75d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3545421956 >> 172.21.15.138:0/3545421956 conn(0x558eb6a73760 msgr2=0x558eb6a74900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3545421956 shutdown_connections 2024-08-20T01:23:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.436+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3545421956 wait complete. 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.437+0000 7f7ca9e09740 1 -- start start 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.437+0000 7f7ca9e09740 1 Processor -- start 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a81a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a7ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x558eb6913ab0 con 0x558eb6a75930 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x558eb6981140 con 0x558eb6a821e0 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558eb6a84ec0 con 0x558eb6a76250 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.438+0000 7f7ca9e09740 10 client.0 _ll_register_callbacks cb 0x558eb69945d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7c97e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a7ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47360/0 (socket says 172.21.15.138:47360) 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3080122014 learned_addr learned my addr 172.21.15.138:0/3080122014 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3080122014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 msgr2=0x558eb6a7ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a7ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.439+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3080122014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 msgr2=0x558eb6a81a10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.440+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a81a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.440+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3080122014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eb6982170 con 0x558eb6a821e0 2024-08-20T01:23:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.442+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034040 tx=0x7f7c98009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9480380089bce2c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.442+0000 7f7c8e000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:23:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.442+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98036040 con 0x558eb6a821e0 2024-08-20T01:23:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.445+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9541+0+0 (secure 0 0 0) 0x7f7c98004240 con 0x558eb6a821e0 2024-08-20T01:23:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.446+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c880047a0 con 0x558eb6a821e0 2024-08-20T01:23:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.446+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7c98003ba0 con 0x558eb6a821e0 2024-08-20T01:23:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.446+0000 7f7ca9e09740 10 client.27293 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.446+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558eb673d790 con 0x558eb6a821e0 2024-08-20T01:23:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c880047a0 con 0x558eb6a821e0 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f7c9803c040 con 0x558eb6a821e0 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 10 client.27293 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 10 client.27293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 20 client.27293 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6a91e50 0x558eb6a942c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 4 client.27293 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.447+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eb69ed160 con 0x558eb6a91e50 2024-08-20T01:23:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.448+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6a91e50 0x558eb6a942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.450+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6a91e50 0x558eb6a942c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52021aa0c83d7042 server_cookie=616d02de71fc1a59 in_seq=0 out_seq=0 2024-08-20T01:23:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.450+0000 7f7c8e000640 10 client.27293 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.451+0000 7f7c8e000640 1 -- 172.21.15.138:0/3080122014 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558eb69ed160 con 0x558eb6a91e50 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.451+0000 7f7c8e000640 10 client.27293 handle_command_reply: tid=0 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.451+0000 7f7ca9e09740 1 client.27293 shutdown 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 msgr2=0x558eb6a7f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7c98034040 tx=0x7f7c98009500 comp rx=0 tx=0).stop 2024-08-20T01:23:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 shutdown_connections 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6a91e50 0x558eb6a942c0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a7f390 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a7ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3080122014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a81a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 >> 172.21.15.138:0/3080122014 conn(0x558eb6a73760 msgr2=0x558eb6a79290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.452+0000 7f7c8e000640 0 client.27293 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.453+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 shutdown_connections 2024-08-20T01:23:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.453+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3080122014 wait complete. 2024-08-20T01:23:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.455+0000 7f7ca9e09740 20 client.27293 trim_cache size 0 max 16384 2024-08-20T01:23:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.490+0000 7f7ca9e09740 1 Processor -- start 2024-08-20T01:23:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.490+0000 7f7ca9e09740 1 -- start start 2024-08-20T01:23:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a99bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01b8c70 con 0x558eb6a75930 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01b64b0 con 0x558eb6a821e0 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0074000 con 0x558eb6a76250 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.491+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47362/0 (socket says 172.21.15.138:47362) 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3171459911 learned_addr learned my addr 172.21.15.138:0/3171459911 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3171459911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 msgr2=0x558eb6a99bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a99bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3171459911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 msgr2=0x558eb6a74b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:25.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.492+0000 7f7ca4e00640 1 -- 172.21.15.138:0/3171459911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eb67b6620 con 0x558eb6a821e0 2024-08-20T01:23:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.493+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c98030710 tx=0x7f7c98003f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38496963d6275ff1 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.493+0000 7f7c94c00640 1 -- 172.21.15.138:0/3171459911 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98003ba0 con 0x558eb6a821e0 2024-08-20T01:23:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.493+0000 7f7c94c00640 1 -- 172.21.15.138:0/3171459911 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c98051070 con 0x558eb6a821e0 2024-08-20T01:23:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.493+0000 7f7c94c00640 1 -- 172.21.15.138:0/3171459911 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c980045e0 con 0x558eb6a821e0 2024-08-20T01:23:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.494+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3171459911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 msgr2=0x558eb6a9a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.494+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7c98030710 tx=0x7f7c98003f20 comp rx=0 tx=0).stop 2024-08-20T01:23:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.495+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3171459911 shutdown_connections 2024-08-20T01:23:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.495+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a821e0 0x558eb6a9a110 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.495+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a76250 0x558eb6a99bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.495+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3171459911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a75930 0x558eb6a74b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.495+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3171459911 >> 172.21.15.138:0/3171459911 conn(0x558eb6a73760 msgr2=0x558eb6a74440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.496+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3171459911 shutdown_connections 2024-08-20T01:23:25.497 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.496+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3171459911 wait complete. 2024-08-20T01:23:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.497+0000 7f7ca9e09740 1 -- start start 2024-08-20T01:23:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.498+0000 7f7ca9e09740 1 Processor -- start 2024-08-20T01:23:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a87d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a821e0 0x558eb6a88250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0074650 con 0x558eb6a821e0 2024-08-20T01:23:25.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca01b5de0 con 0x558eb6a76250 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca0074d20 con 0x558eb6a75930 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7ca9e09740 10 client.0 _ll_register_callbacks cb 0x7f7c90005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7c9fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:56784/0 (socket says 172.21.15.138:56784) 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.500+0000 7f7c9fe00640 1 -- 172.21.15.138:0/3367528942 learned_addr learned my addr 172.21.15.138:0/3367528942 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.501+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a87d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.501+0000 7f7c9fe00640 1 -- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 msgr2=0x558eb6a87d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.501+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a87d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.501+0000 7f7c9fe00640 1 -- 172.21.15.138:0/3367528942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a821e0 msgr2=0x558eb6a88250 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:23:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.502+0000 7f7ca4e00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a821e0 0x558eb6a88250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.502+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a821e0 0x558eb6a88250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.502+0000 7f7c9fe00640 1 -- 172.21.15.138:0/3367528942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eb6a88a80 con 0x558eb6a75930 2024-08-20T01:23:25.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.503+0000 7f7c9fe00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c6a0 tx=0x7f7c900048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951f9939c68d5d46 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:25.504 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.503+0000 7f7c8e000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:23:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.504+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c900ab130 con 0x558eb6a75930 2024-08-20T01:23:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.504+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9541+0+0 (secure 0 0 0) 0x7f7c900a7720 con 0x558eb6a75930 2024-08-20T01:23:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.504+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c98051070 con 0x558eb6a75930 2024-08-20T01:23:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7c900af730 con 0x558eb6a75930 2024-08-20T01:23:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7ca9e09740 10 client.27271 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558eb6afcc20 con 0x558eb6a75930 2024-08-20T01:23:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c98051070 con 0x558eb6a75930 2024-08-20T01:23:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f7c900a6aa0 con 0x558eb6a75930 2024-08-20T01:23:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7ca9e09740 10 client.27271 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.505+0000 7f7ca9e09740 10 client.27271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.506+0000 7f7ca9e09740 20 client.27271 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.506+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6afd300 0x558eb6aff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.506+0000 7f7ca9e09740 4 client.27271 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.506+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558eb69e8c00 con 0x558eb6afd300 2024-08-20T01:23:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.507+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6afd300 0x558eb6aff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.507+0000 7f7c97e00640 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6afd300 0x558eb6aff720 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61e46a782ebe119 server_cookie=db6f55d867200ba8 in_seq=0 out_seq=0 2024-08-20T01:23:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.507+0000 7f7c8e000640 10 client.27271 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.757+0000 7f7c8e000640 1 -- 172.21.15.138:0/3367528942 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558eb69e8c00 con 0x558eb6afd300 2024-08-20T01:23:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.757+0000 7f7c8e000640 10 client.27271 handle_command_reply: tid=0 2024-08-20T01:23:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.757+0000 7f7ca9e09740 1 client.27271 shutdown 2024-08-20T01:23:25.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.757+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 msgr2=0x558eb6a8a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.757+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000c6a0 tx=0x7f7c900048f0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 shutdown_connections 2024-08-20T01:23:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x558eb6afd300 0x558eb6aff720 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558eb6a821e0 0x558eb6a88250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558eb6a76250 0x558eb6a87d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 --2- 172.21.15.138:0/3367528942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558eb6a75930 0x558eb6a8a7e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7c8e000640 0 client.27271 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 >> 172.21.15.138:0/3367528942 conn(0x558eb6a73760 msgr2=0x558eb6a72760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 shutdown_connections 2024-08-20T01:23:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 1 -- 172.21.15.138:0/3367528942 wait complete. 2024-08-20T01:23:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.758+0000 7f7ca9e09740 20 client.27271 trim_cache size 0 max 16384 2024-08-20T01:23:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:23:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:23:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:23:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:23:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 msgr2=0x7f7c840b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 0x7f7c840b6300 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7c98005f00 tx=0x7f7c98005fd0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 msgr2=0x7f7ca01129b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000e9c0 tx=0x7f7c9000edc0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 shutdown_connections 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7c840b3e40 0x7f7c840b6300 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ca01135e0 0x7f7ca010bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7ca0077d20 0x7f7ca0112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 --2- 172.21.15.138:0/202997341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7ca0077400 0x7f7ca01129b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 >> 172.21.15.138:0/202997341 conn(0x7f7ca0100680 msgr2=0x7f7ca0102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 shutdown_connections 2024-08-20T01:23:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:25.760+0000 7f7ca6e00640 1 -- 172.21.15.138:0/202997341 wait complete. 2024-08-20T01:23:25.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:25 smithi167 ceph-mon[21965]: osdmap e106: 12 total, 12 up, 12 in 2024-08-20T01:23:25.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:25 smithi142 ceph-mon[21799]: osdmap e106: 12 total, 12 up, 12 in 2024-08-20T01:23:26.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:25 smithi138 ceph-mon[18076]: osdmap e106: 12 total, 12 up, 12 in 2024-08-20T01:23:26.263 INFO:tasks.workunit.client.0.smithi138.stdout: 10 27526 19.99 MB/sec warmup 83 sec latency 185.939 ms 2024-08-20T01:23:26.698 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:26 smithi138 ceph-mon[18076]: pgmap v839: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 655 KiB/s wr, 102 op/s 2024-08-20T01:23:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:26 smithi167 ceph-mon[21965]: pgmap v839: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 655 KiB/s wr, 102 op/s 2024-08-20T01:23:26.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:26 smithi142 ceph-mon[21799]: pgmap v839: 129 pgs: 2 active+clean+snaptrim, 7 active+clean+snaptrim_wait, 120 active+clean; 812 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 655 KiB/s wr, 102 op/s 2024-08-20T01:23:27.263 INFO:tasks.workunit.client.0.smithi138.stdout: 10 28275 20.25 MB/sec warmup 84 sec latency 97.053 ms 2024-08-20T01:23:28.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 29100 20.43 MB/sec warmup 85 sec latency 60.194 ms 2024-08-20T01:23:28.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:28 smithi138 ceph-mon[18076]: pgmap v840: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 819 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-08-20T01:23:28.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:28 smithi167 ceph-mon[21965]: pgmap v840: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 819 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-08-20T01:23:28.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:28 smithi142 ceph-mon[21799]: pgmap v840: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 819 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 212 op/s 2024-08-20T01:23:29.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 30043 20.78 MB/sec warmup 86 sec latency 93.121 ms 2024-08-20T01:23:30.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 30689 21.08 MB/sec warmup 87 sec latency 456.189 ms 2024-08-20T01:23:30.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:30 smithi167 ceph-mon[21965]: pgmap v841: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 824 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-08-20T01:23:30.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:30 smithi142 ceph-mon[21799]: pgmap v841: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 824 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-08-20T01:23:31.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:30 smithi138 ceph-mon[18076]: pgmap v841: 129 pgs: 7 active+clean+snaptrim, 19 active+clean+snaptrim_wait, 103 active+clean; 824 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 2.0 MiB/s wr, 279 op/s 2024-08-20T01:23:31.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 31305 21.17 MB/sec warmup 88 sec latency 415.632 ms 2024-08-20T01:23:32.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 32162 21.41 MB/sec warmup 89 sec latency 1415.682 ms 2024-08-20T01:23:32.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:32 smithi167 ceph-mon[21965]: pgmap v842: 129 pgs: 129 active+clean; 841 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-08-20T01:23:32.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:32 smithi142 ceph-mon[21799]: pgmap v842: 129 pgs: 129 active+clean; 841 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-08-20T01:23:33.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:32 smithi138 ceph-mon[18076]: pgmap v842: 129 pgs: 129 active+clean; 841 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 4.2 MiB/s wr, 468 op/s 2024-08-20T01:23:33.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 33285 21.79 MB/sec warmup 90 sec latency 2416.116 ms 2024-08-20T01:23:34.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 34169 22.28 MB/sec warmup 91 sec latency 3416.175 ms 2024-08-20T01:23:34.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:34 smithi142 ceph-mon[21799]: pgmap v843: 129 pgs: 129 active+clean; 844 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-08-20T01:23:34.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:34 smithi142 ceph-mon[21799]: osdmap e107: 12 total, 12 up, 12 in 2024-08-20T01:23:35.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:34 smithi138 ceph-mon[18076]: pgmap v843: 129 pgs: 129 active+clean; 844 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-08-20T01:23:35.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:34 smithi138 ceph-mon[18076]: osdmap e107: 12 total, 12 up, 12 in 2024-08-20T01:23:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:34 smithi167 ceph-mon[21965]: pgmap v843: 129 pgs: 129 active+clean; 844 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.6 MiB/s wr, 499 op/s 2024-08-20T01:23:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:34 smithi167 ceph-mon[21965]: osdmap e107: 12 total, 12 up, 12 in 2024-08-20T01:23:35.264 INFO:tasks.workunit.client.0.smithi138.stdout: 10 34639 22.14 MB/sec warmup 92 sec latency 4416.231 ms 2024-08-20T01:23:36.266 INFO:tasks.workunit.client.0.smithi138.stdout: 10 35241 22.18 MB/sec warmup 93 sec latency 5418.225 ms 2024-08-20T01:23:36.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:36 smithi142 ceph-mon[21799]: pgmap v845: 129 pgs: 129 active+clean; 858 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 576 op/s 2024-08-20T01:23:37.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:36 smithi138 ceph-mon[18076]: pgmap v845: 129 pgs: 129 active+clean; 858 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 576 op/s 2024-08-20T01:23:37.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:36 smithi167 ceph-mon[21965]: pgmap v845: 129 pgs: 129 active+clean; 858 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.7 MiB/s wr, 576 op/s 2024-08-20T01:23:37.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 36500 22.75 MB/sec warmup 94 sec latency 5508.026 ms 2024-08-20T01:23:38.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 36852 22.78 MB/sec warmup 95 sec latency 1904.585 ms 2024-08-20T01:23:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 -- 172.21.15.138:0/3053971447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810a610 msgr2=0x7f5cf810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 --2- 172.21.15.138:0/3053971447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810a610 0x7f5cf810aa50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009860 tx=0x7f5ce402a340 comp rx=0 tx=0).stop 2024-08-20T01:23:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 -- 172.21.15.138:0/3053971447 shutdown_connections 2024-08-20T01:23:38.894 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 --2- 172.21.15.138:0/3053971447 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf810af90 0x7f5cf8111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 --2- 172.21.15.138:0/3053971447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810a610 0x7f5cf810aa50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 --2- 172.21.15.138:0/3053971447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf8104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 -- 172.21.15.138:0/3053971447 >> 172.21.15.138:0/3053971447 conn(0x7f5cf8100610 msgr2=0x7f5cf8102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 -- 172.21.15.138:0/3053971447 shutdown_connections 2024-08-20T01:23:38.895 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.890+0000 7f5cfe800640 1 -- 172.21.15.138:0/3053971447 wait complete. 2024-08-20T01:23:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 Processor -- start 2024-08-20T01:23:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 -- start start 2024-08-20T01:23:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf810a610 0x7f5cf819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.896 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810af90 0x7f5cf81a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8114c20 con 0x7f5cf810af90 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8114aa0 con 0x7f5cf8104c10 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cfe800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf8114da0 con 0x7f5cf810a610 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35344/0 (socket says 172.21.15.138:35344) 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 -- 172.21.15.138:0/2496332817 learned_addr learned my addr 172.21.15.138:0/2496332817 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810af90 0x7f5cf81a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 -- 172.21.15.138:0/2496332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf810a610 msgr2=0x7f5cf819b6e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf810a610 0x7f5cf819b6e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 -- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810af90 msgr2=0x7f5cf81a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810af90 0x7f5cf81a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.897 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.895+0000 7f5cf7400640 1 -- 172.21.15.138:0/2496332817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81a1240 con 0x7f5cf8104c10 2024-08-20T01:23:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800e9c0 tx=0x7f5ce800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cefe00640 1 -- 172.21.15.138:0/2496332817 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800cde0 con 0x7f5cf8104c10 2024-08-20T01:23:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cefe00640 1 -- 172.21.15.138:0/2496332817 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce80035f0 con 0x7f5cf8104c10 2024-08-20T01:23:38.898 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81a1530 con 0x7f5cf8104c10 2024-08-20T01:23:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cefe00640 1 -- 172.21.15.138:0/2496332817 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8010690 con 0x7f5cf8104c10 2024-08-20T01:23:38.899 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf81a7bf0 con 0x7f5cf8104c10 2024-08-20T01:23:38.900 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5cefe00640 1 -- 172.21.15.138:0/2496332817 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5ce8004000 con 0x7f5cf8104c10 2024-08-20T01:23:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 Processor -- start 2024-08-20T01:23:38.901 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 -- start start 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b00fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b0125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf9af79210 con 0x55cf9b003930 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf9af68710 con 0x55cf9b0101e0 2024-08-20T01:23:38.902 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.897+0000 7f5d017ce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cf9af6c750 con 0x55cf9b004250 2024-08-20T01:23:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b00fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33380/0 (socket says 172.21.15.138:33380) 2024-08-20T01:23:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 -- 172.21.15.138:0/4187922609 learned_addr learned my addr 172.21.15.138:0/4187922609 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:38.903 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b0125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cefe00640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 0x7f5cdc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/4187922609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 msgr2=0x55cf9b0125d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.904 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b0125d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/4187922609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 msgr2=0x55cf9b003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/4187922609 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9af34de0 con 0x55cf9b004250 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cefe00640 1 -- 172.21.15.138:0/2496332817 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f5ce80e47a0 con 0x7f5cf8104c10 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b00fca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80e1af0 tx=0x7f5ce806fdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60770550ca2165f6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 0x7f5cdc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cecc00640 1 -- 172.21.15.138:0/4187922609 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80aad80 con 0x55cf9b004250 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cecc00640 1 -- 172.21.15.138:0/4187922609 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce80aaac0 con 0x55cf9b004250 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cecc00640 1 -- 172.21.15.138:0/4187922609 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80042f0 con 0x55cf9b004250 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- 172.21.15.138:0/4187922609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 msgr2=0x55cf9b00fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 0x7f5cdc0b6200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00107d0 tx=0x7f5cf0004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.905 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b00fca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80e1af0 tx=0x7f5ce806fdd0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- 172.21.15.138:0/4187922609 shutdown_connections 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b0125d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b00fca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- 172.21.15.138:0/4187922609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- 172.21.15.138:0/4187922609 >> 172.21.15.138:0/4187922609 conn(0x55cf9b001760 msgr2=0x55cf9b002900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- 172.21.15.138:0/4187922609 shutdown_connections 2024-08-20T01:23:38.906 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- 172.21.15.138:0/4187922609 wait complete. 2024-08-20T01:23:38.907 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- start start 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 Processor -- start 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b00ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cf9aea1ab0 con 0x55cf9b003930 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cf9af0f140 con 0x55cf9b004250 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cf9b012e60 con 0x55cf9b0101e0 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5d017ce740 10 client.0 _ll_register_callbacks cb 0x55cf9af225d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b00ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33392/0 (socket says 172.21.15.138:33392) 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/2650092089 learned_addr learned my addr 172.21.15.138:0/2650092089 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.908 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/2650092089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 msgr2=0x55cf9b00d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 msgr2=0x55cf9b00ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b00ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 -- 172.21.15.138:0/2650092089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9af10170 con 0x55cf9b003930 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80e1ac0 tx=0x7f5ce8078c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383828a5fd096ef3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.902+0000 7f5ce1600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80aa720 con 0x55cf9b003930 2024-08-20T01:23:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f5ce80037a0 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce80aaac0 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5ce80aa900 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 10 client.17283 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cf9accb790 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ce80aaac0 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f5ce80e1d70 con 0x55cf9b003930 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 10 client.17283 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 20 client.17283 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b01fbe0 0x55cf9b021fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 4 client.17283 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.907+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf9af7b160 con 0x55cf9b01fbe0 2024-08-20T01:23:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b01fbe0 0x55cf9b021fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5ce1600640 10 client.17283 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b01fbe0 0x55cf9b021fd0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdfda87338889bf6 server_cookie=26357ca5ea3fa47a in_seq=0 out_seq=0 2024-08-20T01:23:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5ce1600640 1 -- 172.21.15.138:0/2650092089 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf9af7b160 con 0x55cf9b01fbe0 2024-08-20T01:23:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5ce1600640 10 client.17283 handle_command_reply: tid=0 2024-08-20T01:23:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 client.17283 shutdown 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 msgr2=0x55cf9b00f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80e1ac0 tx=0x7f5ce8078c70 comp rx=0 tx=0).stop 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 shutdown_connections 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b01fbe0 0x55cf9b021fd0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b00ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2650092089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b00f9d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5ce1600640 0 client.17283 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 >> 172.21.15.138:0/2650092089 conn(0x55cf9b001760 msgr2=0x55cf9b007300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 shutdown_connections 2024-08-20T01:23:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 1 -- 172.21.15.138:0/2650092089 wait complete. 2024-08-20T01:23:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.909+0000 7f5d017ce740 20 client.17283 trim_cache size 0 max 16384 2024-08-20T01:23:38.924 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:38 smithi138 ceph-mon[18076]: pgmap v846: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-08-20T01:23:38.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 Processor -- start 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 -- start start 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b008470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b002b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b47e0 con 0x55cf9b003930 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b2020 con 0x55cf9b004250 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5d017ce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81afe90 con 0x55cf9b0101e0 2024-08-20T01:23:38.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:35372/0 (socket says 172.21.15.138:35372) 2024-08-20T01:23:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5cf6a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b002b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 -- 172.21.15.138:0/524293165 learned_addr learned my addr 172.21.15.138:0/524293165 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:38.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.936+0000 7f5cf7400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b008470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 -- 172.21.15.138:0/524293165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 msgr2=0x55cf9b002b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b002b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 -- 172.21.15.138:0/524293165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 msgr2=0x55cf9b008470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b008470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 -- 172.21.15.138:0/524293165 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf9ad44620 con 0x55cf9b0101e0 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7400640 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b008470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000fc80 tx=0x7f5cf0018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c7e701fcfef24e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5ce3e00640 1 -- 172.21.15.138:0/524293165 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000fd70 con 0x55cf9b0101e0 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5ce3e00640 1 -- 172.21.15.138:0/524293165 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf001fdf0 con 0x55cf9b0101e0 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5ce3e00640 1 -- 172.21.15.138:0/524293165 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf001cd10 con 0x55cf9b0101e0 2024-08-20T01:23:38.943 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 -- 172.21.15.138:0/524293165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 msgr2=0x55cf9b00c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000fc80 tx=0x7f5cf0018000 comp rx=0 tx=0).stop 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 -- 172.21.15.138:0/524293165 shutdown_connections 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b0101e0 0x55cf9b00c550 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b004250 0x55cf9b002b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 --2- 172.21.15.138:0/524293165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b008470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.941+0000 7f5d017ce740 1 -- 172.21.15.138:0/524293165 >> 172.21.15.138:0/524293165 conn(0x55cf9b001760 msgr2=0x55cf9b002440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:38.944 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.944+0000 7f5d017ce740 1 -- 172.21.15.138:0/524293165 shutdown_connections 2024-08-20T01:23:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.944+0000 7f5d017ce740 1 -- 172.21.15.138:0/524293165 wait complete. 2024-08-20T01:23:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.944+0000 7f5d017ce740 1 -- start start 2024-08-20T01:23:38.949 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.944+0000 7f5d017ce740 1 Processor -- start 2024-08-20T01:23:38.950 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b014e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b015370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b04e0 con 0x55cf9b003930 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b1950 con 0x55cf9b0101e0 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf81b0bb0 con 0x55cf9b004250 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 10 client.0 _ll_register_callbacks cb 0x7f5cf0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:35386/0 (socket says 172.21.15.138:35386) 2024-08-20T01:23:38.951 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7e00640 1 -- 172.21.15.138:0/2284988784 learned_addr learned my addr 172.21.15.138:0/2284988784 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b015370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.954 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b014e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 -- 172.21.15.138:0/2284988784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 msgr2=0x55cf9b015370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b015370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 -- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 msgr2=0x55cf9b082590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 -- 172.21.15.138:0/2284988784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf9b082cd0 con 0x55cf9b003930 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7e00640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5cf7400640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b014e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c6a0 tx=0x7f5ce80039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c464b75e4ef5751c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5ce1600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80aa720 con 0x55cf9b003930 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f5ce80e1d70 con 0x55cf9b003930 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cf001fdf0 con 0x55cf9b003930 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5ce80aad80 con 0x55cf9b003930 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 10 client.17289 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:38.955 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.949+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55cf9b08a070 con 0x55cf9b003930 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5cf001fdf0 con 0x55cf9b003930 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f5ce80e4320 con 0x55cf9b003930 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 10 client.17289 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 10 client.17289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 20 client.17289 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b08a570 0x55cf9b08c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 4 client.17289 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:23:38.956 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf9b00fd60 con 0x55cf9b08a570 2024-08-20T01:23:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b08a570 0x55cf9b08c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.954+0000 7f5ce1600640 10 client.17289 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:38.956+0000 7f5cf6a00640 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b08a570 0x55cf9b08c990 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d449f897379c982 server_cookie=6f96ef447e12f202 in_seq=0 out_seq=0 2024-08-20T01:23:38.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:38 smithi142 ceph-mon[21799]: pgmap v846: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-08-20T01:23:39.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:38 smithi167 ceph-mon[21965]: pgmap v846: 129 pgs: 129 active+clean; 877 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.1 MiB/s wr, 598 op/s 2024-08-20T01:23:39.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 36902 22.57 MB/sec warmup 96 sec latency 2904.647 ms 2024-08-20T01:23:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.573+0000 7f5ce1600640 1 -- 172.21.15.138:0/2284988784 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf9b00fd60 con 0x55cf9b08a570 2024-08-20T01:23:39.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.573+0000 7f5ce1600640 10 client.17289 handle_command_reply: tid=0 2024-08-20T01:23:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.573+0000 7f5d017ce740 1 client.17289 shutdown 2024-08-20T01:23:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 msgr2=0x55cf9b014e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b014e10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800c6a0 tx=0x7f5ce80039d0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 shutdown_connections 2024-08-20T01:23:39.577 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cf9b08a570 0x55cf9b08c990 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf9b0101e0 0x55cf9b082590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cf9b004250 0x55cf9b015370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 --2- 172.21.15.138:0/2284988784 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cf9b003930 0x55cf9b014e10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5ce1600640 0 client.17289 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 >> 172.21.15.138:0/2284988784 conn(0x55cf9b001760 msgr2=0x55cf9b008340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 shutdown_connections 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 1 -- 172.21.15.138:0/2284988784 wait complete. 2024-08-20T01:23:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.575+0000 7f5d017ce740 20 client.17289 trim_cache size 0 max 16384 2024-08-20T01:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:23:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 msgr2=0x7f5cdc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 0x7f5cdc0b6200 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00107d0 tx=0x7f5cf0004a40 comp rx=0 tx=0).stop 2024-08-20T01:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 msgr2=0x7f5cf819b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800e9c0 tx=0x7f5ce800ee80 comp rx=0 tx=0).stop 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 shutdown_connections 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5cdc0b3d40 0x7f5cdc0b6200 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5cf810af90 0x7f5cf81a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5cf810a610 0x7f5cf819b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 --2- 172.21.15.138:0/2496332817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5cf8104c10 0x7f5cf819b1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 >> 172.21.15.138:0/2496332817 conn(0x7f5cf8100610 msgr2=0x7f5cf8102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 shutdown_connections 2024-08-20T01:23:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:39.580+0000 7f5cfe800640 1 -- 172.21.15.138:0/2496332817 wait complete. 2024-08-20T01:23:40.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 36936 22.36 MB/sec warmup 97 sec latency 3904.749 ms 2024-08-20T01:23:40.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:40 smithi142 ceph-mon[21799]: pgmap v847: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 539 op/s 2024-08-20T01:23:41.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:40 smithi138 ceph-mon[18076]: pgmap v847: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 539 op/s 2024-08-20T01:23:41.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:40 smithi167 ceph-mon[21965]: pgmap v847: 129 pgs: 129 active+clean; 879 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.8 MiB/s wr, 539 op/s 2024-08-20T01:23:41.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 36936 22.13 MB/sec warmup 98 sec latency 4904.794 ms 2024-08-20T01:23:42.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37035 21.95 MB/sec warmup 99 sec latency 5799.449 ms 2024-08-20T01:23:42.989 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:42 smithi138 ceph-mon[18076]: pgmap v848: 129 pgs: 129 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-08-20T01:23:42.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:42 smithi142 ceph-mon[21799]: pgmap v848: 129 pgs: 129 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-08-20T01:23:43.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:42 smithi167 ceph-mon[21965]: pgmap v848: 129 pgs: 129 active+clean; 893 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.0 MiB/s wr, 648 op/s 2024-08-20T01:23:43.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37220 21.80 MB/sec warmup 100 sec latency 3905.921 ms 2024-08-20T01:23:44.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37228 21.58 MB/sec warmup 101 sec latency 1393.069 ms 2024-08-20T01:23:44.977 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:44 smithi138 ceph-mon[18076]: pgmap v849: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 690 op/s 2024-08-20T01:23:45.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:44 smithi167 ceph-mon[21965]: pgmap v849: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 690 op/s 2024-08-20T01:23:45.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:44 smithi142 ceph-mon[21799]: pgmap v849: 129 pgs: 129 active+clean; 890 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 690 op/s 2024-08-20T01:23:45.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37228 21.37 MB/sec warmup 102 sec latency 2393.138 ms 2024-08-20T01:23:46.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37228 21.17 MB/sec warmup 103 sec latency 3393.180 ms 2024-08-20T01:23:46.969 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:46 smithi138 ceph-mon[18076]: pgmap v850: 129 pgs: 129 active+clean; 911 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 751 op/s 2024-08-20T01:23:47.043 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:46 smithi142 ceph-mon[21799]: pgmap v850: 129 pgs: 129 active+clean; 911 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 751 op/s 2024-08-20T01:23:47.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:46 smithi167 ceph-mon[21965]: pgmap v850: 129 pgs: 129 active+clean; 911 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 12 MiB/s wr, 751 op/s 2024-08-20T01:23:47.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37230 20.96 MB/sec warmup 104 sec latency 4393.232 ms 2024-08-20T01:23:48.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37230 20.76 MB/sec warmup 105 sec latency 5393.369 ms 2024-08-20T01:23:48.973 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:48 smithi138 ceph-mon[18076]: pgmap v851: 129 pgs: 129 active+clean; 929 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 666 op/s 2024-08-20T01:23:49.016 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:48 smithi167 ceph-mon[21965]: pgmap v851: 129 pgs: 129 active+clean; 929 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 666 op/s 2024-08-20T01:23:49.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:48 smithi142 ceph-mon[21799]: pgmap v851: 129 pgs: 129 active+clean; 929 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 12 MiB/s wr, 666 op/s 2024-08-20T01:23:49.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37230 20.57 MB/sec warmup 106 sec latency 6393.441 ms 2024-08-20T01:23:50.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37230 20.37 MB/sec warmup 107 sec latency 7393.481 ms 2024-08-20T01:23:50.987 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:50 smithi138 ceph-mon[18076]: pgmap v852: 129 pgs: 129 active+clean; 928 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-08-20T01:23:51.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:50 smithi167 ceph-mon[21965]: pgmap v852: 129 pgs: 129 active+clean; 928 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-08-20T01:23:51.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:50 smithi142 ceph-mon[21799]: pgmap v852: 129 pgs: 129 active+clean; 928 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 11 MiB/s wr, 565 op/s 2024-08-20T01:23:51.268 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37230 20.19 MB/sec warmup 108 sec latency 8393.552 ms 2024-08-20T01:23:52.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37285 20.01 MB/sec warmup 109 sec latency 9393.622 ms 2024-08-20T01:23:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/1315336213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c105970 msgr2=0x7f4e6c10f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- 172.21.15.138:0/1315336213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c105970 0x7f4e6c10f3b0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ca0 tx=0x7f4e6402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/1315336213 shutdown_connections 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- 172.21.15.138:0/1315336213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c105970 0x7f4e6c10f3b0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- 172.21.15.138:0/1315336213 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- 172.21.15.138:0/1315336213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c06ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/1315336213 >> 172.21.15.138:0/1315336213 conn(0x7f4e6c077270 msgr2=0x7f4e6c077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/1315336213 shutdown_connections 2024-08-20T01:23:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/1315336213 wait complete. 2024-08-20T01:23:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 Processor -- start 2024-08-20T01:23:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- start start 2024-08-20T01:23:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c19af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.714 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c105970 0x7f4e6c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c112850 con 0x7f4e6c104fd0 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1126d0 con 0x7f4e6c105970 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1129d0 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53730/0 (socket says 172.21.15.138:53730) 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c19af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e63e00640 1 -- 172.21.15.138:0/993440180 learned_addr learned my addr 172.21.15.138:0/993440180 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6be00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c105970 0x7f4e6c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 -- 172.21.15.138:0/993440180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c105970 msgr2=0x7f4e6c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c105970 0x7f4e6c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 -- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 msgr2=0x7f4e6c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.715 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 -- 172.21.15.138:0/993440180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6c1a0fd0 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e63e00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e6b400640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c19af30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9f0 tx=0x7f4e5c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e68c00640 1 -- 172.21.15.138:0/993440180 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c00cd80 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6c1a12c0 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e68c00640 1 -- 172.21.15.138:0/993440180 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e5c0035f0 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e68c00640 1 -- 172.21.15.138:0/993440180 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c010690 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.716 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.711+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6c1a7b60 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.718+0000 7f4e751c3740 1 Processor -- start 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.718+0000 7f4e751c3740 1 -- start start 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d7f44c210 con 0x560d7f4e33c0 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d7f43b710 con 0x560d7f4d6b10 2024-08-20T01:23:52.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560d7f43f750 con 0x560d7f4d7430 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48644/0 (socket says 172.21.15.138:48644) 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53732/0 (socket says 172.21.15.138:53732) 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 -- 172.21.15.138:0/2603949110 learned_addr learned my addr 172.21.15.138:0/2603949110 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 -- 172.21.15.138:0/2603949110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 msgr2=0x560d7f4e2e80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e2e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 -- 172.21.15.138:0/2603949110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 msgr2=0x560d7f4e57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 -- 172.21.15.138:0/2603949110 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7f407de0 con 0x560d7f4d6b10 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e68c00640 1 -- 172.21.15.138:0/993440180 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4e5c0037a0 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ca0 tx=0x7f4e640093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666555e77441fcb2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e5be00640 1 -- 172.21.15.138:0/2603949110 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e64031cf0 con 0x560d7f4d6b10 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e5be00640 1 -- 172.21.15.138:0/2603949110 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e64036030 con 0x560d7f4d6b10 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e5be00640 1 -- 172.21.15.138:0/2603949110 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e640042a0 con 0x560d7f4d6b10 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/2603949110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 msgr2=0x560d7f4d6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4e64009ca0 tx=0x7f4e640093e0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/2603949110 shutdown_connections 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- 172.21.15.138:0/2603949110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4d6ef0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/2603949110 >> 172.21.15.138:0/2603949110 conn(0x560d7f4d4940 msgr2=0x560d7f4d5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e68c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 0x7f4e500b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/2603949110 shutdown_connections 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e68c00640 1 -- 172.21.15.138:0/993440180 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f4e5c0e4610 con 0x7f4e6c06a7a0 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/2603949110 wait complete. 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e63e00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 0x7f4e500b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- start start 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e63e00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 0x7f4e500b6280 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009640 tx=0x7f4e54005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 Processor -- start 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x560d7f374ab0 con 0x560d7f4e33c0 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x560d7f3e2140 con 0x560d7f4d6b10 2024-08-20T01:23:52.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x560d7f4e5ec0 con 0x560d7f4d7430 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 10 client.0 _ll_register_callbacks cb 0x560d7f3f55d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48650/0 (socket says 172.21.15.138:48650) 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 -- 172.21.15.138:0/1519104352 learned_addr learned my addr 172.21.15.138:0/1519104352 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 -- 172.21.15.138:0/1519104352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 msgr2=0x560d7f4e0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 -- 172.21.15.138:0/1519104352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 msgr2=0x560d7f4e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 -- 172.21.15.138:0/1519104352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7f3e3170 con 0x560d7f4d6b10 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e6b400640 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9c0 tx=0x7f4e5c003cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3650b37effdcfc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:23:52.726 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0af240 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f4e5c0e1c40 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e64036030 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0af480 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 10 client.27329 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560d7f19e790 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e64036030 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f4e5c0e4220 con 0x560d7f4d6b10 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 10 client.27329 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 10 client.27329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 20 client.27329 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f4f2f40 0x560d7f4f5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 4 client.27329 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.719+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d7f44e160 con 0x560d7f4f2f40 2024-08-20T01:23:52.727 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e63e00640 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f4f2f40 0x560d7f4f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.728 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e63e00640 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f4f2f40 0x560d7f4f5330 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18985ced1399c6b server_cookie=3fc6db1d319c33bf in_seq=0 out_seq=0 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e58c00640 10 client.27329 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e58c00640 1 -- 172.21.15.138:0/1519104352 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560d7f44e160 con 0x560d7f4f2f40 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e58c00640 10 client.27329 handle_command_reply: tid=0 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 client.27329 shutdown 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 msgr2=0x560d7f4e2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9c0 tx=0x7f4e5c003cf0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 shutdown_connections 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f4f2f40 0x560d7f4f5330 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d7430 0x560d7f4e0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e58c00640 0 client.27329 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 --2- 172.21.15.138:0/1519104352 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d6b10 0x560d7f4e2c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 >> 172.21.15.138:0/1519104352 conn(0x560d7f4d4940 msgr2=0x560d7f4da4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 shutdown_connections 2024-08-20T01:23:52.729 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 1 -- 172.21.15.138:0/1519104352 wait complete. 2024-08-20T01:23:52.730 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.725+0000 7f4e751c3740 20 client.27329 trim_cache size 0 max 16384 2024-08-20T01:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 Processor -- start 2024-08-20T01:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- start start 2024-08-20T01:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d6b10 0x560d7f4d41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4fb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b43d0 con 0x560d7f4e33c0 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b1c10 con 0x560d7f4d7430 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1afa80 con 0x560d7f4d6b10 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48660/0 (socket says 172.21.15.138:48660) 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 -- 172.21.15.138:0/110809978 learned_addr learned my addr 172.21.15.138:0/110809978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d6b10 0x560d7f4d41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 -- 172.21.15.138:0/110809978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d6b10 msgr2=0x560d7f4d41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d6b10 0x560d7f4d41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 -- 172.21.15.138:0/110809978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 msgr2=0x560d7f4fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 -- 172.21.15.138:0/110809978 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d7f217620 con 0x560d7f4d7430 2024-08-20T01:23:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009fd0 tx=0x7f4e5401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d692fa093c728fd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e5b400640 1 -- 172.21.15.138:0/110809978 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e54005020 con 0x560d7f4d7430 2024-08-20T01:23:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e5b400640 1 -- 172.21.15.138:0/110809978 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e540047a0 con 0x560d7f4d7430 2024-08-20T01:23:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e5b400640 1 -- 172.21.15.138:0/110809978 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5402b6d0 con 0x560d7f4d7430 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/110809978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 msgr2=0x560d7f4fabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009fd0 tx=0x7f4e5401a000 comp rx=0 tx=0).stop 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/110809978 shutdown_connections 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4e33c0 0x560d7f4fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4fabf0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- 172.21.15.138:0/110809978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4d6b10 0x560d7f4d41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/110809978 >> 172.21.15.138:0/110809978 conn(0x560d7f4d4940 msgr2=0x560d7f4db650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/110809978 shutdown_connections 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/110809978 wait complete. 2024-08-20T01:23:52.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- start start 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 Processor -- start 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4eb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4e33c0 0x560d7f4dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b00d0 con 0x560d7f4d6b10 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b1540 con 0x560d7f4d7430 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6c1b07a0 con 0x560d7f4e33c0 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 10 client.0 _ll_register_callbacks cb 0x7f4e5c0056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:53768/0 (socket says 172.21.15.138:53768) 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 -- 172.21.15.138:0/705806257 learned_addr learned my addr 172.21.15.138:0/705806257 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e63e00640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4eb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6be00640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4e33c0 0x560d7f4dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 -- 172.21.15.138:0/705806257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4e33c0 msgr2=0x560d7f4dc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4e33c0 0x560d7f4dc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 -- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 msgr2=0x560d7f4eb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4eb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 -- 172.21.15.138:0/705806257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d7f4dcf30 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e6b400640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0e45e0 tx=0x7f4e5c003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd8f244d0b11e3a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e58c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c08f020 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f4e5c0e1c40 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e540047a0 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f4e5c0aa5f0 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 10 client.17307 fetch_fsmap learned FSMap version 48 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560d7f55d530 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.741+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e5c0b5020 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f4e5c0a9b50 con 0x560d7f4d6b10 2024-08-20T01:23:52.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 10 client.17307 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 10 client.17307 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 20 client.17307 populate_metadata read hostname 'smithi138' 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f55db40 0x560d7f55ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 4 client.17307 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d7f449c00 con 0x560d7f55db40 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e63e00640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f55db40 0x560d7f55ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e63e00640 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f55db40 0x560d7f55ff60 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1658a577d615ab server_cookie=deec63eb6779d9ba in_seq=0 out_seq=0 2024-08-20T01:23:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.747+0000 7f4e58c00640 10 client.17307 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e58c00640 1 -- 172.21.15.138:0/705806257 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d7f449c00 con 0x560d7f55db40 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e58c00640 10 client.17307 handle_command_reply: tid=0 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 client.17307 shutdown 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 msgr2=0x560d7f4eb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0e45e0 tx=0x7f4e5c003d90 comp rx=0 tx=0).stop 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 shutdown_connections 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x560d7f55db40 0x560d7f55ff60 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x560d7f4e33c0 0x560d7f4dc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d7f4d7430 0x560d7f4eb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 --2- 172.21.15.138:0/705806257 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x560d7f4d6b10 0x560d7f4eb150 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e58c00640 0 client.17307 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 >> 172.21.15.138:0/705806257 conn(0x560d7f4d4940 msgr2=0x560d7f4daca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 shutdown_connections 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 1 -- 172.21.15.138:0/705806257 wait complete. 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e751c3740 20 client.17307 trim_cache size 0 max 16384 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:23:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 msgr2=0x7f4e500b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 0x7f4e500b6280 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4e54009640 tx=0x7f4e54005f90 comp rx=0 tx=0).stop 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 msgr2=0x7f4e6c19af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c19af30 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c00e9f0 tx=0x7f4e5c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 shutdown_connections 2024-08-20T01:23:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4e500b3dc0 0x7f4e500b6280 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4e6c105970 0x7f4e6c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e6c104fd0 0x7f4e6c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 --2- 172.21.15.138:0/993440180 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4e6c06a7a0 0x7f4e6c19af30 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 >> 172.21.15.138:0/993440180 conn(0x7f4e6c077270 msgr2=0x7f4e6c1081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 shutdown_connections 2024-08-20T01:23:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:23:52.818+0000 7f4e70c00640 1 -- 172.21.15.138:0/993440180 wait complete. 2024-08-20T01:23:53.041 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:52 smithi138 ceph-mon[18076]: pgmap v853: 129 pgs: 129 active+clean; 897 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-20T01:23:53.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:52 smithi167 ceph-mon[21965]: pgmap v853: 129 pgs: 129 active+clean; 897 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-20T01:23:53.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:52 smithi142 ceph-mon[21799]: pgmap v853: 129 pgs: 129 active+clean; 897 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 11 MiB/s wr, 628 op/s 2024-08-20T01:23:53.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37285 19.82 MB/sec warmup 110 sec latency 10393.694 ms 2024-08-20T01:23:54.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37362 19.70 MB/sec warmup 111 sec latency 11393.738 ms 2024-08-20T01:23:55.002 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:54 smithi138 ceph-mon[18076]: pgmap v854: 129 pgs: 129 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-08-20T01:23:55.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:54 smithi167 ceph-mon[21965]: pgmap v854: 129 pgs: 129 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-08-20T01:23:55.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:54 smithi142 ceph-mon[21799]: pgmap v854: 129 pgs: 129 active+clean; 891 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 7.2 MiB/s wr, 356 op/s 2024-08-20T01:23:55.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 37722 19.74 MB/sec warmup 112 sec latency 12138.137 ms 2024-08-20T01:23:56.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 38048 19.66 MB/sec warmup 113 sec latency 12794.116 ms 2024-08-20T01:23:57.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:56 smithi138 ceph-mon[18076]: pgmap v855: 129 pgs: 129 active+clean; 892 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.9 MiB/s wr, 320 op/s 2024-08-20T01:23:57.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:56 smithi167 ceph-mon[21965]: pgmap v855: 129 pgs: 129 active+clean; 892 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.9 MiB/s wr, 320 op/s 2024-08-20T01:23:57.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:56 smithi142 ceph-mon[21799]: pgmap v855: 129 pgs: 129 active+clean; 892 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.9 MiB/s wr, 320 op/s 2024-08-20T01:23:57.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 39031 19.90 MB/sec warmup 114 sec latency 65.390 ms 2024-08-20T01:23:58.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 40232 20.45 MB/sec warmup 115 sec latency 62.762 ms 2024-08-20T01:23:59.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:58 smithi138 ceph-mon[18076]: pgmap v856: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-08-20T01:23:59.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:58 smithi167 ceph-mon[21965]: pgmap v856: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-08-20T01:23:59.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:58 smithi142 ceph-mon[21799]: pgmap v856: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 277 op/s 2024-08-20T01:23:59.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 40918 20.53 MB/sec warmup 116 sec latency 132.614 ms 2024-08-20T01:24:00.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:23:59 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:24:00.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:23:59 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:24:00.270 INFO:tasks.workunit.client.0.smithi138.stdout: 10 41508 20.55 MB/sec warmup 117 sec latency 79.893 ms 2024-08-20T01:24:00.291 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:23:59 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:24:01.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: pgmap v857: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-08-20T01:24:01.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:24:01.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:00 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:24:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: pgmap v857: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-08-20T01:24:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:24:01.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:00 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:24:01.269 INFO:tasks.workunit.client.0.smithi138.stdout: 10 42395 20.69 MB/sec warmup 118 sec latency 175.588 ms 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: pgmap v857: 129 pgs: 129 active+clean; 889 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 164 op/s 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-08-20T01:24:01.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:24:01.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:00 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:24:01.644 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.6 2024-08-20T01:24:01.645 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:24:02.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:01 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:24:02.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:01 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:24:02.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:24:02.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:01 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:24:02.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:01 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:24:02.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:24:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:01 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi138 to 176.4M 2024-08-20T01:24:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:01 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi138 to 185021132: error parsing value: Value '185021132' is below minimum 939524096 2024-08-20T01:24:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:24:02.273 INFO:tasks.workunit.client.0.smithi138.stdout: 10 42438 20.55 MB/sec warmup 119 sec latency 326.108 ms 2024-08-20T01:24:02.285 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:24:03.195 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:02 smithi138 ceph-mon[18076]: pgmap v858: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-08-20T01:24:03.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:02 smithi167 ceph-mon[21965]: pgmap v858: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-08-20T01:24:03.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:02 smithi142 ceph-mon[21799]: pgmap v858: 129 pgs: 129 active+clean; 873 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 2.8 MiB/s wr, 313 op/s 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.573+0000 7f396d800640 1 -- 172.21.15.138:0/3790924131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396810a6c0 msgr2=0x7f3968112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.573+0000 7f396d800640 1 --2- 172.21.15.138:0/3790924131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396810a6c0 0x7f3968112d30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3958009900 tx=0x7f395802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 -- 172.21.15.138:0/3790924131 shutdown_connections 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 --2- 172.21.15.138:0/3790924131 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3968113300 0x7f3968115710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 --2- 172.21.15.138:0/3790924131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f396810a6c0 0x7f3968112d30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 --2- 172.21.15.138:0/3790924131 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968109cc0 0x7f396810a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 -- 172.21.15.138:0/3790924131 >> 172.21.15.138:0/3790924131 conn(0x7f3968078610 msgr2=0x7f396807aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 -- 172.21.15.138:0/3790924131 shutdown_connections 2024-08-20T01:24:03.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.574+0000 7f396d800640 1 -- 172.21.15.138:0/3790924131 wait complete. 2024-08-20T01:24:03.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.575+0000 7f396d800640 1 Processor -- start 2024-08-20T01:24:03.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 -- start start 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 0x7f396819f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 0x7f39681a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f3968118f70 con 0x7f3968113300 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f3968118df0 con 0x7f3968109cc0 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f396d800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f39681190f0 con 0x7f396810a6c0 2024-08-20T01:24:03.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f3967400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f3967400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58474/0 (socket says 172.21.15.138:58474) 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.576+0000 7f3967400640 1 -- 172.21.15.138:0/2859366318 learned_addr learned my addr 172.21.15.138:0/2859366318 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f396ce00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 0x7f39681a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967400640 1 -- 172.21.15.138:0/2859366318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 msgr2=0x7f396819f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967e00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 0x7f396819f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967400640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 0x7f396819f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967400640 1 -- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 msgr2=0x7f39681a4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967400640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 0x7f39681a4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:03.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f3967400640 1 -- 172.21.15.138:0/2859366318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39681a5430 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.577+0000 7f396ce00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 0x7f39681a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f3967e00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 0x7f396819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f3967400640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3958005ab0 tx=0x7f3958003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395803f070 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3958031d60 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3958035030 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39681a56c0 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.578+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39681abde0 con 0x7f396810a6c0 2024-08-20T01:24:03.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.579+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396810a0a0 con 0x7f396810a6c0 2024-08-20T01:24:03.582 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.582+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f3958042070 con 0x7f396810a6c0 2024-08-20T01:24:03.583 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.584+0000 7f3964c00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 0x7f394c0b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:03.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.584+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9453+0+0 (secure 0 0 0) 0x7f3958105110 con 0x7f396810a6c0 2024-08-20T01:24:03.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.584+0000 7f3967e00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 0x7f394c0b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:03.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.585+0000 7f3967e00640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 0x7f394c0b6320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3950009b40 tx=0x7f3950005e50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:03.585 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:03.585+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f39580cc020 con 0x7f396810a6c0 2024-08-20T01:24:04.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.015+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39681ac0d0 con 0x7f396810a6c0 2024-08-20T01:24:04.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.015+0000 7f3964c00640 1 -- 172.21.15.138:0/2859366318 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11939 (secure 0 0 0) 0x7f39580cb920 con 0x7f396810a6c0 2024-08-20T01:24:04.016 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:24:04.016 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 48 2024-08-20T01:24:04.016 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":48,"btime":"2024-08-20T01:22:21:451368+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:22:21.168170+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:24:04.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 msgr2=0x7f394c0b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 0x7f394c0b6320 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3950009b40 tx=0x7f3950005e50 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 msgr2=0x7f396819f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3958005ab0 tx=0x7f3958003c80 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 shutdown_connections 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f394c0b3e60 0x7f394c0b6320 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3968113300 0x7f39681a4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f396810a6c0 0x7f396819f8c0 secure :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3958005ab0 tx=0x7f3958003c80 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 --2- 172.21.15.138:0/2859366318 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3968109cc0 0x7f396819f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.019+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 >> 172.21.15.138:0/2859366318 conn(0x7f3968078610 msgr2=0x7f3968078a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.020+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 shutdown_connections 2024-08-20T01:24:04.020 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:04.020+0000 7f396d800640 1 -- 172.21.15.138:0/2859366318 wait complete. 2024-08-20T01:24:04.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:03 smithi167 ceph-mon[21965]: pgmap v859: 129 pgs: 129 active+clean; 878 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-08-20T01:24:04.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:03 smithi142 ceph-mon[21799]: pgmap v859: 129 pgs: 129 active+clean; 878 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-08-20T01:24:04.271 INFO:tasks.workunit.client.0.smithi138.stdout: 10 43134 38.91 MB/sec execute 1 sec latency 275.298 ms 2024-08-20T01:24:04.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:03 smithi138 ceph-mon[18076]: pgmap v859: 129 pgs: 129 active+clean; 878 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 335 op/s 2024-08-20T01:24:04.924 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 146.6 --await-for 45.0' 2024-08-20T01:24:04.924 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 146.6 --await-for 45.0 2024-08-20T01:24:05.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:04 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2859366318' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:05.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:04 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2859366318' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:05.271 INFO:tasks.workunit.client.0.smithi138.stdout: 10 43726 41.21 MB/sec execute 2 sec latency 325.335 ms 2024-08-20T01:24:05.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:04 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2859366318' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:05.654 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:24:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.034+0000 7f013fe00640 1 -- 172.21.15.138:0/522273777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 msgr2=0x7f0138108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.034+0000 7f013fe00640 1 --2- 172.21.15.138:0/522273777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f0138108830 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0124009900 tx=0x7f012402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.035+0000 7f013fe00640 1 -- 172.21.15.138:0/522273777 shutdown_connections 2024-08-20T01:24:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.035+0000 7f013fe00640 1 --2- 172.21.15.138:0/522273777 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0138108d70 0x7f013810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.035+0000 7f013fe00640 1 --2- 172.21.15.138:0/522273777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f0138108830 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.035+0000 7f013fe00640 1 --2- 172.21.15.138:0/522273777 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f01381029f0 0x7f0138102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.035+0000 7f013fe00640 1 -- 172.21.15.138:0/522273777 >> 172.21.15.138:0/522273777 conn(0x7f01380fe430 msgr2=0x7f0138100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.037+0000 7f013fe00640 1 -- 172.21.15.138:0/522273777 shutdown_connections 2024-08-20T01:24:06.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.037+0000 7f013fe00640 1 -- 172.21.15.138:0/522273777 wait complete. 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.041+0000 7f013fe00640 1 Processor -- start 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.041+0000 7f013fe00640 1 -- start start 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01381029f0 0x7f013819b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f013819b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 0x7f01381a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112a80 con 0x7f0138108d70 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112900 con 0x7f01381083f0 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0138112c00 con 0x7f01381029f0 2024-08-20T01:24:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 0x7f01381a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 0x7f01381a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40570/0 (socket says 172.21.15.138:40570) 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.042+0000 7f013de00640 1 -- 172.21.15.138:0/2121818459 learned_addr learned my addr 172.21.15.138:0/2121818459 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013d400640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01381029f0 0x7f013819b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f013819b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 -- 172.21.15.138:0/2121818459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01381029f0 msgr2=0x7f013819b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01381029f0 0x7f013819b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 -- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 msgr2=0x7f01381a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 0x7f01381a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.043+0000 7f013ca00640 1 -- 172.21.15.138:0/2121818459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01381a1160 con 0x7f01381083f0 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.044+0000 7f013ca00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f013819b600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f012402a9e0 tx=0x7f0124003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.044+0000 7f0136000640 1 -- 172.21.15.138:0/2121818459 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012403f070 con 0x7f01381083f0 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.045+0000 7f0136000640 1 -- 172.21.15.138:0/2121818459 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0124031b50 con 0x7f01381083f0 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.045+0000 7f0136000640 1 -- 172.21.15.138:0/2121818459 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012403f070 con 0x7f01381083f0 2024-08-20T01:24:06.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.045+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01381a13f0 con 0x7f01381083f0 2024-08-20T01:24:06.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.045+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01381a17a0 con 0x7f01381083f0 2024-08-20T01:24:06.051 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.052+0000 7f0142c85740 1 Processor -- start 2024-08-20T01:24:06.053 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.054+0000 7f0136000640 1 -- 172.21.15.138:0/2121818459 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f0124042070 con 0x7f01381083f0 2024-08-20T01:24:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.056+0000 7f0142c85740 1 -- start start 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.056+0000 7f0136000640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 0x7f01200b6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.056+0000 7f0136000640 1 -- 172.21.15.138:0/2121818459 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f0124104570 con 0x7f01381083f0 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.056+0000 7f013de00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 0x7f01200b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.057+0000 7f0142c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x558410551d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.057+0000 7f0142c85740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 0x5584105664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.057+0000 7f013d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x558410551d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f013d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x558410551d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40576/0 (socket says 172.21.15.138:40576) 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f013d400640 1 -- 172.21.15.138:0/155929393 learned_addr learned my addr 172.21.15.138:0/155929393 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f013de00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 0x7f01200b6530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f01300049d0 tx=0x7f0130009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f013ca00640 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 0x5584105664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f0142c85740 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584105669f0 0x558410568de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584104c7210 con 0x558410551930 2024-08-20T01:24:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5584104b6710 con 0x558410552250 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.058+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5584104ba750 con 0x5584105669f0 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013de00640 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584105669f0 0x558410568de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013ca00640 1 -- 172.21.15.138:0/155929393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584105669f0 msgr2=0x558410568de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013ca00640 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584105669f0 0x558410568de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013ca00640 1 -- 172.21.15.138:0/155929393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 msgr2=0x558410551d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013ca00640 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x558410551d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.059+0000 7f013ca00640 1 -- 172.21.15.138:0/155929393 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558410482de0 con 0x558410552250 2024-08-20T01:24:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.060+0000 7f013ca00640 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 0x5584105664b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0124101860 tx=0x7f012402a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2184409fdd4cf248 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.061+0000 7f012a600640 1 -- 172.21.15.138:0/155929393 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124049070 con 0x558410552250 2024-08-20T01:24:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.061+0000 7f012a600640 1 -- 172.21.15.138:0/155929393 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01240cf320 con 0x558410552250 2024-08-20T01:24:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.061+0000 7f012a600640 1 -- 172.21.15.138:0/155929393 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240d3940 con 0x558410552250 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 msgr2=0x5584105664b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 0x5584105664b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0124101860 tx=0x7f012402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 shutdown_connections 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5584105669f0 0x558410568de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410552250 0x5584105664b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 --2- 172.21.15.138:0/155929393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x558410551d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.062+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 >> 172.21.15.138:0/155929393 conn(0x55841054f760 msgr2=0x558410550900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.063+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 shutdown_connections 2024-08-20T01:24:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.063+0000 7f0142c85740 1 -- 172.21.15.138:0/155929393 wait complete. 2024-08-20T01:24:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 -- start start 2024-08-20T01:24:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 Processor -- start 2024-08-20T01:24:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x5584105662a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584105669f0 0x558410563ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5584103efab0 con 0x558410551930 2024-08-20T01:24:06.064 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55841045d140 con 0x5584105669f0 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.064+0000 7f0142c85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x558410569490 con 0x558410552250 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f0142c85740 10 client.0 _ll_register_callbacks cb 0x5584104705d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x5584105662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x5584105662a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40592/0 (socket says 172.21.15.138:40592) 2024-08-20T01:24:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58516/0 (socket says 172.21.15.138:58516) 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013d400640 1 -- 172.21.15.138:0/1034734160 learned_addr learned my addr 172.21.15.138:0/1034734160 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.065+0000 7f013de00640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584105669f0 0x558410563ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 -- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584105669f0 msgr2=0x558410563ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584105669f0 0x558410563ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 -- 172.21.15.138:0/1034734160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 msgr2=0x5584105662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x5584105662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 -- 172.21.15.138:0/1034734160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841045e170 con 0x558410552250 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f013ca00640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558410554ab0 tx=0x7f012402a8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a774e3feb877dc06 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.066+0000 7f011ea00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124049070 con 0x558410552250 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f0124104130 con 0x558410552250 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01240cf320 con 0x558410552250 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f0124101cc0 con 0x558410552250 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f0142c85740 10 client.27319 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:06.067 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.067+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558410219790 con 0x558410552250 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01240cf320 con 0x558410552250 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f01240ca030 con 0x558410552250 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 10 client.27319 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 10 client.27319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 20 client.27319 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55841055d770 0x55841055fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 4 client.27319 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.068+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584104c9160 con 0x55841055d770 2024-08-20T01:24:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.069+0000 7f013d400640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55841055d770 0x55841055fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.070+0000 7f013d400640 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55841055d770 0x55841055fb60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77e7622e9021f02d server_cookie=92e2facc1bf5452e in_seq=0 out_seq=0 2024-08-20T01:24:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.070+0000 7f011ea00640 10 client.27319 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.071+0000 7f011ea00640 1 -- 172.21.15.138:0/1034734160 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584104c9160 con 0x55841055d770 2024-08-20T01:24:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.071+0000 7f011ea00640 10 client.27319 handle_command_reply: tid=0 2024-08-20T01:24:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.071+0000 7f0142c85740 1 client.27319 shutdown 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 msgr2=0x5584105636e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x558410554ab0 tx=0x7f012402a8f0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 shutdown_connections 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55841055d770 0x55841055fb60 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f011ea00640 0 client.27319 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584105669f0 0x558410563ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.072 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x5584105636e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 --2- 172.21.15.138:0/1034734160 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410551930 0x5584105662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 >> 172.21.15.138:0/1034734160 conn(0x55841054f760 msgr2=0x558410555300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.072+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 shutdown_connections 2024-08-20T01:24:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.073+0000 7f0142c85740 1 -- 172.21.15.138:0/1034734160 wait complete. 2024-08-20T01:24:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.073+0000 7f0142c85740 20 client.27319 trim_cache size 0 max 16384 2024-08-20T01:24:06.087 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:05 smithi138 ceph-mon[18076]: pgmap v860: 129 pgs: 129 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-08-20T01:24:06.087 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:05 smithi138 ceph-mon[18076]: osdmap e108: 12 total, 12 up, 12 in 2024-08-20T01:24:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.102+0000 7f0142c85740 1 Processor -- start 2024-08-20T01:24:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.102+0000 7f0142c85740 1 -- start start 2024-08-20T01:24:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x558410588740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 0x558410564d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f01381b8b20 con 0x558410552250 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f01380727f0 con 0x558410551930 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f0142c85740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0138070680 con 0x55841055d770 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f013d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x558410588740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f013ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f013ca00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40594/0 (socket says 172.21.15.138:40594) 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.103+0000 7f013ca00640 1 -- 172.21.15.138:0/1711445343 learned_addr learned my addr 172.21.15.138:0/1711445343 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013de00640 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 0x558410564d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013ca00640 1 -- 172.21.15.138:0/1711445343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 msgr2=0x558410564d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013ca00640 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 0x558410564d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013ca00640 1 -- 172.21.15.138:0/1711445343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 msgr2=0x558410588740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013ca00640 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x558410588740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013ca00640 1 -- 172.21.15.138:0/1711445343 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558410292620 con 0x558410552250 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.104+0000 7f013de00640 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 0x558410564d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.105+0000 7f013ca00640 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f01240092a0 tx=0x7f01240caf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e8fe597d84a8bd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.105+0000 7f0129c00640 1 -- 172.21.15.138:0/1711445343 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124049030 con 0x558410552250 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.105+0000 7f0129c00640 1 -- 172.21.15.138:0/1711445343 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01240c6740 con 0x558410552250 2024-08-20T01:24:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.105+0000 7f0129c00640 1 -- 172.21.15.138:0/1711445343 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01240ca030 con 0x558410552250 2024-08-20T01:24:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.106+0000 7f0142c85740 1 -- 172.21.15.138:0/1711445343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 msgr2=0x558410588c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.106+0000 7f0142c85740 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f01240092a0 tx=0x7f01240caf50 comp rx=0 tx=0).stop 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.107+0000 7f0142c85740 1 -- 172.21.15.138:0/1711445343 shutdown_connections 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.107+0000 7f0142c85740 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55841055d770 0x558410564d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.107+0000 7f0142c85740 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x558410552250 0x558410588c80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.107+0000 7f0142c85740 1 --2- 172.21.15.138:0/1711445343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x558410588740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.107+0000 7f0142c85740 1 -- 172.21.15.138:0/1711445343 >> 172.21.15.138:0/1711445343 conn(0x55841054f760 msgr2=0x558410556470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.109+0000 7f0142c85740 1 -- 172.21.15.138:0/1711445343 shutdown_connections 2024-08-20T01:24:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.109+0000 7f0142c85740 1 -- 172.21.15.138:0/1711445343 wait complete. 2024-08-20T01:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.110+0000 7f0142c85740 1 -- start start 2024-08-20T01:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.110+0000 7f0142c85740 1 Processor -- start 2024-08-20T01:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f0142c85740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f0142c85740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x558410557d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.110 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f0142c85740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f013ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x558410557d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f013ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x558410557d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58532/0 (socket says 172.21.15.138:58532) 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.111+0000 7f013d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f013d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37786/0 (socket says 172.21.15.138:37786) 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f013de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f013de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40604/0 (socket says 172.21.15.138:40604) 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f013ca00640 1 -- 172.21.15.138:0/1180875906 learned_addr learned my addr 172.21.15.138:0/1180875906 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f0138070cd0 con 0x55841055d770 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f0138072120 con 0x558410551930 2024-08-20T01:24:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.112+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f0138071380 con 0x558410552250 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013de00640 1 -- 172.21.15.138:0/1180875906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 msgr2=0x558410557d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013de00640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x558410557d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013de00640 1 -- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 msgr2=0x55841056ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013de00640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013de00640 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558410558b70 con 0x55841055d770 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f013d400640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f0142c85740 10 client.0 _ll_register_callbacks cb 0x7f012c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.113+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01240c6740 con 0x55841055d770 2024-08-20T01:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.114+0000 7f013de00640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0130009aa0 tx=0x7f0130013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f6c4b742001f3e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.115+0000 7f011ea00640 10 client.17328 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:24:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.115+0000 7f011ea00640 1 -- 172.21.15.138:0/1180875906 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f013001dc30 con 0x55841055d770 2024-08-20T01:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.115+0000 7f011ea00640 1 -- 172.21.15.138:0/1180875906 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f0130009050 con 0x55841055d770 2024-08-20T01:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.116+0000 7f011ea00640 1 -- 172.21.15.138:0/1180875906 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f013001dc30 con 0x55841055d770 2024-08-20T01:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.116+0000 7f0142c85740 10 client.17328 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.116+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({fsmap=48}) -- 0x5584105e97a0 con 0x55841055d770 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.116+0000 7f011ea00640 1 -- 172.21.15.138:0/1180875906 <== mon.0 v2:172.21.15.138:3300/0 4 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f0130004b20 con 0x55841055d770 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.116+0000 7f0142c85740 10 client.17328 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f0142c85740 10 client.17328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f0142c85740 20 client.17328 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584105669f0 0x5584105ebe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f0142c85740 4 client.17328 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:24:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584104c4c00 con 0x5584105669f0 2024-08-20T01:24:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.117+0000 7f013d400640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584105669f0 0x5584105ebe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.118+0000 7f013d400640 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584105669f0 0x5584105ebe30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c979874d70436e server_cookie=cc3b86d5ff37ddd7 in_seq=0 out_seq=0 2024-08-20T01:24:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.118+0000 7f011ea00640 10 client.17328 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:05 smithi167 ceph-mon[21965]: pgmap v860: 129 pgs: 129 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-08-20T01:24:06.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:05 smithi167 ceph-mon[21965]: osdmap e108: 12 total, 12 up, 12 in 2024-08-20T01:24:06.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:05 smithi142 ceph-mon[21799]: pgmap v860: 129 pgs: 129 active+clean; 883 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 4.1 MiB/s wr, 425 op/s 2024-08-20T01:24:06.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:05 smithi142 ceph-mon[21799]: osdmap e108: 12 total, 12 up, 12 in 2024-08-20T01:24:06.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 43992 32.73 MB/sec execute 3 sec latency 290.584 ms 2024-08-20T01:24:06.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.532+0000 7f846b200640 1 -- 172.21.15.138:0/510870418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108330 msgr2=0x7f8464108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.532+0000 7f846b200640 1 --2- 172.21.15.138:0/510870418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108330 0x7f8464108770 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8454009860 tx=0x7f845402a340 comp rx=0 tx=0).stop 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.535+0000 7f846b200640 1 -- 172.21.15.138:0/510870418 shutdown_connections 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.535+0000 7f846b200640 1 --2- 172.21.15.138:0/510870418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108cb0 0x7f846410f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.535+0000 7f846b200640 1 --2- 172.21.15.138:0/510870418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108330 0x7f8464108770 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.535+0000 7f846b200640 1 --2- 172.21.15.138:0/510870418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 0x7f8464102d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.535+0000 7f846b200640 1 -- 172.21.15.138:0/510870418 >> 172.21.15.138:0/510870418 conn(0x7f84640fe350 msgr2=0x7f8464100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.536+0000 7f846b200640 1 -- 172.21.15.138:0/510870418 shutdown_connections 2024-08-20T01:24:06.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.536+0000 7f846b200640 1 -- 172.21.15.138:0/510870418 wait complete. 2024-08-20T01:24:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.537+0000 7f846b200640 1 Processor -- start 2024-08-20T01:24:06.537 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.537+0000 7f846b200640 1 -- start start 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 0x7f8464111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 0x7f8464112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 0x7f8464117650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8464110750 con 0x7f8464102920 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84641105d0 con 0x7f8464108330 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f846b200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84641108d0 con 0x7f8464108cb0 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f8469400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 0x7f8464112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f8469400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 0x7f8464112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37808/0 (socket says 172.21.15.138:37808) 2024-08-20T01:24:06.538 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.538+0000 7f8469400640 1 -- 172.21.15.138:0/2082913368 learned_addr learned my addr 172.21.15.138:0/2082913368 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.539+0000 7f8469e00640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 0x7f8464111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.539+0000 7f846a800640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 0x7f8464117650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.540+0000 7f846a800640 1 -- 172.21.15.138:0/2082913368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 msgr2=0x7f8464112320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.540+0000 7f846a800640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 0x7f8464112320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.540 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.540+0000 7f846a800640 1 -- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 msgr2=0x7f8464111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.540+0000 7f846a800640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 0x7f8464111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.540+0000 7f846a800640 1 -- 172.21.15.138:0/2082913368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8464117cf0 con 0x7f8464108cb0 2024-08-20T01:24:06.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.541+0000 7f846a800640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 0x7f8464117650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f845800c970 tx=0x7f845800ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.541 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.542+0000 7f845ea00640 1 -- 172.21.15.138:0/2082913368 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458007bb0 con 0x7f8464108cb0 2024-08-20T01:24:06.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.542+0000 7f845ea00640 1 -- 172.21.15.138:0/2082913368 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8458007d50 con 0x7f8464108cb0 2024-08-20T01:24:06.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.542+0000 7f845ea00640 1 -- 172.21.15.138:0/2082913368 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8458018660 con 0x7f8464108cb0 2024-08-20T01:24:06.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.542+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8464117fe0 con 0x7f8464108cb0 2024-08-20T01:24:06.542 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.542+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641b4690 con 0x7f8464108cb0 2024-08-20T01:24:06.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.554+0000 7f845ea00640 1 -- 172.21.15.138:0/2082913368 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f845800f450 con 0x7f8464108cb0 2024-08-20T01:24:06.558 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.559+0000 7f846ef22740 1 Processor -- start 2024-08-20T01:24:06.559 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.559+0000 7f846ef22740 1 -- start start 2024-08-20T01:24:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f011ea00640 1 -- 172.21.15.138:0/1180875906 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584104c4c00 con 0x5584105669f0 2024-08-20T01:24:06.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f011ea00640 10 client.17328 handle_command_reply: tid=0 2024-08-20T01:24:06.562 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f846ef22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f0142c85740 1 client.17328 shutdown 2024-08-20T01:24:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 msgr2=0x558410558340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0130009aa0 tx=0x7f0130013040 comp rx=0 tx=0).stop 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 shutdown_connections 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5584105669f0 0x5584105ebe30 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55841055d770 0x558410558340 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x558410552250 0x558410557d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 --2- 172.21.15.138:0/1180875906 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558410551930 0x55841056ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 >> 172.21.15.138:0/1180875906 conn(0x55841054f760 msgr2=0x558410564d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f011ea00640 0 client.17328 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.563+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 shutdown_connections 2024-08-20T01:24:06.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f0142c85740 1 -- 172.21.15.138:0/1180875906 wait complete. 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f846ef22740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f8469400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f8469400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37814/0 (socket says 172.21.15.138:37814) 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58552/0 (socket says 172.21.15.138:58552) 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.562+0000 7f846ef22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c80a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f846ef22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c79b670 con 0x55ab7c807e80 2024-08-20T01:24:06.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f846ef22740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c76f7c0 con 0x55ab7c7fbef0 2024-08-20T01:24:06.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c75ecc0 con 0x55ab7c7fb5d0 2024-08-20T01:24:06.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f8469400640 1 -- 172.21.15.138:0/415837234 learned_addr learned my addr 172.21.15.138:0/415837234 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.561+0000 7f845ea00640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 0x7f844c0b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f0142c85740 20 client.17328 trim_cache size 0 max 16384 2024-08-20T01:24:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:24:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:24:06.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:24:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:24:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 msgr2=0x7f01200b6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.566 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.564+0000 7f845ea00640 1 -- 172.21.15.138:0/2082913368 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f84580e42c0 con 0x7f8464108cb0 2024-08-20T01:24:06.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 0x7f01200b6530 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f01300049d0 tx=0x7f0130009500 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f8469400640 1 -- 172.21.15.138:0/415837234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 msgr2=0x55ab7c7fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 msgr2=0x7f013819b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f013819b600 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f012402a9e0 tx=0x7f0124003b20 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 shutdown_connections 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f01200b4070 0x7f01200b6530 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0138108d70 0x7f01381a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f01381083f0 0x7f013819b600 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 --2- 172.21.15.138:0/2121818459 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01381029f0 0x7f013819b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 >> 172.21.15.138:0/2121818459 conn(0x7f01380fe430 msgr2=0x7f01381049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f8469400640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.566+0000 7f8469400640 1 -- 172.21.15.138:0/415837234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 msgr2=0x55ab7c80a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 shutdown_connections 2024-08-20T01:24:06.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f013fe00640 1 -- 172.21.15.138:0/2121818459 wait complete. 2024-08-20T01:24:06.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.565+0000 7f846a800640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c80a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.567+0000 7f8469400640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c80a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.568+0000 7f8469400640 1 -- 172.21.15.138:0/415837234 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7c6f2110 con 0x55ab7c7fbef0 2024-08-20T01:24:06.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.568+0000 7f8469e00640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:06.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.568+0000 7f846a800640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c80a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:06.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.568+0000 7f8469400640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 0x7f844c0b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.568+0000 7f8469400640 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8454035040 tx=0x7f8454030b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da44d2074f61fd0c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.569+0000 7f8453000640 1 -- 172.21.15.138:0/415837234 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454037040 con 0x55ab7c7fbef0 2024-08-20T01:24:06.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.569+0000 7f8453000640 1 -- 172.21.15.138:0/415837234 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8454030900 con 0x55ab7c7fbef0 2024-08-20T01:24:06.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 msgr2=0x55ab7c807940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8454035040 tx=0x7f8454030b50 comp rx=0 tx=0).stop 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 shutdown_connections 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c80a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c807940 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 --2- 172.21.15.138:0/415837234 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.571+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 >> 172.21.15.138:0/415837234 conn(0x55ab7c7f9400 msgr2=0x55ab7c7fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.572+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 shutdown_connections 2024-08-20T01:24:06.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.572+0000 7f846ef22740 1 -- 172.21.15.138:0/415837234 wait complete. 2024-08-20T01:24:06.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.573+0000 7f846ef22740 1 -- start start 2024-08-20T01:24:06.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.573+0000 7f846ef22740 1 Processor -- start 2024-08-20T01:24:06.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f8469400640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 0x7f844c0b6440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8454048000 tx=0x7f8454004860 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c804d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.574 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c704780 con 0x55ab7c7fb5d0 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c762d00 con 0x55ab7c7fbef0 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ab7c5c17a0 con 0x55ab7c807e80 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.574+0000 7f846ef22740 10 client.0 _ll_register_callbacks cb 0x55ab7c70c940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.575+0000 7f846a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.575 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.575+0000 7f846a800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58566/0 (socket says 172.21.15.138:58566) 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f8469400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c804d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 -- 172.21.15.138:0/905499588 learned_addr learned my addr 172.21.15.138:0/905499588 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 -- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 msgr2=0x55ab7c804d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c804d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 -- 172.21.15.138:0/905499588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 msgr2=0x55ab7c8078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.576 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.576+0000 7f846a800640 1 -- 172.21.15.138:0/905499588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7c7119c0 con 0x55ab7c807e80 2024-08-20T01:24:06.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.577+0000 7f846a800640 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f84580e1c10 tx=0x7f84580aa300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce8b536d001e2fb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.577 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.577+0000 7f8447400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:24:06.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.578+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580c1020 con 0x55ab7c807e80 2024-08-20T01:24:06.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.578+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f84580a9d30 con 0x55ab7c807e80 2024-08-20T01:24:06.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.578+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ab7c771710 con 0x55ab7c807e80 2024-08-20T01:24:06.578 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.578+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f84580aa5e0 con 0x55ab7c807e80 2024-08-20T01:24:06.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.579+0000 7f846ef22740 10 client.27337 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:06.579 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.580+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ab7c4cdb90 con 0x55ab7c807e80 2024-08-20T01:24:06.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.580+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ab7c771710 con 0x55ab7c807e80 2024-08-20T01:24:06.580 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.580+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f8458020030 con 0x55ab7c807e80 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 10 client.27337 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 10 client.27337 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 20 client.27337 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c817870 0x55ab7c819cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 4 client.27337 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab7c76d1b0 con 0x55ab7c817870 2024-08-20T01:24:06.581 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.581+0000 7f8469e00640 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c817870 0x55ab7c819cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.582+0000 7f8447400640 10 client.27337 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.584 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.585+0000 7f8469e00640 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c817870 0x55ab7c819cf0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96138a1d61556a83 server_cookie=3def29abc975d28c in_seq=0 out_seq=0 2024-08-20T01:24:06.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.587+0000 7f8447400640 1 -- 172.21.15.138:0/905499588 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab7c76d1b0 con 0x55ab7c817870 2024-08-20T01:24:06.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.587+0000 7f8447400640 10 client.27337 handle_command_reply: tid=0 2024-08-20T01:24:06.587 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.587+0000 7f846ef22740 1 client.27337 shutdown 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 msgr2=0x55ab7c8054c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f84580e1c10 tx=0x7f84580aa300 comp rx=0 tx=0).stop 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 shutdown_connections 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c817870 0x55ab7c819cf0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c807e80 0x55ab7c8054c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c804d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 --2- 172.21.15.138:0/905499588 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.588 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.588+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 >> 172.21.15.138:0/905499588 conn(0x55ab7c7f9400 msgr2=0x55ab7c7fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.589+0000 7f8447400640 0 client.27337 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.590+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 shutdown_connections 2024-08-20T01:24:06.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.590+0000 7f846ef22740 1 -- 172.21.15.138:0/905499588 wait complete. 2024-08-20T01:24:06.590 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.590+0000 7f846ef22740 20 client.27337 trim_cache size 0 max 16384 2024-08-20T01:24:06.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.626+0000 7f846ef22740 1 Processor -- start 2024-08-20T01:24:06.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 -- start start 2024-08-20T01:24:06.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7ff6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fbef0 0x55ab7c8035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 0x55ab7c803af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84641c10d0 con 0x55ab7c7fb5d0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84641be910 con 0x55ab7c807e80 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.627+0000 7f846ef22740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84641bc780 con 0x55ab7c7fbef0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f8469e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7ff6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f8469e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7ff6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40638/0 (socket says 172.21.15.138:40638) 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f8469e00640 1 -- 172.21.15.138:0/392140231 learned_addr learned my addr 172.21.15.138:0/392140231 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 0x55ab7c803af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f8469400640 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fbef0 0x55ab7c8035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 -- 172.21.15.138:0/392140231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fbef0 msgr2=0x55ab7c8035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fbef0 0x55ab7c8035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 -- 172.21.15.138:0/392140231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 msgr2=0x55ab7c7ff6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7ff6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.628+0000 7f846a800640 1 -- 172.21.15.138:0/392140231 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab7c803230 con 0x55ab7c807e80 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.629+0000 7f846a800640 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 0x55ab7c803af0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84580e16c0 tx=0x7f84580aa370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c45ada13bfde5e09 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.629+0000 7f8452600640 1 -- 172.21.15.138:0/392140231 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580c1070 con 0x55ab7c807e80 2024-08-20T01:24:06.630 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.629+0000 7f8452600640 1 -- 172.21.15.138:0/392140231 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84580ab1d0 con 0x55ab7c807e80 2024-08-20T01:24:06.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.629+0000 7f8452600640 1 -- 172.21.15.138:0/392140231 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84580a9f90 con 0x55ab7c807e80 2024-08-20T01:24:06.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 -- 172.21.15.138:0/392140231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 msgr2=0x55ab7c803af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:06.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 0x55ab7c803af0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84580e16c0 tx=0x7f84580aa370 comp rx=0 tx=0).stop 2024-08-20T01:24:06.631 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 -- 172.21.15.138:0/392140231 shutdown_connections 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c807e80 0x55ab7c803af0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fbef0 0x55ab7c8035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 --2- 172.21.15.138:0/392140231 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c7ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 -- 172.21.15.138:0/392140231 >> 172.21.15.138:0/392140231 conn(0x55ab7c7f9400 msgr2=0x55ab7c7fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 -- 172.21.15.138:0/392140231 shutdown_connections 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.630+0000 7f846ef22740 1 -- 172.21.15.138:0/392140231 wait complete. 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.631+0000 7f846ef22740 1 -- start start 2024-08-20T01:24:06.632 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.631+0000 7f846ef22740 1 Processor -- start 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.632+0000 7f846ef22740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.632+0000 7f846ef22740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c810640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.632+0000 7f846ef22740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f8469e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:58588/0 (socket says 172.21.15.138:58588) 2024-08-20T01:24:06.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f846ef22740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f84641bcdd0 con 0x55ab7c807e80 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f846ef22740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f84641be240 con 0x55ab7c7fbef0 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f8469e00640 1 -- 172.21.15.138:0/1696387230 learned_addr learned my addr 172.21.15.138:0/1696387230 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.633+0000 7f8469400640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c810640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.634+0000 7f846a800640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.634+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f84641bd4a0 con 0x55ab7c7fb5d0 2024-08-20T01:24:06.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.634+0000 7f846ef22740 10 client.0 _ll_register_callbacks cb 0x7f845800af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:06.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.635+0000 7f8469400640 1 -- 172.21.15.138:0/1696387230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 msgr2=0x55ab7c8100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.635+0000 7f8469400640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.635+0000 7f8469400640 1 -- 172.21.15.138:0/1696387230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 msgr2=0x55ab7c801a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:06.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.635+0000 7f8469400640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c801a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:06.635 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.635+0000 7f8469400640 1 -- 172.21.15.138:0/1696387230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab7c802240 con 0x55ab7c7fbef0 2024-08-20T01:24:06.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.636+0000 7f8469e00640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:06.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.636+0000 7f8469400640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c810640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84540303f0 tx=0x7f8454040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb5b11e1c2b5096 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:06.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.636+0000 7f8447400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:24:06.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.637+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540046c0 con 0x55ab7c7fbef0 2024-08-20T01:24:06.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.637+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9541+0+0 (secure 0 0 0) 0x7f8454030650 con 0x55ab7c7fbef0 2024-08-20T01:24:06.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.638+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84580ab1d0 con 0x55ab7c7fbef0 2024-08-20T01:24:06.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.638+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8454030850 con 0x55ab7c7fbef0 2024-08-20T01:24:06.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.638+0000 7f846ef22740 10 client.27368 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:06.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.638+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55ab7c882280 con 0x55ab7c7fbef0 2024-08-20T01:24:06.639 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84580ab1d0 con 0x55ab7c7fbef0 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7f8454050030 con 0x55ab7c7fbef0 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 10 client.27368 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 10 client.27368 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 20 client.27368 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c882620 0x55ab7c884a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:06.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 4 client.27368 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.6, "await_for": 45.0}] 2024-08-20T01:24:06.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.639+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.6, "await_for": 45.0}) -- 0x55ab7c76a8f0 con 0x55ab7c882620 2024-08-20T01:24:06.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.640+0000 7f8469e00640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c882620 0x55ab7c884a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:06.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.640+0000 7f8447400640 10 client.27368 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:06.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:06.641+0000 7f8469e00640 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c882620 0x55ab7c884a10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc865366fb854377 server_cookie=c931ce70a50c9ce2 in_seq=0 out_seq=0 2024-08-20T01:24:07.271 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44156 27.70 MB/sec execute 4 sec latency 580.428 ms 2024-08-20T01:24:07.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.560+0000 7f8447400640 1 -- 172.21.15.138:0/1696387230 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ab7c76a8f0 con 0x55ab7c882620 2024-08-20T01:24:07.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.560+0000 7f8447400640 10 client.27368 handle_command_reply: tid=0 2024-08-20T01:24:07.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.560+0000 7f846ef22740 1 client.27368 shutdown 2024-08-20T01:24:07.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.560+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 msgr2=0x55ab7c810640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:07.560 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.560+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c810640 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f84540303f0 tx=0x7f8454040040 comp rx=0 tx=0).stop 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 shutdown_connections 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ab7c882620 0x55ab7c884a10 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ab7c807e80 0x55ab7c801a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab7c7fbef0 0x55ab7c810640 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 --2- 172.21.15.138:0/1696387230 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ab7c7fb5d0 0x55ab7c8100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f8447400640 0 client.27368 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 >> 172.21.15.138:0/1696387230 conn(0x55ab7c7f9400 msgr2=0x55ab7c803530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 shutdown_connections 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 1 -- 172.21.15.138:0/1696387230 wait complete. 2024-08-20T01:24:07.561 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.561+0000 7f846ef22740 20 client.27368 trim_cache size 0 max 16384 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 48, 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 58, 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "c11dd335": { 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 58, 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:24:07.562 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 146.6, 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:07.563 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 msgr2=0x7f844c0b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 0x7f844c0b6440 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f8454048000 tx=0x7f8454004860 comp rx=0 tx=0).stop 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 msgr2=0x7f8464117650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 0x7f8464117650 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f845800c970 tx=0x7f845800ce30 comp rx=0 tx=0).stop 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 shutdown_connections 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f844c0b3f80 0x7f844c0b6440 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.564 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8464108cb0 0x7f8464117650 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8464108330 0x7f8464112320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 --2- 172.21.15.138:0/2082913368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8464102920 0x7f8464111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:07.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 >> 172.21.15.138:0/2082913368 conn(0x7f84640fe350 msgr2=0x7f8464100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:07.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 shutdown_connections 2024-08-20T01:24:07.565 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:07.562+0000 7f846b200640 1 -- 172.21.15.138:0/2082913368 wait complete. 2024-08-20T01:24:08.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:07 smithi167 ceph-mon[21965]: pgmap v862: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 704 op/s 2024-08-20T01:24:08.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:07 smithi142 ceph-mon[21799]: pgmap v862: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 704 op/s 2024-08-20T01:24:08.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 22.16 MB/sec execute 5 sec latency 1580.551 ms 2024-08-20T01:24:08.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:07 smithi138 ceph-mon[18076]: pgmap v862: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 6.1 MiB/s wr, 704 op/s 2024-08-20T01:24:08.925 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'c11dd335', quiesce took 7.3 seconds. Will release after: 19.3 2024-08-20T01:24:09.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 18.47 MB/sec execute 6 sec latency 2580.693 ms 2024-08-20T01:24:10.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:09 smithi167 ceph-mon[21965]: pgmap v863: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 717 op/s 2024-08-20T01:24:10.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:09 smithi142 ceph-mon[21799]: pgmap v863: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 717 op/s 2024-08-20T01:24:10.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 15.83 MB/sec execute 7 sec latency 3580.776 ms 2024-08-20T01:24:10.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:09 smithi138 ceph-mon[18076]: pgmap v863: 129 pgs: 8 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 95 active+clean; 889 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 6.1 MiB/s wr, 717 op/s 2024-08-20T01:24:11.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 13.85 MB/sec execute 8 sec latency 4580.924 ms 2024-08-20T01:24:12.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:11 smithi167 ceph-mon[21965]: pgmap v864: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 890 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-08-20T01:24:12.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:11 smithi142 ceph-mon[21799]: pgmap v864: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 890 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-08-20T01:24:12.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 12.31 MB/sec execute 9 sec latency 5581.067 ms 2024-08-20T01:24:12.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:11 smithi138 ceph-mon[18076]: pgmap v864: 129 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 121 active+clean; 890 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.5 MiB/s wr, 667 op/s 2024-08-20T01:24:13.272 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 11.08 MB/sec execute 10 sec latency 6581.211 ms 2024-08-20T01:24:14.182 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:13 smithi138 ceph-mon[18076]: pgmap v865: 129 pgs: 129 active+clean; 893 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-08-20T01:24:14.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:13 smithi167 ceph-mon[21965]: pgmap v865: 129 pgs: 129 active+clean; 893 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-08-20T01:24:14.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:13 smithi142 ceph-mon[21799]: pgmap v865: 129 pgs: 129 active+clean; 893 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.7 MiB/s wr, 615 op/s 2024-08-20T01:24:14.275 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 10.07 MB/sec execute 11 sec latency 7584.300 ms 2024-08-20T01:24:15.275 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 9.23 MB/sec execute 12 sec latency 8584.362 ms 2024-08-20T01:24:15.569 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:15 smithi138 ceph-mon[18076]: osdmap e109: 12 total, 12 up, 12 in 2024-08-20T01:24:15.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:15 smithi167 ceph-mon[21965]: osdmap e109: 12 total, 12 up, 12 in 2024-08-20T01:24:15.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:15 smithi142 ceph-mon[21799]: osdmap e109: 12 total, 12 up, 12 in 2024-08-20T01:24:16.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 8.52 MB/sec execute 13 sec latency 9584.436 ms 2024-08-20T01:24:16.711 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:16 smithi138 ceph-mon[18076]: pgmap v867: 129 pgs: 129 active+clean; 906 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 519 op/s 2024-08-20T01:24:16.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:16 smithi167 ceph-mon[21965]: pgmap v867: 129 pgs: 129 active+clean; 906 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 519 op/s 2024-08-20T01:24:16.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:16 smithi142 ceph-mon[21799]: pgmap v867: 129 pgs: 129 active+clean; 906 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.2 MiB/s wr, 519 op/s 2024-08-20T01:24:17.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 7.91 MB/sec execute 14 sec latency 10584.491 ms 2024-08-20T01:24:18.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 7.39 MB/sec execute 15 sec latency 11584.545 ms 2024-08-20T01:24:18.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:18 smithi138 ceph-mon[18076]: pgmap v868: 129 pgs: 129 active+clean; 941 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.3 MiB/s wr, 334 op/s 2024-08-20T01:24:18.794 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:18 smithi142 ceph-mon[21799]: pgmap v868: 129 pgs: 129 active+clean; 941 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.3 MiB/s wr, 334 op/s 2024-08-20T01:24:18.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:18 smithi167 ceph-mon[21965]: pgmap v868: 129 pgs: 129 active+clean; 941 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.3 MiB/s wr, 334 op/s 2024-08-20T01:24:19.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 6.92 MB/sec execute 16 sec latency 12584.614 ms 2024-08-20T01:24:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 -- 172.21.15.138:0/4090988111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b410efc0 msgr2=0x7fb8b41113b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 --2- 172.21.15.138:0/4090988111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b410efc0 0x7fb8b41113b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009be0 tx=0x7fb8ac02a6d0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 -- 172.21.15.138:0/4090988111 shutdown_connections 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 --2- 172.21.15.138:0/4090988111 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b410efc0 0x7fb8b41113b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 --2- 172.21.15.138:0/4090988111 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075b30 0x7fb8b410ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 --2- 172.21.15.138:0/4090988111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8b4075210 0x7fb8b40755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 -- 172.21.15.138:0/4090988111 >> 172.21.15.138:0/4090988111 conn(0x7fb8b40fe420 msgr2=0x7fb8b4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.703+0000 7fb8bc600640 1 -- 172.21.15.138:0/4090988111 shutdown_connections 2024-08-20T01:24:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.704+0000 7fb8bc600640 1 -- 172.21.15.138:0/4090988111 wait complete. 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.704+0000 7fb8bc600640 1 Processor -- start 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.704+0000 7fb8bc600640 1 -- start start 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 0x7fb8b419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8b410efc0 0x7fb8b41a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4114ba0 con 0x7fb8b4075210 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4114a20 con 0x7fb8b410efc0 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8bc600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b4114d20 con 0x7fb8b4075b30 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46192/0 (socket says 172.21.15.138:46192) 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9c00640 1 -- 172.21.15.138:0/1415670385 learned_addr learned my addr 172.21.15.138:0/1415670385 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9200640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 0x7fb8b419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9c00640 1 -- 172.21.15.138:0/1415670385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 msgr2=0x7fb8b419b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.705+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 0x7fb8b419b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b9c00640 1 -- 172.21.15.138:0/1415670385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8b410efc0 msgr2=0x7fb8b41a09b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8b410efc0 0x7fb8b41a09b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b9c00640 1 -- 172.21.15.138:0/1415670385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b41a1180 con 0x7fb8b4075210 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b9200640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 0x7fb8b419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800e9f0 tx=0x7fb8a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b2000640 1 -- 172.21.15.138:0/1415670385 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a800cd80 con 0x7fb8b4075210 2024-08-20T01:24:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b41a1470 con 0x7fb8b4075210 2024-08-20T01:24:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b2000640 1 -- 172.21.15.138:0/1415670385 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a80054d0 con 0x7fb8b4075210 2024-08-20T01:24:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8b2000640 1 -- 172.21.15.138:0/1415670385 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8010690 con 0x7fb8b4075210 2024-08-20T01:24:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.706+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b41a7b30 con 0x7fb8b4075210 2024-08-20T01:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.709+0000 7fb8bf5cb740 1 Processor -- start 2024-08-20T01:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 -- start start 2024-08-20T01:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f879b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5564d4efd210 con 0x5564d4f93e80 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5564d4eec710 con 0x5564d4f87ef0 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5564d4ef0750 con 0x5564d4f875d0 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46198/0 (socket says 172.21.15.138:46198) 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.710+0000 7fb8b9200640 1 -- 172.21.15.138:0/3830221658 learned_addr learned my addr 172.21.15.138:0/3830221658 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 -- 172.21.15.138:0/3830221658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 msgr2=0x5564d4f879b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f879b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 -- 172.21.15.138:0/3830221658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 msgr2=0x5564d4f93940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f93940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 -- 172.21.15.138:0/3830221658 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4eb8de0 con 0x5564d4f93e80 2024-08-20T01:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b9200640 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00114f0 tx=0x7fb8a00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880d297a1009c25d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8a6a00640 1 -- 172.21.15.138:0/3830221658 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0005020 con 0x5564d4f93e80 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8a6a00640 1 -- 172.21.15.138:0/3830221658 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a00047a0 con 0x5564d4f93e80 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8a6a00640 1 -- 172.21.15.138:0/3830221658 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a000f840 con 0x5564d4f93e80 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.711+0000 7fb8b2000640 1 -- 172.21.15.138:0/1415670385 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb8a8004e90 con 0x7fb8b4075210 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3830221658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 msgr2=0x5564d4f96270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00114f0 tx=0x7fb8a00119b0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3830221658 shutdown_connections 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f96270 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3830221658 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f879b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3830221658 >> 172.21.15.138:0/3830221658 conn(0x5564d4f853b0 msgr2=0x5564d4f865a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3830221658 shutdown_connections 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.712+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3830221658 wait complete. 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.713+0000 7fb8bf5cb740 1 -- start start 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.713+0000 7fb8bf5cb740 1 Processor -- start 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.713+0000 7fb8b2000640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 0x7fb89c0b61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.713+0000 7fb8b2000640 1 -- 172.21.15.138:0/1415670385 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7fb8a80e4840 con 0x7fb8b4075210 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.713+0000 7fb8ba600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 0x7fb89c0b61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f91310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f91850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5564d4e25ab0 con 0x5564d4f93e80 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5564d4e93140 con 0x5564d4f87ef0 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5564d4f96bb0 con 0x5564d4f875d0 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8bf5cb740 10 client.0 _ll_register_callbacks cb 0x5564d4ea65d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8ba600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f91850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:40470/0 (socket says 172.21.15.138:40470) 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f91310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 -- 172.21.15.138:0/3516299462 learned_addr learned my addr 172.21.15.138:0/3516299462 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 -- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 msgr2=0x5564d4f91310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f91310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 -- 172.21.15.138:0/3516299462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 msgr2=0x5564d4f91850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f91850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8b9c00640 1 -- 172.21.15.138:0/3516299462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4e94170 con 0x5564d4f875d0 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.714+0000 7fb8ba600640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb8ba600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 0x7fb89c0b61a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac004af0 tx=0x7fb8ac030620 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80e1bf0 tx=0x7fb8a80048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e184bcc41cfff8c server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb89b400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80c4020 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5564d4ec3a30 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7fb8a80e4420 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb8a80e4620 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb8bf5cb740 10 client.27352 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.715+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5564d4c4f790 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5564d4ec3a30 con 0x5564d4f875d0 2024-08-20T01:24:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fb8a80b3c70 con 0x5564d4f875d0 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 10 client.27352 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 10 client.27352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 20 client.27352 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d4fac650 0x5564d4faea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 4 client.27352 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.716+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d4eff160 con 0x5564d4fac650 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.717+0000 7fb8b9200640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d4fac650 0x5564d4faea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.717+0000 7fb8b9200640 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d4fac650 0x5564d4faea40 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50587345fc8f7d3f server_cookie=2b8ef15bbcd446ed in_seq=0 out_seq=0 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.717+0000 7fb89b400640 10 client.27352 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb89b400640 1 -- 172.21.15.138:0/3516299462 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564d4eff160 con 0x5564d4fac650 2024-08-20T01:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb89b400640 10 client.27352 handle_command_reply: tid=0 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 client.27352 shutdown 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 msgr2=0x5564d4f90dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80e1bf0 tx=0x7fb8a80048f0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 shutdown_connections 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d4fac650 0x5564d4faea40 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f93e80 0x5564d4f91850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f91310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb89b400640 0 client.27352 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3516299462 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f90dd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 >> 172.21.15.138:0/3516299462 conn(0x5564d4f853b0 msgr2=0x5564d4f8afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 shutdown_connections 2024-08-20T01:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3516299462 wait complete. 2024-08-20T01:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.718+0000 7fb8bf5cb740 20 client.27352 trim_cache size 0 max 16384 2024-08-20T01:24:19.733 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.733+0000 7fb8bf5cb740 1 Processor -- start 2024-08-20T01:24:19.734 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.733+0000 7fb8bf5cb740 1 -- start start 2024-08-20T01:24:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.733+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.733+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f90640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f93e80 0x5564d4fb42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.735 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41b0300 con 0x5564d4f875d0 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41adb40 con 0x5564d4f87ef0 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ab9d0 con 0x5564d4f93e80 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46214/0 (socket says 172.21.15.138:46214) 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 -- 172.21.15.138:0/875145448 learned_addr learned my addr 172.21.15.138:0/875145448 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9200640 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f90640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 -- 172.21.15.138:0/875145448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f93e80 msgr2=0x5564d4fb42f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f93e80 0x5564d4fb42f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 -- 172.21.15.138:0/875145448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 msgr2=0x5564d4f90640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f90640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.736 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 -- 172.21.15.138:0/875145448 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d4faedb0 con 0x5564d4f875d0 2024-08-20T01:24:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.734+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8004a40 tx=0x7fb8a80a69c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706d92a691b95332 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8a6000640 1 -- 172.21.15.138:0/875145448 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80ba030 con 0x5564d4f875d0 2024-08-20T01:24:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8a6000640 1 -- 172.21.15.138:0/875145448 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a80af410 con 0x5564d4f875d0 2024-08-20T01:24:19.737 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8a6000640 1 -- 172.21.15.138:0/875145448 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a80c4040 con 0x5564d4f875d0 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/875145448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 msgr2=0x5564d4f92ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8004a40 tx=0x7fb8a80a69c0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/875145448 shutdown_connections 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f93e80 0x5564d4fb42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f87ef0 0x5564d4f90640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/875145448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f875d0 0x5564d4f92ed0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.735+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/875145448 >> 172.21.15.138:0/875145448 conn(0x5564d4f853b0 msgr2=0x5564d4f915c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/875145448 shutdown_connections 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/875145448 wait complete. 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- start start 2024-08-20T01:24:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 Processor -- start 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f87ef0 0x5564d4f977e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ac020 con 0x5564d4f87ef0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ad470 con 0x5564d4f93e80 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.736+0000 7fb8bf5cb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b41ac6d0 con 0x5564d4f875d0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8bf5cb740 10 client.0 _ll_register_callbacks cb 0x7fb8a8004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8b9200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f87ef0 0x5564d4f977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:50574/0 (socket says 172.21.15.138:50574) 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 -- 172.21.15.138:0/3537986787 learned_addr learned my addr 172.21.15.138:0/3537986787 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 -- 172.21.15.138:0/3537986787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 msgr2=0x5564d4f972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 -- 172.21.15.138:0/3537986787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f87ef0 msgr2=0x5564d4f977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f87ef0 0x5564d4f977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.737+0000 7fb8ba600640 1 -- 172.21.15.138:0/3537986787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d4cc8620 con 0x5564d4f93e80 2024-08-20T01:24:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.755+0000 7fb8ba600640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009be0 tx=0x7fb8ac009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f320033228e74df9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:19.756 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.755+0000 7fb89b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:24:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac036040 con 0x5564d4f93e80 2024-08-20T01:24:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7fb8ac030d80 con 0x5564d4f93e80 2024-08-20T01:24:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8a80af410 con 0x5564d4f93e80 2024-08-20T01:24:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fb8ac059050 con 0x5564d4f93e80 2024-08-20T01:24:19.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 10 client.27380 fetch_fsmap learned FSMap version 48 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5564d5016840 con 0x5564d4f93e80 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8a80af410 con 0x5564d4f93e80 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 48) ==== 8374+0+0 (secure 0 0 0) 0x7fb8ac04b5f0 con 0x5564d4f93e80 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 10 client.27380 fetch_fsmap finished waiting for FSMap version 48 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 10 client.27380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 20 client.27380 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d50170a0 0x5564d50194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 4 client.27380 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.756+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564d4efac00 con 0x5564d50170a0 2024-08-20T01:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.757+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d50170a0 0x5564d50194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.757+0000 7fb8b9c00640 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d50170a0 0x5564d50194c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9127b65547f14cf server_cookie=29090bdbfdafaefc in_seq=0 out_seq=0 2024-08-20T01:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.757+0000 7fb89b400640 10 client.27380 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb89b400640 1 -- 172.21.15.138:0/3537986787 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564d4efac00 con 0x5564d50170a0 2024-08-20T01:24:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb89b400640 10 client.27380 handle_command_reply: tid=0 2024-08-20T01:24:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 client.27380 shutdown 2024-08-20T01:24:19.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 msgr2=0x5564d50152b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009be0 tx=0x7fb8ac009500 comp rx=0 tx=0).stop 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 shutdown_connections 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5564d50170a0 0x5564d50194c0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564d4f93e80 0x5564d50152b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5564d4f87ef0 0x5564d4f977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb89b400640 0 client.27380 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 --2- 172.21.15.138:0/3537986787 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5564d4f875d0 0x5564d4f972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.818+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 >> 172.21.15.138:0/3537986787 conn(0x5564d4f853b0 msgr2=0x5564d4f860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 shutdown_connections 2024-08-20T01:24:19.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bf5cb740 1 -- 172.21.15.138:0/3537986787 wait complete. 2024-08-20T01:24:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bf5cb740 20 client.27380 trim_cache size 0 max 16384 2024-08-20T01:24:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:24:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:24:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:24:19.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 msgr2=0x7fb89c0b61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 0x7fb89c0b61a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac004af0 tx=0x7fb8ac030620 comp rx=0 tx=0).stop 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 msgr2=0x7fb8b419b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.819+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800e9f0 tx=0x7fb8a800edf0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 shutdown_connections 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb89c0b3ce0 0x7fb89c0b61a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb8b410efc0 0x7fb8b41a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb8b4075b30 0x7fb8b419b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 --2- 172.21.15.138:0/1415670385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8b4075210 0x7fb8b419b0e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 >> 172.21.15.138:0/1415670385 conn(0x7fb8b40fe420 msgr2=0x7fb8b41111a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 shutdown_connections 2024-08-20T01:24:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:19.820+0000 7fb8bc600640 1 -- 172.21.15.138:0/1415670385 wait complete. 2024-08-20T01:24:20.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 6.52 MB/sec execute 17 sec latency 13584.679 ms 2024-08-20T01:24:20.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:20 smithi138 ceph-mon[18076]: pgmap v869: 129 pgs: 129 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.1 MiB/s wr, 332 op/s 2024-08-20T01:24:20.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:20 smithi167 ceph-mon[21965]: pgmap v869: 129 pgs: 129 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.1 MiB/s wr, 332 op/s 2024-08-20T01:24:20.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:20 smithi142 ceph-mon[21799]: pgmap v869: 129 pgs: 129 active+clean; 945 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.1 MiB/s wr, 332 op/s 2024-08-20T01:24:21.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 6.16 MB/sec execute 18 sec latency 14584.744 ms 2024-08-20T01:24:22.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 5.83 MB/sec execute 19 sec latency 15584.807 ms 2024-08-20T01:24:22.782 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:22 smithi138 ceph-mon[18076]: pgmap v870: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-08-20T01:24:22.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:22 smithi167 ceph-mon[21965]: pgmap v870: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-08-20T01:24:22.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:22 smithi142 ceph-mon[21799]: pgmap v870: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 7.1 MiB/s wr, 282 op/s 2024-08-20T01:24:23.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 5.54 MB/sec execute 20 sec latency 16584.873 ms 2024-08-20T01:24:24.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 5.28 MB/sec execute 21 sec latency 17585.030 ms 2024-08-20T01:24:24.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:24 smithi138 ceph-mon[18076]: pgmap v871: 129 pgs: 129 active+clean; 912 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 6.8 MiB/s wr, 244 op/s 2024-08-20T01:24:24.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:24 smithi167 ceph-mon[21965]: pgmap v871: 129 pgs: 129 active+clean; 912 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 6.8 MiB/s wr, 244 op/s 2024-08-20T01:24:24.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:24 smithi142 ceph-mon[21799]: pgmap v871: 129 pgs: 129 active+clean; 912 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 6.8 MiB/s wr, 244 op/s 2024-08-20T01:24:25.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 5.04 MB/sec execute 22 sec latency 18585.148 ms 2024-08-20T01:24:26.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 4.82 MB/sec execute 23 sec latency 19585.294 ms 2024-08-20T01:24:26.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:26 smithi138 ceph-mon[18076]: pgmap v872: 129 pgs: 129 active+clean; 911 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 6.5 MiB/s wr, 245 op/s 2024-08-20T01:24:26.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:26 smithi167 ceph-mon[21965]: pgmap v872: 129 pgs: 129 active+clean; 911 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 6.5 MiB/s wr, 245 op/s 2024-08-20T01:24:26.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:26 smithi142 ceph-mon[21799]: pgmap v872: 129 pgs: 129 active+clean; 911 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 6.5 MiB/s wr, 245 op/s 2024-08-20T01:24:27.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 4.62 MB/sec execute 24 sec latency 20585.389 ms 2024-08-20T01:24:28.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:24:28.276 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 4.43 MB/sec execute 25 sec latency 21585.443 ms 2024-08-20T01:24:28.611 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:24:28.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:28 smithi138 ceph-mon[18076]: pgmap v873: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-08-20T01:24:28.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:28 smithi167 ceph-mon[21965]: pgmap v873: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-08-20T01:24:28.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:28 smithi142 ceph-mon[21799]: pgmap v873: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 4.6 MiB/s wr, 201 op/s 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 -- 172.21.15.138:0/1710654601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 msgr2=0x7f195c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 --2- 172.21.15.138:0/1710654601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c111830 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1954009be0 tx=0x7f195402a6c0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 -- 172.21.15.138:0/1710654601 shutdown_connections 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 --2- 172.21.15.138:0/1710654601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c111830 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 --2- 172.21.15.138:0/1710654601 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c10a6b0 0x7f195c10aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 --2- 172.21.15.138:0/1710654601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c104ca0 0x7f195c105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 -- 172.21.15.138:0/1710654601 >> 172.21.15.138:0/1710654601 conn(0x7f195c1006f0 msgr2=0x7f195c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 -- 172.21.15.138:0/1710654601 shutdown_connections 2024-08-20T01:24:29.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.174+0000 7f1960e00640 1 -- 172.21.15.138:0/1710654601 wait complete. 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.175+0000 7f1960e00640 1 Processor -- start 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.175+0000 7f1960e00640 1 -- start start 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c104ca0 0x7f195c111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 0x7f195c10e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f195c114cb0 con 0x7f195c10a6b0 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f195c114b30 con 0x7f195c10b030 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1960e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f195c114e30 con 0x7f195c104ca0 2024-08-20T01:24:29.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1951400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 0x7f195c10e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f195a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f195a800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45734/0 (socket says 172.21.15.138:45734) 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f195a800640 1 -- 172.21.15.138:0/3621097763 learned_addr learned my addr 172.21.15.138:0/3621097763 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1951400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 0x7f195c10e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39232/0 (socket says 172.21.15.138:39232) 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.176+0000 7f1959e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c104ca0 0x7f195c111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 -- 172.21.15.138:0/3621097763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c104ca0 msgr2=0x7f195c111600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c104ca0 0x7f195c111600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 -- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 msgr2=0x7f195c10e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 0x7f195c10e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 -- 172.21.15.138:0/3621097763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c1ab990 con 0x7f195c10b030 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f195a800640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1954004af0 tx=0x7f1954003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954041020 con 0x7f195c10b030 2024-08-20T01:24:29.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c1abb60 con 0x7f195c10b030 2024-08-20T01:24:29.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.177+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1954030970 con 0x7f195c10b030 2024-08-20T01:24:29.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.178+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1954036040 con 0x7f195c10b030 2024-08-20T01:24:29.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.178+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c1abfb0 con 0x7f195c10b030 2024-08-20T01:24:29.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.179+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195c105080 con 0x7f195c10b030 2024-08-20T01:24:29.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.181+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f1954044020 con 0x7f195c10b030 2024-08-20T01:24:29.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.182+0000 7f1953400640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 0x7f19400b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:29.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.183+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f1954105530 con 0x7f195c10b030 2024-08-20T01:24:29.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.183+0000 7f1959e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 0x7f19400b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:29.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.183+0000 7f1959e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 0x7f19400b6200 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f194c009530 tx=0x7f194c009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:29.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.184+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f195404b030 con 0x7f195c10b030 2024-08-20T01:24:29.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 4.26 MB/sec execute 26 sec latency 22585.507 ms 2024-08-20T01:24:29.402 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.403+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f195c02e340 con 0x7f195c10b030 2024-08-20T01:24:29.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.404+0000 7f1953400640 1 -- 172.21.15.138:0/3621097763 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11937 (secure 0 0 0) 0x7f19540cc440 con 0x7f195c10b030 2024-08-20T01:24:29.404 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:24:29.404 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":49,"btime":"2024-08-20T01:24:28:537859+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:24:27.572423+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:24:29.404 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 49 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 msgr2=0x7f19400b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 0x7f19400b6200 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f194c009530 tx=0x7f194c009260 comp rx=0 tx=0).stop 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 msgr2=0x7f195c10edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1954004af0 tx=0x7f1954003c90 comp rx=0 tx=0).stop 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 shutdown_connections 2024-08-20T01:24:29.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f19400b3d40 0x7f19400b6200 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f195c10b030 0x7f195c10edf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f195c10a6b0 0x7f195c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 --2- 172.21.15.138:0/3621097763 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f195c104ca0 0x7f195c111600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 >> 172.21.15.138:0/3621097763 conn(0x7f195c1006f0 msgr2=0x7f195c100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 shutdown_connections 2024-08-20T01:24:29.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:29.406+0000 7f1960e00640 1 -- 172.21.15.138:0/3621097763 wait complete. 2024-08-20T01:24:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:29 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:29.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:29 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3621097763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:29 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:29.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:29 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3621097763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:30.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:29 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:30.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:29 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3621097763' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:24:30.167 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id c11dd335 --release --await' 2024-08-20T01:24:30.167 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id c11dd335 --release --await 2024-08-20T01:24:30.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44160 4.10 MB/sec execute 27 sec latency 23585.579 ms 2024-08-20T01:24:30.543 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:24:30.875 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:30 smithi138 ceph-mon[18076]: pgmap v874: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-08-20T01:24:30.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:30 smithi167 ceph-mon[21965]: pgmap v874: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-08-20T01:24:30.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:30 smithi142 ceph-mon[21799]: pgmap v874: 129 pgs: 129 active+clean; 909 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 1.4 MiB/s wr, 133 op/s 2024-08-20T01:24:31.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 -- 172.21.15.138:0/13758847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4450104b00 msgr2=0x7f4450104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 --2- 172.21.15.138:0/13758847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4450104b00 0x7f4450104ee0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f4444009900 tx=0x7f444402a410 comp rx=0 tx=0).stop 2024-08-20T01:24:31.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 -- 172.21.15.138:0/13758847 shutdown_connections 2024-08-20T01:24:31.191 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 --2- 172.21.15.138:0/13758847 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010ae90 0x7f4450111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 --2- 172.21.15.138:0/13758847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f445010a510 0x7f445010a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 --2- 172.21.15.138:0/13758847 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4450104b00 0x7f4450104ee0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 -- 172.21.15.138:0/13758847 >> 172.21.15.138:0/13758847 conn(0x7f4450100550 msgr2=0x7f4450102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 -- 172.21.15.138:0/13758847 shutdown_connections 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.157+0000 7f4456400640 1 -- 172.21.15.138:0/13758847 wait complete. 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.158+0000 7f4456400640 1 Processor -- start 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.158+0000 7f4456400640 1 -- start start 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.158+0000 7f4456400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.158+0000 7f4456400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010a510 0x7f445019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.158+0000 7f4456400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f445010ae90 0x7f44501a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4456400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4450114ba0 con 0x7f445010ae90 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4456400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4450114a20 con 0x7f4450104b00 2024-08-20T01:24:31.192 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4456400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4450114d20 con 0x7f445010a510 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f445010ae90 0x7f44501a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45760/0 (socket says 172.21.15.138:45760) 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 -- 172.21.15.138:0/2667884501 learned_addr learned my addr 172.21.15.138:0/2667884501 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f444fe00640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010a510 0x7f445019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 -- 172.21.15.138:0/2667884501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010a510 msgr2=0x7f445019b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010a510 0x7f445019b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 -- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f445010ae90 msgr2=0x7f44501a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f445010ae90 0x7f44501a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.159+0000 7f4455000640 1 -- 172.21.15.138:0/2667884501 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501a0fa0 con 0x7f4450104b00 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f4455000640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4444005ab0 tx=0x7f4444003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f444d600640 1 -- 172.21.15.138:0/2667884501 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444403f070 con 0x7f4450104b00 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44501a1230 con 0x7f4450104b00 2024-08-20T01:24:31.193 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f444d600640 1 -- 172.21.15.138:0/2667884501 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4444031d60 con 0x7f4450104b00 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f444d600640 1 -- 172.21.15.138:0/2667884501 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4444035030 con 0x7f4450104b00 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.160+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a7950 con 0x7f4450104b00 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 Processor -- start 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 -- start start 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 0x5590038a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038a2730 0x5590038a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559003837670 con 0x5590038a2730 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55900380bd70 con 0x559003895e80 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.163+0000 7f445a5bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5590037fb270 con 0x5590038967a0 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f444d600640 1 -- 172.21.15.138:0/2667884501 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4444042070 con 0x7f4450104b00 2024-08-20T01:24:31.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45770/0 (socket says 172.21.15.138:45770) 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 -- 172.21.15.138:0/3267343768 learned_addr learned my addr 172.21.15.138:0/3267343768 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f444fe00640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038a2730 0x5590038a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455000640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 0x5590038a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 -- 172.21.15.138:0/3267343768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 msgr2=0x5590038a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 0x5590038a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 -- 172.21.15.138:0/3267343768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038a2730 msgr2=0x5590038a4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038a2730 0x5590038a4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 -- 172.21.15.138:0/3267343768 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559003797870 con 0x559003895e80 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455000640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 0x5590038a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.164+0000 7f4455a00640 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4448009f50 tx=0x7f44480118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d38c7ad40994958 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f4442000640 1 -- 172.21.15.138:0/3267343768 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448004d20 con 0x559003895e80 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f4442000640 1 -- 172.21.15.138:0/3267343768 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4448004050 con 0x559003895e80 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f4442000640 1 -- 172.21.15.138:0/3267343768 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448005280 con 0x559003895e80 2024-08-20T01:24:31.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f444d600640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 0x7f44380b63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f445a5bd740 1 -- 172.21.15.138:0/3267343768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 msgr2=0x559003896260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.165+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4448009f50 tx=0x7f44480118d0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 -- 172.21.15.138:0/3267343768 shutdown_connections 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038a2730 0x5590038a4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038967a0 0x5590038a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3267343768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x559003896260 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 -- 172.21.15.138:0/3267343768 >> 172.21.15.138:0/3267343768 conn(0x559003893cb0 msgr2=0x559003894e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f444d600640 1 -- 172.21.15.138:0/2667884501 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f4444104b00 con 0x7f4450104b00 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f4455000640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 0x7f44380b63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 -- 172.21.15.138:0/3267343768 shutdown_connections 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 -- 172.21.15.138:0/3267343768 wait complete. 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 -- start start 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f4455000640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 0x7f44380b63a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x559003894b90 tx=0x7f444402a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.196 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.166+0000 7f445a5bd740 1 Processor -- start 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900389f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559003781ff0 con 0x5590038967a0 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5590037ff2b0 con 0x559003895e80 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55900365dbb0 con 0x5590038a2730 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f445a5bd740 10 client.0 _ll_register_callbacks cb 0x559003506d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f444fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f4455a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f444fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39268/0 (socket says 172.21.15.138:39268) 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f444fe00640 1 -- 172.21.15.138:0/2641223106 learned_addr learned my addr 172.21.15.138:0/2641223106 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.167+0000 7f4455a00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900389f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 -- 172.21.15.138:0/2641223106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 msgr2=0x55900389f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900389f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 -- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 msgr2=0x5590038a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 -- 172.21.15.138:0/2641223106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900355ab00 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f4455a00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900389f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f444fe00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f443c00bfc0 tx=0x7f443c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9ac07f3541bf4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f4436a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c004020 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4448004050 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f443c0041c0 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f443c025050 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f445a5bd740 10 client.17376 fetch_fsmap learned FSMap version 49 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.168+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5590035d1960 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4448004050 con 0x5590038967a0 2024-08-20T01:24:31.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f443c003890 con 0x5590038967a0 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 10 client.17376 fetch_fsmap finished waiting for FSMap version 49 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 10 client.17376 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 20 client.17376 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5590038b1d40 0x5590038b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 4 client.17376 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55900380dcc0 con 0x5590038b1d40 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.169+0000 7f4455a00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5590038b1d40 0x5590038b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.170+0000 7f4455a00640 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5590038b1d40 0x5590038b4190 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf25e6aa461d457 server_cookie=d6a0358d05ba0e66 in_seq=0 out_seq=0 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.170+0000 7f4436a00640 10 client.17376 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f4436a00640 1 -- 172.21.15.138:0/2641223106 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55900380dcc0 con 0x5590038b1d40 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f4436a00640 10 client.17376 handle_command_reply: tid=0 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 client.17376 shutdown 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 msgr2=0x55900389f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f443c00bfc0 tx=0x7f443c0118d0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 shutdown_connections 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5590038b1d40 0x5590038b4190 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900389f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x55900389f340 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 --2- 172.21.15.138:0/2641223106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 >> 172.21.15.138:0/2641223106 conn(0x559003893cb0 msgr2=0x559003899820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f4436a00640 0 client.17376 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 shutdown_connections 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 1 -- 172.21.15.138:0/2641223106 wait complete. 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.171+0000 7f445a5bd740 20 client.17376 trim_cache size 0 max 16384 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 Processor -- start 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 -- start start 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x559003892b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x5590038aa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44501b43d0 con 0x5590038967a0 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44501b1c40 con 0x559003895e80 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f445a5bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44501afab0 con 0x5590038a2730 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.189+0000 7f4455a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45786/0 (socket says 172.21.15.138:45786) 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 -- 172.21.15.138:0/3193062848 learned_addr learned my addr 172.21.15.138:0/3193062848 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x5590038aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f444fe00640 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x559003892b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 -- 172.21.15.138:0/3193062848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 msgr2=0x5590038aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x5590038aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 -- 172.21.15.138:0/3193062848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 msgr2=0x559003892b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x559003892b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.201 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 -- 172.21.15.138:0/3193062848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559003894990 con 0x559003895e80 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.190+0000 7f4455a00640 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f444801b040 tx=0x7f4448005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72958ad92548eb81 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f4441600640 1 -- 172.21.15.138:0/3193062848 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448038070 con 0x559003895e80 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f4441600640 1 -- 172.21.15.138:0/3193062848 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4448004df0 con 0x559003895e80 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f4441600640 1 -- 172.21.15.138:0/3193062848 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448005750 con 0x559003895e80 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 -- 172.21.15.138:0/3193062848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 msgr2=0x5590038ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f444801b040 tx=0x7f4448005bd0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 -- 172.21.15.138:0/3193062848 shutdown_connections 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x5590038aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5590038967a0 0x559003892b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 --2- 172.21.15.138:0/3193062848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559003895e80 0x5590038ae3b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 -- 172.21.15.138:0/3193062848 >> 172.21.15.138:0/3193062848 conn(0x559003893cb0 msgr2=0x559003892420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 -- 172.21.15.138:0/3193062848 shutdown_connections 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.191+0000 7f445a5bd740 1 -- 172.21.15.138:0/3193062848 wait complete. 2024-08-20T01:24:31.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.192+0000 7f445a5bd740 1 -- start start 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.192+0000 7f445a5bd740 1 Processor -- start 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.192+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 0x5590038a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.192+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.192+0000 7f445a5bd740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900391ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f445a5bd740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f44501b0100 con 0x559003895e80 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f445a5bd740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f44501b1570 con 0x5590038967a0 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f445a5bd740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f44501b07d0 con 0x5590038a2730 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f445a5bd740 10 client.0 _ll_register_callbacks cb 0x7f443c00bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f4455a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 0x5590038a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45788/0 (socket says 172.21.15.138:45788) 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 -- 172.21.15.138:0/1638085823 learned_addr learned my addr 172.21.15.138:0/1638085823 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f4455a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 0x5590038a87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39272/0 (socket says 172.21.15.138:39272) 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f4455a00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900391ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 -- 172.21.15.138:0/1638085823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 msgr2=0x55900391ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900391ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 -- 172.21.15.138:0/1638085823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 msgr2=0x5590038a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 0x5590038a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f444fe00640 1 -- 172.21.15.138:0/1638085823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55900391bf80 con 0x5590038967a0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.193+0000 7f4455a00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900391ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f444fe00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f443c00bf90 tx=0x7f443c003880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49f52c075667a17b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f4436a00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443c00b480 con 0x5590038967a0 2024-08-20T01:24:31.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4448004df0 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f443c004130 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f443c003a70 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f445a5bd740 10 client.27404 fetch_fsmap learned FSMap version 49 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55900391d220 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.194+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4448004df0 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7f443c010850 con 0x5590038967a0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 10 client.27404 fetch_fsmap finished waiting for FSMap version 49 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 10 client.27404 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 20 client.27404 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55900391da00 0x55900391fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 4 client.27404 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c11dd335", "release": true, "await": true}] 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c11dd335", "release": true, "await": true}) -- 0x559003809760 con 0x55900391da00 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f4455a00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55900391da00 0x55900391fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f4455a00640 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55900391da00 0x55900391fe20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33647f0fc812a2a4 server_cookie=bf55a3caa1950b22 in_seq=0 out_seq=0 2024-08-20T01:24:31.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.195+0000 7f4436a00640 10 client.27404 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:31.233 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.233+0000 7f4436a00640 1 -- 172.21.15.138:0/1638085823 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559003809760 con 0x55900391da00 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.233+0000 7f4436a00640 10 client.27404 handle_command_reply: tid=0 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.233+0000 7f445a5bd740 1 client.27404 shutdown 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.233+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 msgr2=0x5590038a8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.233+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f443c00bf90 tx=0x7f443c003880 comp rx=0 tx=0).stop 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 shutdown_connections 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55900391da00 0x55900391fe20 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5590038a2730 0x55900391ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590038967a0 0x5590038a8d00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 --2- 172.21.15.138:0/1638085823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x559003895e80 0x5590038a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 >> 172.21.15.138:0/1638085823 conn(0x559003893cb0 msgr2=0x559003892c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.234 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f4436a00640 0 client.27404 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 shutdown_connections 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 1 -- 172.21.15.138:0/1638085823 wait complete. 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.234+0000 7f445a5bd740 20 client.27404 trim_cache size 0 max 16384 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 49, 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 60, 2024-08-20T01:24:31.235 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "c11dd335": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 60, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 146.6, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:24:31.236 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 msgr2=0x7f44380b63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.237 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 0x7f44380b63a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x559003894b90 tx=0x7f444402a410 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 msgr2=0x7f445019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4444005ab0 tx=0x7f4444003c80 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 shutdown_connections 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f44380b3ee0 0x7f44380b63a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f445010ae90 0x7f44501a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f445010a510 0x7f445019b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 --2- 172.21.15.138:0/2667884501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4450104b00 0x7f445019aef0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 >> 172.21.15.138:0/2667884501 conn(0x7f4450100550 msgr2=0x7f4450103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.235+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 shutdown_connections 2024-08-20T01:24:31.238 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:24:31.236+0000 7f4456400640 1 -- 172.21.15.138:0/2667884501 wait complete. 2024-08-20T01:24:31.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44166 3.96 MB/sec execute 28 sec latency 24585.658 ms 2024-08-20T01:24:32.227 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'c11dd335', total seconds elapsed: 30.6 2024-08-20T01:24:32.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 44475 4.08 MB/sec execute 29 sec latency 25264.045 ms 2024-08-20T01:24:32.867 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:32 smithi138 ceph-mon[18076]: pgmap v875: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 825 KiB/s wr, 124 op/s 2024-08-20T01:24:32.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:32 smithi167 ceph-mon[21965]: pgmap v875: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 825 KiB/s wr, 124 op/s 2024-08-20T01:24:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 -- 172.21.15.138:0/86991942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc074b40 msgr2=0x7ff8fc109e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 --2- 172.21.15.138:0/86991942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc074b40 0x7ff8fc109e80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009900 tx=0x7ff8f002a410 comp rx=0 tx=0).stop 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 -- 172.21.15.138:0/86991942 shutdown_connections 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 --2- 172.21.15.138:0/86991942 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc117640 0x7ff8fc119a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 --2- 172.21.15.138:0/86991942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc10a3c0 0x7ff8fc10a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 --2- 172.21.15.138:0/86991942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc074b40 0x7ff8fc109e80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 -- 172.21.15.138:0/86991942 >> 172.21.15.138:0/86991942 conn(0x7ff8fc06dd10 msgr2=0x7ff8fc06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 -- 172.21.15.138:0/86991942 shutdown_connections 2024-08-20T01:24:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.990+0000 7ff903200640 1 -- 172.21.15.138:0/86991942 wait complete. 2024-08-20T01:24:32.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:32 smithi142 ceph-mon[21799]: pgmap v875: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 825 KiB/s wr, 124 op/s 2024-08-20T01:24:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.993+0000 7ff903200640 1 Processor -- start 2024-08-20T01:24:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.993+0000 7ff903200640 1 -- start start 2024-08-20T01:24:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.993+0000 7ff903200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.993+0000 7ff903200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc10a3c0 0x7ff8fc1a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.993+0000 7ff903200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc117640 0x7ff8fc1a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff903200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc11d2b0 con 0x7ff8fc117640 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff903200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc11d130 con 0x7ff8fc074b40 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff903200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc11d430 con 0x7ff8fc10a3c0 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45798/0 (socket says 172.21.15.138:45798) 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 -- 172.21.15.138:0/3977292186 learned_addr learned my addr 172.21.15.138:0/3977292186 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff902800640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc117640 0x7ff8fc1a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901400640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc10a3c0 0x7ff8fc1a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 -- 172.21.15.138:0/3977292186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc10a3c0 msgr2=0x7ff8fc1a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc10a3c0 0x7ff8fc1a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 -- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc117640 msgr2=0x7ff8fc1a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc117640 0x7ff8fc1a9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.994+0000 7ff901e00640 1 -- 172.21.15.138:0/3977292186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8fc1a9850 con 0x7ff8fc074b40 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff901e00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005ab0 tx=0x7ff8f0003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:32.995 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff8f6a00640 1 -- 172.21.15.138:0/3977292186 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f003f070 con 0x7ff8fc074b40 2024-08-20T01:24:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8fc1a9ae0 con 0x7ff8fc074b40 2024-08-20T01:24:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff8f6a00640 1 -- 172.21.15.138:0/3977292186 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f0031d60 con 0x7ff8fc074b40 2024-08-20T01:24:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff8f6a00640 1 -- 172.21.15.138:0/3977292186 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0035030 con 0x7ff8fc074b40 2024-08-20T01:24:32.996 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.995+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8fc1b0200 con 0x7ff8fc074b40 2024-08-20T01:24:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 Processor -- start 2024-08-20T01:24:32.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 -- start start 2024-08-20T01:24:32.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7c7b210 con 0x55ccb7d1abd0 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7c6a710 con 0x55ccb7d0e320 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff90788a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7c6e750 con 0x55ccb7d0ec40 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff8f6a00640 1 -- 172.21.15.138:0/3977292186 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff8f0042070 con 0x7ff8fc074b40 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff902800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff902800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45802/0 (socket says 172.21.15.138:45802) 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:32.999+0000 7ff902800640 1 -- 172.21.15.138:0/720463867 learned_addr learned my addr 172.21.15.138:0/720463867 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff901e00640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff901400640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 -- 172.21.15.138:0/720463867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 msgr2=0x55ccb7d1a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 -- 172.21.15.138:0/720463867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 msgr2=0x55ccb7d1cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 -- 172.21.15.138:0/720463867 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb7c36de0 con 0x55ccb7d0e320 2024-08-20T01:24:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff901400640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff902800640 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8008da0 tx=0x7ff8f800d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474df237ff7bfbed server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.000+0000 7ff8eb000640 1 -- 172.21.15.138:0/720463867 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800f830 con 0x55ccb7d0e320 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.001+0000 7ff8eb000640 1 -- 172.21.15.138:0/720463867 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f800de20 con 0x55ccb7d0e320 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.001+0000 7ff8eb000640 1 -- 172.21.15.138:0/720463867 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f800e3b0 con 0x55ccb7d0e320 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.001+0000 7ff8f6a00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 0x7ff8e40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.001+0000 7ff90788a740 1 -- 172.21.15.138:0/720463867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 msgr2=0x55ccb7d0e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.001+0000 7ff90788a740 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8008da0 tx=0x7ff8f800d710 comp rx=0 tx=0).stop 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff8f6a00640 1 -- 172.21.15.138:0/3977292186 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7ff8f0104b20 con 0x7ff8fc074b40 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff901e00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 0x7ff8e40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 -- 172.21.15.138:0/720463867 shutdown_connections 2024-08-20T01:24:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 --2- 172.21.15.138:0/720463867 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d0e700 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 -- 172.21.15.138:0/720463867 >> 172.21.15.138:0/720463867 conn(0x55ccb7d0c150 msgr2=0x55ccb7d0d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 -- 172.21.15.138:0/720463867 shutdown_connections 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff90788a740 1 -- 172.21.15.138:0/720463867 wait complete. 2024-08-20T01:24:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.002+0000 7ff901e00640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 0x7ff8e40b6300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00c73b0 tx=0x7ff8f002a410 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.003+0000 7ff90788a740 1 -- start start 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.003+0000 7ff90788a740 1 Processor -- start 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0e320 0x55ccb7d1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d17d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7ba3ab0 con 0x55ccb7d0ec40 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7c11140 con 0x55ccb7d1abd0 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55ccb7d1d8b0 con 0x55ccb7d0e320 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff90788a740 10 client.0 _ll_register_callbacks cb 0x55ccb7c245d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff901400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff902800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0e320 0x55ccb7d1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff901400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39308/0 (socket says 172.21.15.138:39308) 2024-08-20T01:24:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.004+0000 7ff901400640 1 -- 172.21.15.138:0/3910100933 learned_addr learned my addr 172.21.15.138:0/3910100933 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff902800640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d17d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff901400640 1 -- 172.21.15.138:0/3910100933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0e320 msgr2=0x55ccb7d1a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff901400640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0e320 0x55ccb7d1a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff901400640 1 -- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 msgr2=0x55ccb7d17d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff901400640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d17d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.005+0000 7ff901400640 1 -- 172.21.15.138:0/3910100933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb7c12170 con 0x55ccb7d0ec40 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.006+0000 7ff901400640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e6b0 tx=0x7ff8ec00eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86e676d4ed9b886 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.006+0000 7ff8df400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.007+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec01c060 con 0x55ccb7d0ec40 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.007+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7ff8ec009a80 con 0x55ccb7d0ec40 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.007+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f800de20 con 0x55ccb7d0ec40 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.007+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8ec023050 con 0x55ccb7d0ec40 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.008+0000 7ff90788a740 10 client.17388 fetch_fsmap learned FSMap version 49 2024-08-20T01:24:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.008+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ccb79cd790 con 0x55ccb7d0ec40 2024-08-20T01:24:33.008 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.008+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f800de20 con 0x55ccb7d0ec40 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.008+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7ff8ec015340 con 0x55ccb7d0ec40 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 10 client.17388 fetch_fsmap finished waiting for FSMap version 49 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 10 client.17388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 20 client.17388 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d2a820 0x55ccb7d2cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 4 client.17388 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.009+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccb7c7d160 con 0x55ccb7d2a820 2024-08-20T01:24:33.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.010+0000 7ff902800640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d2a820 0x55ccb7d2cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.011+0000 7ff902800640 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d2a820 0x55ccb7d2cc10 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525ce238569da8ea server_cookie=472a5b31beea0067 in_seq=0 out_seq=0 2024-08-20T01:24:33.010 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.011+0000 7ff8df400640 10 client.17388 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.014+0000 7ff8df400640 1 -- 172.21.15.138:0/3910100933 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccb7c7d160 con 0x55ccb7d2a820 2024-08-20T01:24:33.013 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.014+0000 7ff8df400640 10 client.17388 handle_command_reply: tid=0 2024-08-20T01:24:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.014+0000 7ff90788a740 1 client.17388 shutdown 2024-08-20T01:24:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.014+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 msgr2=0x55ccb7d17820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.014 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.014+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00e6b0 tx=0x7ff8ec00eb70 comp rx=0 tx=0).stop 2024-08-20T01:24:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 shutdown_connections 2024-08-20T01:24:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d2a820 0x55ccb7d2cc10 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d17d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.015 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff8df400640 0 client.17388 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17820 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 --2- 172.21.15.138:0/3910100933 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0e320 0x55ccb7d1a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.015+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 >> 172.21.15.138:0/3910100933 conn(0x55ccb7d0c150 msgr2=0x55ccb7d11cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.016+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 shutdown_connections 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.016+0000 7ff90788a740 1 -- 172.21.15.138:0/3910100933 wait complete. 2024-08-20T01:24:33.016 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.016+0000 7ff90788a740 20 client.17388 trim_cache size 0 max 16384 2024-08-20T01:24:33.035 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.036+0000 7ff90788a740 1 Processor -- start 2024-08-20T01:24:33.036 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 -- start start 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d12e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b89c0 con 0x55ccb7d0ec40 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b6230 con 0x55ccb7d0e320 2024-08-20T01:24:33.037 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.037+0000 7ff90788a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b40a0 con 0x55ccb7d1abd0 2024-08-20T01:24:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.038+0000 7ff902800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d12e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.038+0000 7ff902800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d12e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:45828/0 (socket says 172.21.15.138:45828) 2024-08-20T01:24:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.038+0000 7ff902800640 1 -- 172.21.15.138:0/1682240743 learned_addr learned my addr 172.21.15.138:0/1682240743 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:33.038 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.038+0000 7ff902800640 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.038+0000 7ff901400640 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 -- 172.21.15.138:0/1682240743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 msgr2=0x55ccb7d12e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d12e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 -- 172.21.15.138:0/1682240743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 msgr2=0x55ccb7d17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 -- 172.21.15.138:0/1682240743 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccb7d0ea70 con 0x55ccb7d1abd0 2024-08-20T01:24:33.039 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.039+0000 7ff902800640 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d028a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8018b60 tx=0x7ff8f8000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90962779219cb3d4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.040+0000 7ff8ea600640 1 -- 172.21.15.138:0/1682240743 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80049c0 con 0x55ccb7d1abd0 2024-08-20T01:24:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.040+0000 7ff8ea600640 1 -- 172.21.15.138:0/1682240743 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f8004b60 con 0x55ccb7d1abd0 2024-08-20T01:24:33.040 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.040+0000 7ff8ea600640 1 -- 172.21.15.138:0/1682240743 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f80052a0 con 0x55ccb7d1abd0 2024-08-20T01:24:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.041+0000 7ff90788a740 1 -- 172.21.15.138:0/1682240743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 msgr2=0x55ccb7d028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.041 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.041+0000 7ff90788a740 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d028a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8018b60 tx=0x7ff8f8000f40 comp rx=0 tx=0).stop 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 -- 172.21.15.138:0/1682240743 shutdown_connections 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d028a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 --2- 172.21.15.138:0/1682240743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d0e320 0x55ccb7d12e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 -- 172.21.15.138:0/1682240743 >> 172.21.15.138:0/1682240743 conn(0x55ccb7d0c150 msgr2=0x55ccb7d02190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 -- 172.21.15.138:0/1682240743 shutdown_connections 2024-08-20T01:24:33.042 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.042+0000 7ff90788a740 1 -- 172.21.15.138:0/1682240743 wait complete. 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.043+0000 7ff90788a740 1 -- start start 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.043+0000 7ff90788a740 1 Processor -- start 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b46f0 con 0x55ccb7d0e320 2024-08-20T01:24:33.043 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b5b60 con 0x55ccb7d1abd0 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff8fc1b4dc0 con 0x55ccb7d0ec40 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff90788a740 10 client.0 _ll_register_callbacks cb 0x7ff8fc075000 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff901400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff902800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff902800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39312/0 (socket says 172.21.15.138:39312) 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff902800640 1 -- 172.21.15.138:0/2818504740 learned_addr learned my addr 172.21.15.138:0/2818504740 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:33.044 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.044+0000 7ff902800640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.045+0000 7ff902800640 1 -- 172.21.15.138:0/2818504740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 msgr2=0x55ccb7d1e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.045+0000 7ff902800640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.045+0000 7ff902800640 1 -- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 msgr2=0x55ccb7d1eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:33.045 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.045+0000 7ff902800640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.045+0000 7ff902800640 1 -- 172.21.15.138:0/2818504740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccb7a46620 con 0x55ccb7d0e320 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.046+0000 7ff902800640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8008ce0 tx=0x7ff8f8008fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca64b8800bb2b99 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.046+0000 7ff8df400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.046+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8045060 con 0x55ccb7d0e320 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.046+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7ff8f800f630 con 0x55ccb7d0e320 2024-08-20T01:24:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.046+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8f8004b60 con 0x55ccb7d0e320 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff8f804c050 con 0x55ccb7d0e320 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 10 client.17394 fetch_fsmap learned FSMap version 49 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55ccb7d951f0 con 0x55ccb7d0e320 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8f8004b60 con 0x55ccb7d0e320 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 49) ==== 8370+0+0 (secure 0 0 0) 0x7ff8f8030070 con 0x55ccb7d0e320 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 10 client.17394 fetch_fsmap finished waiting for FSMap version 49 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 20 client.17394 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d956a0 0x55ccb7d97ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 4 client.17394 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:24:33.047 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.047+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ccb7c78c00 con 0x55ccb7d956a0 2024-08-20T01:24:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.048+0000 7ff901400640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d956a0 0x55ccb7d97ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.050+0000 7ff901400640 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d956a0 0x55ccb7d97ac0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1093e5aff949c788 server_cookie=7fc44bb2e17c460b in_seq=0 out_seq=0 2024-08-20T01:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.050+0000 7ff8df400640 10 client.17394 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.187+0000 7ff8df400640 1 -- 172.21.15.138:0/2818504740 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ccb7c78c00 con 0x55ccb7d956a0 2024-08-20T01:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.187+0000 7ff8df400640 10 client.17394 handle_command_reply: tid=0 2024-08-20T01:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.187+0000 7ff90788a740 1 client.17394 shutdown 2024-08-20T01:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 msgr2=0x55ccb7d230b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8008ce0 tx=0x7ff8f8008fb0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 shutdown_connections 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55ccb7d956a0 0x55ccb7d97ac0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ccb7d1abd0 0x55ccb7d1eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55ccb7d0ec40 0x55ccb7d1e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 --2- 172.21.15.138:0/2818504740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55ccb7d0e320 0x55ccb7d230b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 >> 172.21.15.138:0/2818504740 conn(0x55ccb7d0c150 msgr2=0x55ccb7d259f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff8df400640 0 client.17394 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 shutdown_connections 2024-08-20T01:24:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.188+0000 7ff90788a740 1 -- 172.21.15.138:0/2818504740 wait complete. 2024-08-20T01:24:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.189+0000 7ff90788a740 20 client.17394 trim_cache size 0 max 16384 2024-08-20T01:24:33.189 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:24:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:24:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:24:33.190 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:24:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 msgr2=0x7ff8e40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 0x7ff8e40b6300 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff8f00c73b0 tx=0x7ff8f002a410 comp rx=0 tx=0).stop 2024-08-20T01:24:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 msgr2=0x7ff8fc1a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0005ab0 tx=0x7ff8f0003d50 comp rx=0 tx=0).stop 2024-08-20T01:24:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 shutdown_connections 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff8e40b3e40 0x7ff8e40b6300 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8fc117640 0x7ff8fc1a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff8fc10a3c0 0x7ff8fc1a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 --2- 172.21.15.138:0/3977292186 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8fc074b40 0x7ff8fc1a37b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.191+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 >> 172.21.15.138:0/3977292186 conn(0x7ff8fc06dd10 msgr2=0x7ff8fc118bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.192+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 shutdown_connections 2024-08-20T01:24:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:33.192+0000 7ff903200640 1 -- 172.21.15.138:0/3977292186 wait complete. 2024-08-20T01:24:33.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 45121 4.80 MB/sec execute 30 sec latency 81.591 ms 2024-08-20T01:24:34.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 45998 6.28 MB/sec execute 31 sec latency 73.884 ms 2024-08-20T01:24:34.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:34 smithi167 ceph-mon[21965]: pgmap v876: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-08-20T01:24:34.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:34 smithi142 ceph-mon[21799]: pgmap v876: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-08-20T01:24:35.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:34 smithi138 ceph-mon[18076]: pgmap v876: 129 pgs: 129 active+clean; 908 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 225 KiB/s wr, 43 op/s 2024-08-20T01:24:35.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 47052 8.11 MB/sec execute 32 sec latency 67.585 ms 2024-08-20T01:24:36.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 47727 8.94 MB/sec execute 33 sec latency 235.436 ms 2024-08-20T01:24:36.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:36 smithi167 ceph-mon[21965]: pgmap v877: 129 pgs: 129 active+clean; 907 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 81 op/s 2024-08-20T01:24:36.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:36 smithi142 ceph-mon[21799]: pgmap v877: 129 pgs: 129 active+clean; 907 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 81 op/s 2024-08-20T01:24:37.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:36 smithi138 ceph-mon[18076]: pgmap v877: 129 pgs: 129 active+clean; 907 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 554 KiB/s wr, 81 op/s 2024-08-20T01:24:37.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 48553 9.64 MB/sec execute 34 sec latency 46.154 ms 2024-08-20T01:24:38.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 49568 11.12 MB/sec execute 35 sec latency 54.042 ms 2024-08-20T01:24:38.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:38 smithi167 ceph-mon[21965]: pgmap v878: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-20T01:24:38.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:38 smithi142 ceph-mon[21799]: pgmap v878: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-20T01:24:39.030 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:38 smithi138 ceph-mon[18076]: pgmap v878: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.7 MiB/s wr, 280 op/s 2024-08-20T01:24:39.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 50393 12.20 MB/sec execute 36 sec latency 53.894 ms 2024-08-20T01:24:40.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 51245 13.18 MB/sec execute 37 sec latency 43.884 ms 2024-08-20T01:24:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:40 smithi167 ceph-mon[21965]: pgmap v879: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-08-20T01:24:40.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:40 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:40.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:40 smithi142 ceph-mon[21799]: pgmap v879: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-08-20T01:24:40.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:40 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:41.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:40 smithi138 ceph-mon[18076]: pgmap v879: 129 pgs: 129 active+clean; 921 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.7 MiB/s wr, 282 op/s 2024-08-20T01:24:41.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:40 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:41.277 INFO:tasks.workunit.client.0.smithi138.stdout: 10 52031 13.84 MB/sec execute 38 sec latency 93.243 ms 2024-08-20T01:24:42.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 52973 14.84 MB/sec execute 39 sec latency 62.747 ms 2024-08-20T01:24:42.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:42 smithi167 ceph-mon[21965]: pgmap v880: 129 pgs: 129 active+clean; 943 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2024-08-20T01:24:42.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:42 smithi142 ceph-mon[21799]: pgmap v880: 129 pgs: 129 active+clean; 943 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2024-08-20T01:24:43.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:42 smithi138 ceph-mon[18076]: pgmap v880: 129 pgs: 129 active+clean; 943 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 4.9 MiB/s wr, 441 op/s 2024-08-20T01:24:43.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 53811 15.36 MB/sec execute 40 sec latency 117.272 ms 2024-08-20T01:24:44.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 54415 15.94 MB/sec execute 41 sec latency 313.119 ms 2024-08-20T01:24:44.944 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:44 smithi138 ceph-mon[18076]: pgmap v881: 129 pgs: 129 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-08-20T01:24:44.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:44 smithi167 ceph-mon[21965]: pgmap v881: 129 pgs: 129 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-08-20T01:24:44.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:44 smithi142 ceph-mon[21799]: pgmap v881: 129 pgs: 129 active+clean; 952 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 5.7 MiB/s wr, 495 op/s 2024-08-20T01:24:45.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 55246 16.52 MB/sec execute 42 sec latency 254.306 ms 2024-08-20T01:24:46.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 56105 17.42 MB/sec execute 43 sec latency 75.010 ms 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.367+0000 7f1341a00640 1 -- 172.21.15.138:0/2607728237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075270 msgr2=0x7f133c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.367+0000 7f1341a00640 1 --2- 172.21.15.138:0/2607728237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075270 0x7f133c075650 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1334009ca0 tx=0x7f133402a8e0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 -- 172.21.15.138:0/2607728237 shutdown_connections 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 --2- 172.21.15.138:0/2607728237 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c113730 0x7f133c115b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 --2- 172.21.15.138:0/2607728237 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c075b90 0x7f133c1131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 --2- 172.21.15.138:0/2607728237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075270 0x7f133c075650 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 -- 172.21.15.138:0/2607728237 >> 172.21.15.138:0/2607728237 conn(0x7f133c0fe420 msgr2=0x7f133c100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 -- 172.21.15.138:0/2607728237 shutdown_connections 2024-08-20T01:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.368+0000 7f1341a00640 1 -- 172.21.15.138:0/2607728237 wait complete. 2024-08-20T01:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.369+0000 7f1341a00640 1 Processor -- start 2024-08-20T01:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.369+0000 7f1341a00640 1 -- start start 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 0x7f133c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c113730 0x7f133c1a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f133c119080 con 0x7f133c113730 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f133c118f00 con 0x7f133c075b90 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f1341a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f133c119200 con 0x7f133c075270 2024-08-20T01:24:46.370 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f133a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f133a000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38344/0 (socket says 172.21.15.138:38344) 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f133a000640 1 -- 172.21.15.138:0/3062813580 learned_addr learned my addr 172.21.15.138:0/3062813580 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f133aa00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 0x7f133c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.370+0000 7f133b400640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c113730 0x7f133c1a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 -- 172.21.15.138:0/3062813580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 msgr2=0x7f133c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 0x7f133c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 -- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c113730 msgr2=0x7f133c1a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c113730 0x7f133c1a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 -- 172.21.15.138:0/3062813580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a5700 con 0x7f133c075b90 2024-08-20T01:24:46.371 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133aa00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 0x7f133c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.371+0000 7f133a000640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d950 tx=0x7f132c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.372+0000 7f1333400640 1 -- 172.21.15.138:0/3062813580 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c017030 con 0x7f133c075b90 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.372+0000 7f1333400640 1 -- 172.21.15.138:0/3062813580 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132c004020 con 0x7f133c075b90 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.372+0000 7f1333400640 1 -- 172.21.15.138:0/3062813580 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0047a0 con 0x7f133c075b90 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.372+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a59f0 con 0x7f133c075b90 2024-08-20T01:24:46.372 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.372+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1ac0b0 con 0x7f133c075b90 2024-08-20T01:24:46.377 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.377+0000 7f134486f740 1 Processor -- start 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 -- start start 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf82ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b88ceec210 con 0x55b88cf77250 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b88cedb710 con 0x55b88cf831e0 2024-08-20T01:24:46.378 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f134486f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b88cedf750 con 0x55b88cf76930 2024-08-20T01:24:46.379 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.378+0000 7f133b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55880/0 (socket says 172.21.15.138:55880) 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 -- 172.21.15.138:0/4019301760 learned_addr learned my addr 172.21.15.138:0/4019301760 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133aa00640 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf82ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 -- 172.21.15.138:0/4019301760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 msgr2=0x55b88cf855d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf855d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 -- 172.21.15.138:0/4019301760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 msgr2=0x55b88cf82ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf82ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 -- 172.21.15.138:0/4019301760 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88cea7de0 con 0x55b88cf76930 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f1333400640 1 -- 172.21.15.138:0/3062813580 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f132c009a30 con 0x7f133c075b90 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.379+0000 7f133b400640 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1328009990 tx=0x7f1328009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50c10adb205c8dbc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.380+0000 7f1327e00640 1 -- 172.21.15.138:0/4019301760 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328017070 con 0x55b88cf76930 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.380+0000 7f1327e00640 1 -- 172.21.15.138:0/4019301760 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1328013470 con 0x55b88cf76930 2024-08-20T01:24:46.380 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.380+0000 7f1327e00640 1 -- 172.21.15.138:0/4019301760 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280125d0 con 0x55b88cf76930 2024-08-20T01:24:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 -- 172.21.15.138:0/4019301760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 msgr2=0x55b88cf76d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1328009990 tx=0x7f1328009e50 comp rx=0 tx=0).stop 2024-08-20T01:24:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 -- 172.21.15.138:0/4019301760 shutdown_connections 2024-08-20T01:24:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.381 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf82ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 --2- 172.21.15.138:0/4019301760 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf76d10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 -- 172.21.15.138:0/4019301760 >> 172.21.15.138:0/4019301760 conn(0x55b88cf74760 msgr2=0x55b88cf75900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 -- 172.21.15.138:0/4019301760 shutdown_connections 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.381+0000 7f134486f740 1 -- 172.21.15.138:0/4019301760 wait complete. 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.382+0000 7f134486f740 1 -- start start 2024-08-20T01:24:46.382 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.382+0000 7f134486f740 1 Processor -- start 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf76930 0x55b88cf82940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf802c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b88ce14ab0 con 0x55b88cf76930 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b88ce82140 con 0x55b88cf831e0 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b88cf85ec0 con 0x55b88cf77250 2024-08-20T01:24:46.383 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.383+0000 7f134486f740 10 client.0 _ll_register_callbacks cb 0x55b88ce955d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f133aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f133aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55894/0 (socket says 172.21.15.138:55894) 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f133aa00640 1 -- 172.21.15.138:0/2259594264 learned_addr learned my addr 172.21.15.138:0/2259594264 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f133b400640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf76930 0x55b88cf82940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f1333400640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 0x7f13200b6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.384 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.384+0000 7f133a000640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf802c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.385+0000 7f1333400640 1 -- 172.21.15.138:0/3062813580 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f132c0e3d90 con 0x7f133c075b90 2024-08-20T01:24:46.385 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.385+0000 7f133aa00640 1 -- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 msgr2=0x55b88cf802c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.385+0000 7f133aa00640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf802c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.385+0000 7f133aa00640 1 -- 172.21.15.138:0/2259594264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf76930 msgr2=0x55b88cf82940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f133aa00640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf76930 0x55b88cf82940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f133aa00640 1 -- 172.21.15.138:0/2259594264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88ce83170 con 0x55b88cf77250 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.385+0000 7f133b400640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 0x7f13200b6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f133a000640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f133aa00640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f133402ae00 tx=0x7f1334031cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c8bfcaaa2c34bd6 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.386 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f1324c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334003d80 con 0x55b88cf77250 2024-08-20T01:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1328013470 con 0x55b88cf77250 2024-08-20T01:24:46.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.386+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f1334004040 con 0x55b88cf77250 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.387+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1334004240 con 0x55b88cf77250 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 10 client.27394 fetch_fsmap learned FSMap version 50 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b88cc3e790 con 0x55b88cf77250 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1328013470 con 0x55b88cf77250 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f1334030ba0 con 0x55b88cf77250 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 10 client.27394 fetch_fsmap finished waiting for FSMap version 50 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 10 client.27394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 20 client.27394 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:46.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.388+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88cf92d70 0x55b88cf95160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.389+0000 7f134486f740 4 client.27394 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.389+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b88ceee160 con 0x55b88cf92d70 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.389+0000 7f133a000640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88cf92d70 0x55b88cf95160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.389+0000 7f133b400640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 0x7f13200b6260 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1328013e70 tx=0x7f1328009060 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.390+0000 7f133a000640 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88cf92d70 0x55b88cf95160 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba52fd9153c18cdf server_cookie=2b9c99343daa9510 in_seq=0 out_seq=0 2024-08-20T01:24:46.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.390+0000 7f1324c00640 10 client.27394 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.393+0000 7f1324c00640 1 -- 172.21.15.138:0/2259594264 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b88ceee160 con 0x55b88cf92d70 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.394+0000 7f1324c00640 10 client.27394 handle_command_reply: tid=0 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.394+0000 7f134486f740 1 client.27394 shutdown 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.394+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 msgr2=0x55b88cf7fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.394+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f133402ae00 tx=0x7f1334031cc0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.394 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 shutdown_connections 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88cf92d70 0x55b88cf95160 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf802c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf77250 0x55b88cf7fd80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 --2- 172.21.15.138:0/2259594264 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf76930 0x55b88cf82940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 >> 172.21.15.138:0/2259594264 conn(0x55b88cf74760 msgr2=0x55b88cf7a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.395+0000 7f1324c00640 0 client.27394 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.396+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 shutdown_connections 2024-08-20T01:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.397+0000 7f134486f740 1 -- 172.21.15.138:0/2259594264 wait complete. 2024-08-20T01:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.397+0000 7f134486f740 20 client.27394 trim_cache size 0 max 16384 2024-08-20T01:24:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.424+0000 7f134486f740 1 Processor -- start 2024-08-20T01:24:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.424+0000 7f134486f740 1 -- start start 2024-08-20T01:24:46.424 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf9af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf7abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b8b10 con 0x55b88cf77250 2024-08-20T01:24:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b63b0 con 0x55b88cf831e0 2024-08-20T01:24:46.425 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f134486f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b4220 con 0x55b88cf76930 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f133aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f133aa00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55904/0 (socket says 172.21.15.138:55904) 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f133aa00640 1 -- 172.21.15.138:0/3958293759 learned_addr learned my addr 172.21.15.138:0/3958293759 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.425+0000 7f133b400640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf7abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 -- 172.21.15.138:0/3958293759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 msgr2=0x55b88cf7abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf7abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 -- 172.21.15.138:0/3958293759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 msgr2=0x55b88cf9af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133a000640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf9af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.426 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf9af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 -- 172.21.15.138:0/3958293759 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b88ccb7620 con 0x55b88cf76930 2024-08-20T01:24:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133b400640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf7abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:24:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133a000640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf9af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:24:46.427 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.426+0000 7f133aa00640 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1334009bf0 tx=0x7f13340093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70d56a0552f33213 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.427+0000 7f1327400640 1 -- 172.21.15.138:0/3958293759 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334057070 con 0x55b88cf76930 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.427+0000 7f1327400640 1 -- 172.21.15.138:0/3958293759 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1334036040 con 0x55b88cf76930 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.427+0000 7f1327400640 1 -- 172.21.15.138:0/3958293759 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334057070 con 0x55b88cf76930 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 -- 172.21.15.138:0/3958293759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 msgr2=0x55b88cf9a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1334009bf0 tx=0x7f13340093d0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 -- 172.21.15.138:0/3958293759 shutdown_connections 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf831e0 0x55b88cf7abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.428 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf9af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 --2- 172.21.15.138:0/3958293759 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf76930 0x55b88cf9a9f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.428+0000 7f134486f740 1 -- 172.21.15.138:0/3958293759 >> 172.21.15.138:0/3958293759 conn(0x55b88cf74760 msgr2=0x55b88cf7e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.430+0000 7f134486f740 1 -- 172.21.15.138:0/3958293759 shutdown_connections 2024-08-20T01:24:46.429 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.430+0000 7f134486f740 1 -- 172.21.15.138:0/3958293759 wait complete. 2024-08-20T01:24:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.430+0000 7f134486f740 1 -- start start 2024-08-20T01:24:46.430 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.431+0000 7f134486f740 1 Processor -- start 2024-08-20T01:24:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.431+0000 7f134486f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.431+0000 7f134486f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf8b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.431 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.431+0000 7f134486f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.432+0000 7f133b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.432+0000 7f133b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:55910/0 (socket says 172.21.15.138:55910) 2024-08-20T01:24:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.432+0000 7f133aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.432 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.432+0000 7f133aa00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38376/0 (socket says 172.21.15.138:38376) 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b4870 con 0x55b88cf77250 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b5ce0 con 0x55b88cf76930 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f133a000640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf8b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1b4f40 con 0x55b88cf831e0 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f134486f740 10 client.0 _ll_register_callbacks cb 0x7f1334009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.433+0000 7f133b400640 1 -- 172.21.15.138:0/3865740421 learned_addr learned my addr 172.21.15.138:0/3865740421 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:24:46.433 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133aa00640 1 -- 172.21.15.138:0/3865740421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 msgr2=0x55b88cff6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133aa00640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133aa00640 1 -- 172.21.15.138:0/3865740421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 msgr2=0x55b88cf8b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:24:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133aa00640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf8b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133aa00640 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b88cff9630 con 0x55b88cf76930 2024-08-20T01:24:46.434 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133b400640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.434+0000 7f133a000640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.435+0000 7f133aa00640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f133402ae00 tx=0x7f1334004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ea604dd9081abb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.435+0000 7f1324c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.435+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334057070 con 0x55b88cf76930 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.436+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f1334004b40 con 0x55b88cf76930 2024-08-20T01:24:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.436+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1334036040 con 0x55b88cf76930 2024-08-20T01:24:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.437+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f1334004d40 con 0x55b88cf76930 2024-08-20T01:24:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.437+0000 7f134486f740 10 client.27434 fetch_fsmap learned FSMap version 50 2024-08-20T01:24:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.437+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55b88d00ee30 con 0x55b88cf76930 2024-08-20T01:24:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.437+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1334036040 con 0x55b88cf76930 2024-08-20T01:24:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) ==== 8366+0+0 (secure 0 0 0) 0x7f13340305e0 con 0x55b88cf76930 2024-08-20T01:24:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 10 client.27434 fetch_fsmap finished waiting for FSMap version 50 2024-08-20T01:24:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 10 client.27434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:24:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 20 client.27434 populate_metadata read hostname 'smithi138' 2024-08-20T01:24:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88d00f300 0x55b88d0116f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:24:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 4 client.27434 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:24:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.438+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b88cee9c00 con 0x55b88d00f300 2024-08-20T01:24:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.439+0000 7f133a000640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88d00f300 0x55b88d0116f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:24:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.440+0000 7f133a000640 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88d00f300 0x55b88d0116f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71148d9e346a85e8 server_cookie=f4848194fca08105 in_seq=0 out_seq=0 2024-08-20T01:24:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:46.440+0000 7f1324c00640 10 client.27434 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:46.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:46 smithi138 ceph-mon[18076]: pgmap v882: 129 pgs: 129 active+clean; 962 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-08-20T01:24:46.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:46 smithi167 ceph-mon[21965]: pgmap v882: 129 pgs: 129 active+clean; 962 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-08-20T01:24:46.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:46 smithi142 ceph-mon[21799]: pgmap v882: 129 pgs: 129 active+clean; 962 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 6.6 MiB/s wr, 559 op/s 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.222+0000 7f1324c00640 1 -- 172.21.15.138:0/3865740421 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b88cee9c00 con 0x55b88d00f300 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.222+0000 7f1324c00640 10 client.27434 handle_command_reply: tid=0 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.222+0000 7f134486f740 1 client.27434 shutdown 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 msgr2=0x55b88cf8b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f133402ae00 tx=0x7f1334004610 comp rx=0 tx=0).stop 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 shutdown_connections 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b88d00f300 0x55b88d0116f0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b88cf831e0 0x55b88cff6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b88cf77250 0x55b88cf8b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.223 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 --2- 172.21.15.138:0/3865740421 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b88cf76930 0x55b88cf8b180 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f1324c00640 0 client.27434 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 >> 172.21.15.138:0/3865740421 conn(0x55b88cf74760 msgr2=0x55b88cf7aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 shutdown_connections 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 1 -- 172.21.15.138:0/3865740421 wait complete. 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.223+0000 7f134486f740 20 client.27434 trim_cache size 0 max 16384 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:24:47.224 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:24:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 msgr2=0x7f13200b6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 0x7f13200b6260 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1328013e70 tx=0x7f1328009060 comp rx=0 tx=0).stop 2024-08-20T01:24:47.227 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 msgr2=0x7f133c19fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:24:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d950 tx=0x7f132c00de10 comp rx=0 tx=0).stop 2024-08-20T01:24:47.228 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 shutdown_connections 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f13200b3da0 0x7f13200b6260 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c113730 0x7f133c1a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f133c075b90 0x7f133c19fba0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 --2- 172.21.15.138:0/3062813580 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f133c075270 0x7f133c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 >> 172.21.15.138:0/3062813580 conn(0x7f133c0fe420 msgr2=0x7f133c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 shutdown_connections 2024-08-20T01:24:47.229 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:24:47.228+0000 7f1341a00640 1 -- 172.21.15.138:0/3062813580 wait complete. 2024-08-20T01:24:47.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 56474 17.50 MB/sec execute 44 sec latency 377.469 ms 2024-08-20T01:24:48.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 57113 17.61 MB/sec execute 45 sec latency 277.390 ms 2024-08-20T01:24:48.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:48 smithi167 ceph-mon[21965]: pgmap v883: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-08-20T01:24:48.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:48 smithi142 ceph-mon[21799]: pgmap v883: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-08-20T01:24:49.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:48 smithi138 ceph-mon[18076]: pgmap v883: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-08-20T01:24:49.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58025 18.29 MB/sec execute 46 sec latency 226.541 ms 2024-08-20T01:24:50.278 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58689 18.85 MB/sec execute 47 sec latency 323.314 ms 2024-08-20T01:24:50.821 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:50 smithi138 ceph-mon[18076]: pgmap v884: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 465 op/s 2024-08-20T01:24:50.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:50 smithi167 ceph-mon[21965]: pgmap v884: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 465 op/s 2024-08-20T01:24:50.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:50 smithi142 ceph-mon[21799]: pgmap v884: 129 pgs: 129 active+clean; 983 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 6.2 MiB/s wr, 465 op/s 2024-08-20T01:24:51.282 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58695 18.46 MB/sec execute 48 sec latency 1152.799 ms 2024-08-20T01:24:52.282 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58919 18.35 MB/sec execute 49 sec latency 1742.255 ms 2024-08-20T01:24:52.964 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:52 smithi167 ceph-mon[21965]: pgmap v885: 129 pgs: 129 active+clean; 1003 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.7 MiB/s wr, 729 op/s 2024-08-20T01:24:52.992 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:52 smithi142 ceph-mon[21799]: pgmap v885: 129 pgs: 129 active+clean; 1003 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.7 MiB/s wr, 729 op/s 2024-08-20T01:24:53.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:52 smithi138 ceph-mon[18076]: pgmap v885: 129 pgs: 129 active+clean; 1003 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.7 MiB/s wr, 729 op/s 2024-08-20T01:24:53.282 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58919 17.98 MB/sec execute 50 sec latency 1135.412 ms 2024-08-20T01:24:54.282 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58919 17.63 MB/sec execute 51 sec latency 2135.455 ms 2024-08-20T01:24:55.282 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58924 17.29 MB/sec execute 52 sec latency 3118.267 ms 2024-08-20T01:24:55.840 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:55 smithi138 ceph-mon[18076]: pgmap v886: 129 pgs: 129 active+clean; 1002 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-08-20T01:24:56.091 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:55 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:56.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:55 smithi167 ceph-mon[21965]: pgmap v886: 129 pgs: 129 active+clean; 1002 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-08-20T01:24:56.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:55 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:56.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:55 smithi142 ceph-mon[21799]: pgmap v886: 129 pgs: 129 active+clean; 1002 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 7.9 MiB/s wr, 662 op/s 2024-08-20T01:24:56.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:55 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:24:56.305 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58937 17.01 MB/sec execute 53 sec latency 3587.009 ms 2024-08-20T01:24:56.886 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:56 smithi138 ceph-mon[18076]: pgmap v887: 129 pgs: 129 active+clean; 985 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 651 op/s 2024-08-20T01:24:57.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:56 smithi167 ceph-mon[21965]: pgmap v887: 129 pgs: 129 active+clean; 985 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 651 op/s 2024-08-20T01:24:57.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:56 smithi142 ceph-mon[21799]: pgmap v887: 129 pgs: 129 active+clean; 985 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.2 MiB/s wr, 651 op/s 2024-08-20T01:24:57.312 INFO:tasks.workunit.client.0.smithi138.stdout: 10 58996 16.77 MB/sec execute 54 sec latency 1785.225 ms 2024-08-20T01:24:57.979 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:57 smithi138 ceph-mon[18076]: pgmap v888: 129 pgs: 129 active+clean; 958 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 650 op/s 2024-08-20T01:24:58.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:57 smithi167 ceph-mon[21965]: pgmap v888: 129 pgs: 129 active+clean; 958 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 650 op/s 2024-08-20T01:24:58.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:57 smithi142 ceph-mon[21799]: pgmap v888: 129 pgs: 129 active+clean; 958 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 8.8 MiB/s wr, 650 op/s 2024-08-20T01:24:58.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59083 16.52 MB/sec execute 55 sec latency 2218.065 ms 2024-08-20T01:24:59.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59309 16.39 MB/sec execute 56 sec latency 1068.134 ms 2024-08-20T01:25:00.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:24:59 smithi167 ceph-mon[21965]: pgmap v889: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 527 op/s 2024-08-20T01:25:00.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:24:59 smithi142 ceph-mon[21799]: pgmap v889: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 527 op/s 2024-08-20T01:25:00.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:24:59 smithi138 ceph-mon[18076]: pgmap v889: 129 pgs: 129 active+clean; 967 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.7 MiB/s wr, 527 op/s 2024-08-20T01:25:00.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59531 16.28 MB/sec execute 57 sec latency 2068.197 ms 2024-08-20T01:25:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/921325851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f985806a7a0 msgr2=0x7f985806ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- 172.21.15.138:0/921325851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f985806a7a0 0x7f985806ab80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f984c009900 tx=0x7f984c02a400 comp rx=0 tx=0).stop 2024-08-20T01:25:00.373 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/921325851 shutdown_connections 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- 172.21.15.138:0/921325851 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858105970 0x7f985810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- 172.21.15.138:0/921325851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9858104fd0 0x7f9858105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- 172.21.15.138:0/921325851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f985806a7a0 0x7f985806ab80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/921325851 >> 172.21.15.138:0/921325851 conn(0x7f9858077270 msgr2=0x7f9858077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/921325851 shutdown_connections 2024-08-20T01:25:00.374 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/921325851 wait complete. 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 Processor -- start 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- start start 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 0x7f985819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858104fd0 0x7f985819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9858112850 con 0x7f985806a7a0 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98581126d0 con 0x7f9858105970 2024-08-20T01:25:00.375 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98581129d0 con 0x7f9858104fd0 2024-08-20T01:25:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47612/0 (socket says 172.21.15.138:47612) 2024-08-20T01:25:00.387 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 -- 172.21.15.138:0/482436948 learned_addr learned my addr 172.21.15.138:0/482436948 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985d400640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858104fd0 0x7f985819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985de00640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 0x7f985819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 -- 172.21.15.138:0/482436948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858104fd0 msgr2=0x7f985819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858104fd0 0x7f985819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 -- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 msgr2=0x7f985819af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 0x7f985819af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 -- 172.21.15.138:0/482436948 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98581a0ff0 con 0x7f9858105970 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985de00640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 0x7f985819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985e800640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f985400eb10 tx=0x7f985400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f9852a00640 1 -- 172.21.15.138:0/482436948 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854004b20 con 0x7f9858105970 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f9852a00640 1 -- 172.21.15.138:0/482436948 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9854004cc0 con 0x7f9858105970 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.371+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98581a12e0 con 0x7f9858105970 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f9852a00640 1 -- 172.21.15.138:0/482436948 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854010690 con 0x7f9858105970 2024-08-20T01:25:00.388 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98581a7b80 con 0x7f9858105970 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f9863843740 1 Processor -- start 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f9852a00640 1 -- 172.21.15.138:0/482436948 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9854005020 con 0x7f9858105970 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f9863843740 1 -- start start 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.376+0000 7f9863843740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55935656d210 con 0x5593565f8430 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55935655c710 con 0x5593565f7b10 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559356560750 con 0x5593566043c0 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51422/0 (socket says 172.21.15.138:51422) 2024-08-20T01:25:00.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 -- 172.21.15.138:0/2256743075 learned_addr learned my addr 172.21.15.138:0/2256743075 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47628/0 (socket says 172.21.15.138:47628) 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 -- 172.21.15.138:0/2256743075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 msgr2=0x5593566067b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566067b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 -- 172.21.15.138:0/2256743075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 msgr2=0x5593565f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 -- 172.21.15.138:0/2256743075 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559356528de0 con 0x5593565f8430 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985de00640 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f984c031440 tx=0x7f984c0042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa01e65f3d853c2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9852a00640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 0x7f98400b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9847000640 1 -- 172.21.15.138:0/2256743075 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c035030 con 0x5593565f8430 2024-08-20T01:25:00.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9847000640 1 -- 172.21.15.138:0/2256743075 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984c0044d0 con 0x5593565f8430 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9847000640 1 -- 172.21.15.138:0/2256743075 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0035f0 con 0x5593565f8430 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9852a00640 1 -- 172.21.15.138:0/482436948 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f98540e4930 con 0x7f9858105970 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 0x7f98400b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 -- 172.21.15.138:0/2256743075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 msgr2=0x559356603e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f9863843740 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f984c031440 tx=0x7f984c0042e0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.381+0000 7f985d400640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 0x7f98400b6280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9848009730 tx=0x7f98480096c0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2256743075 shutdown_connections 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f8430 0x559356603e80 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- 172.21.15.138:0/2256743075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f7b10 0x5593565f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2256743075 >> 172.21.15.138:0/2256743075 conn(0x5593565f5940 msgr2=0x5593565f6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2256743075 shutdown_connections 2024-08-20T01:25:00.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2256743075 wait complete. 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- start start 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 Processor -- start 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x559356603b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566014c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x559356495ab0 con 0x5593565f7b10 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x559356503140 con 0x5593565f8430 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x559356607040 con 0x5593566043c0 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 10 client.0 _ll_register_callbacks cb 0x5593565165d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47636/0 (socket says 172.21.15.138:47636) 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 -- 172.21.15.138:0/2163692383 learned_addr learned my addr 172.21.15.138:0/2163692383 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985e800640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985de00640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x559356603b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 -- 172.21.15.138:0/2163692383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 msgr2=0x5593566014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 -- 172.21.15.138:0/2163692383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 msgr2=0x559356603b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x559356603b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.393 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 -- 172.21.15.138:0/2163692383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559356504170 con 0x5593565f8430 2024-08-20T01:25:00.406 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985de00640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x559356603b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985d400640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9848004550 tx=0x7f9848004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef79e0db174ed815 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848022830 con 0x5593565f8430 2024-08-20T01:25:00.407 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f98480229d0 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f984c0044d0 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f9848034050 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 10 client.27446 fetch_fsmap learned FSMap version 51 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5593562bf790 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f984c0044d0 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f984802cc20 con 0x5593565f8430 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 10 client.27446 fetch_fsmap finished waiting for FSMap version 51 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 10 client.27446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 20 client.27446 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356625000 0x559356627450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.409 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 4 client.27446 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55935656f160 con 0x559356625000 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.384+0000 7f985de00640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356625000 0x559356627450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f985de00640 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356625000 0x559356627450 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba9ff615cd61815 server_cookie=59996acae800222f in_seq=0 out_seq=0 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f983b400640 10 client.27446 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f983b400640 1 -- 172.21.15.138:0/2163692383 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55935656f160 con 0x559356625000 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f983b400640 10 client.27446 handle_command_reply: tid=0 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 client.27446 shutdown 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 msgr2=0x559356600f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9848004550 tx=0x7f9848004730 comp rx=0 tx=0).stop 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 shutdown_connections 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356625000 0x559356627450 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593566014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x559356600f80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.410 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 --2- 172.21.15.138:0/2163692383 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x559356603b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 >> 172.21.15.138:0/2163692383 conn(0x5593565f5940 msgr2=0x5593565fb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f983b400640 0 client.27446 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 shutdown_connections 2024-08-20T01:25:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 1 -- 172.21.15.138:0/2163692383 wait complete. 2024-08-20T01:25:00.411 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.389+0000 7f9863843740 20 client.27446 trim_cache size 0 max 16384 2024-08-20T01:25:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 Processor -- start 2024-08-20T01:25:00.413 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- start start 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x55935662cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935662d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593565fbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f98581abbd0 con 0x5593565f7b10 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98581a94a0 con 0x5593565f8430 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f985806ab80 con 0x5593566043c0 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x55935662cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935662d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x55935662cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:51436/0 (socket says 172.21.15.138:51436) 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985de00640 1 -- 172.21.15.138:0/4124968779 learned_addr learned my addr 172.21.15.138:0/4124968779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985e800640 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593565fbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 -- 172.21.15.138:0/4124968779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 msgr2=0x5593565fbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.414 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593565fbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 -- 172.21.15.138:0/4124968779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 msgr2=0x55935662cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x55935662cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 -- 172.21.15.138:0/4124968779 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559356338620 con 0x5593565f8430 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f985d400640 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935662d260 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f98480234f0 tx=0x7f9848005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e869589696cc5f89 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9846600640 1 -- 172.21.15.138:0/4124968779 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984803d070 con 0x5593565f8430 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9846600640 1 -- 172.21.15.138:0/4124968779 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9848031070 con 0x5593565f8430 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9846600640 1 -- 172.21.15.138:0/4124968779 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9848022b40 con 0x5593565f8430 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- 172.21.15.138:0/4124968779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 msgr2=0x55935662d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935662d260 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f98480234f0 tx=0x7f9848005830 comp rx=0 tx=0).stop 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- 172.21.15.138:0/4124968779 shutdown_connections 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593566043c0 0x5593565fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935662d260 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.415 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- 172.21.15.138:0/4124968779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593565f7b10 0x55935662cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- 172.21.15.138:0/4124968779 >> 172.21.15.138:0/4124968779 conn(0x5593565f5940 msgr2=0x5593565f6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- 172.21.15.138:0/4124968779 shutdown_connections 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- 172.21.15.138:0/4124968779 wait complete. 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- start start 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 Processor -- start 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593565f7b10 0x55935660c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593566043c0 0x559356607c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9858074d20 con 0x5593566043c0 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f98581a8dd0 con 0x5593565f8430 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f98581a8030 con 0x5593565f7b10 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.407+0000 7f9863843740 10 client.0 _ll_register_callbacks cb 0x7f9848010f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:00.416 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593565f7b10 0x55935660c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:47654/0 (socket says 172.21.15.138:47654) 2024-08-20T01:25:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 -- 172.21.15.138:0/4271536688 learned_addr learned my addr 172.21.15.138:0/4271536688 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:00.417 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985e800640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593566043c0 0x559356607c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 -- 172.21.15.138:0/4271536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593565f7b10 msgr2=0x55935660c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593565f7b10 0x55935660c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 -- 172.21.15.138:0/4271536688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593566043c0 msgr2=0x559356607c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593566043c0 0x559356607c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 -- 172.21.15.138:0/4271536688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559356608490 con 0x5593565f8430 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985d400640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f98480058a0 tx=0x7f9848003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e3dd616026dd717 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:00.418 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984803d070 con 0x5593565f8430 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 9453+0+0 (secure 0 0 0) 0x7f9848023aa0 con 0x5593565f8430 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9848031070 con 0x5593565f8430 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f984804b050 con 0x5593565f8430 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 10 client.27452 fetch_fsmap learned FSMap version 51 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5593566981c0 con 0x5593565f8430 2024-08-20T01:25:00.419 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9848031070 con 0x5593565f8430 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) ==== 8362+0+0 (secure 0 0 0) 0x7f9848042460 con 0x5593565f8430 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 10 client.27452 fetch_fsmap finished waiting for FSMap version 51 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 10 client.27452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 20 client.27452 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356698880 0x55935669aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 4 client.27452 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55935656ac00 con 0x559356698880 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985de00640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356698880 0x55935669aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f985de00640 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356698880 0x55935669aca0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbfe889d97802c4b server_cookie=3a1506498c679e in_seq=0 out_seq=0 2024-08-20T01:25:00.420 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.413+0000 7f983b400640 10 client.27452 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f983b400640 1 -- 172.21.15.138:0/4271536688 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55935656ac00 con 0x559356698880 2024-08-20T01:25:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f983b400640 10 client.27452 handle_command_reply: tid=0 2024-08-20T01:25:00.681 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 client.27452 shutdown 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 msgr2=0x55935660cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f98480058a0 tx=0x7f9848003e20 comp rx=0 tx=0).stop 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 shutdown_connections 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x559356698880 0x55935669aca0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5593566043c0 0x559356607c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5593565f8430 0x55935660cb90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 --2- 172.21.15.138:0/4271536688 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5593565f7b10 0x55935660c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f983b400640 0 client.27452 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 >> 172.21.15.138:0/4271536688 conn(0x5593565f5940 msgr2=0x5593565fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 shutdown_connections 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 1 -- 172.21.15.138:0/4271536688 wait complete. 2024-08-20T01:25:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.678+0000 7f9863843740 20 client.27452 trim_cache size 0 max 16384 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 msgr2=0x7f98400b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.684 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 0x7f98400b6280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9848009730 tx=0x7f98480096c0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 msgr2=0x7f98581a0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f985400eb10 tx=0x7f985400c2d0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 shutdown_connections 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f98400b3dc0 0x7f98400b6280 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9858105970 0x7f98581a0820 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9858104fd0 0x7f985819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 --2- 172.21.15.138:0/482436948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f985806a7a0 0x7f985819af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:00.685 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 >> 172.21.15.138:0/482436948 conn(0x7f9858077270 msgr2=0x7f98581081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 shutdown_connections 2024-08-20T01:25:00.686 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:00.683+0000 7f985f200640 1 -- 172.21.15.138:0/482436948 wait complete. 2024-08-20T01:25:01.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 16.19 MB/sec execute 58 sec latency 3068.277 ms 2024-08-20T01:25:02.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: pgmap v890: 129 pgs: 129 active+clean; 970 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-08-20T01:25:02.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:25:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:01 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:25:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: pgmap v890: 129 pgs: 129 active+clean; 970 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-08-20T01:25:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:25:02.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:25:02.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:01 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:25:02.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: pgmap v890: 129 pgs: 129 active+clean; 970 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 13 MiB/s wr, 753 op/s 2024-08-20T01:25:02.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:25:02.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi167 to 176.0M 2024-08-20T01:25:02.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:01 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi167 to 184592486: error parsing value: Value '184592486' is below minimum 939524096 2024-08-20T01:25:02.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 15.91 MB/sec execute 59 sec latency 4068.344 ms 2024-08-20T01:25:03.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:03.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:25:03.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:02 smithi167 ceph-mon[21965]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:25:03.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:03.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:25:03.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:02 smithi142 ceph-mon[21799]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-08-20T01:25:03.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: Adjusting osd_memory_target on smithi142 to 176.0M 2024-08-20T01:25:03.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:02 smithi138 ceph-mon[18076]: Unable to set osd_memory_target on smithi142 to 184575283: error parsing value: Value '184575283' is below minimum 939524096 2024-08-20T01:25:03.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 15.65 MB/sec execute 60 sec latency 5068.408 ms 2024-08-20T01:25:04.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:03 smithi167 ceph-mon[21965]: pgmap v891: 129 pgs: 129 active+clean; 960 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 562 op/s 2024-08-20T01:25:04.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:03 smithi142 ceph-mon[21799]: pgmap v891: 129 pgs: 129 active+clean; 960 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 562 op/s 2024-08-20T01:25:04.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:03 smithi138 ceph-mon[18076]: pgmap v891: 129 pgs: 129 active+clean; 960 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 12 MiB/s wr, 562 op/s 2024-08-20T01:25:04.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 15.39 MB/sec execute 61 sec latency 6068.473 ms 2024-08-20T01:25:05.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 15.14 MB/sec execute 62 sec latency 7068.537 ms 2024-08-20T01:25:06.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:05 smithi167 ceph-mon[21965]: pgmap v892: 129 pgs: 129 active+clean; 945 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 564 op/s 2024-08-20T01:25:06.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:05 smithi142 ceph-mon[21799]: pgmap v892: 129 pgs: 129 active+clean; 945 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 564 op/s 2024-08-20T01:25:06.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:05 smithi138 ceph-mon[18076]: pgmap v892: 129 pgs: 129 active+clean; 945 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 564 op/s 2024-08-20T01:25:06.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 14.90 MB/sec execute 63 sec latency 8068.634 ms 2024-08-20T01:25:07.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 14.67 MB/sec execute 64 sec latency 9068.677 ms 2024-08-20T01:25:08.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:07 smithi167 ceph-mon[21965]: pgmap v893: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 539 op/s 2024-08-20T01:25:08.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:07 smithi142 ceph-mon[21799]: pgmap v893: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 539 op/s 2024-08-20T01:25:08.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:07 smithi138 ceph-mon[18076]: pgmap v893: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 9.4 MiB/s wr, 539 op/s 2024-08-20T01:25:08.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 14.44 MB/sec execute 65 sec latency 10068.722 ms 2024-08-20T01:25:09.313 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 14.23 MB/sec execute 66 sec latency 11068.764 ms 2024-08-20T01:25:10.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:09 smithi167 ceph-mon[21965]: pgmap v894: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 472 op/s 2024-08-20T01:25:10.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:09 smithi142 ceph-mon[21799]: pgmap v894: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 472 op/s 2024-08-20T01:25:10.295 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:09 smithi138 ceph-mon[18076]: pgmap v894: 129 pgs: 129 active+clean; 939 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.7 MiB/s wr, 472 op/s 2024-08-20T01:25:10.314 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 14.01 MB/sec execute 67 sec latency 12069.784 ms 2024-08-20T01:25:11.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:10 smithi142 ceph-mon[21799]: osdmap e110: 12 total, 12 up, 12 in 2024-08-20T01:25:11.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:10 smithi138 ceph-mon[18076]: osdmap e110: 12 total, 12 up, 12 in 2024-08-20T01:25:11.314 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 13.81 MB/sec execute 68 sec latency 13069.848 ms 2024-08-20T01:25:11.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:10 smithi167 ceph-mon[21965]: osdmap e110: 12 total, 12 up, 12 in 2024-08-20T01:25:12.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:11 smithi142 ceph-mon[21799]: pgmap v896: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 934 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-08-20T01:25:12.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:11 smithi138 ceph-mon[18076]: pgmap v896: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 934 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-08-20T01:25:12.314 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59676 13.61 MB/sec execute 69 sec latency 14069.899 ms 2024-08-20T01:25:12.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:11 smithi167 ceph-mon[21965]: pgmap v896: 129 pgs: 8 active+clean+snaptrim, 22 active+clean+snaptrim_wait, 99 active+clean; 934 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.5 MiB/s wr, 246 op/s 2024-08-20T01:25:13.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 59735 13.44 MB/sec execute 70 sec latency 14924.891 ms 2024-08-20T01:25:13.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.835+0000 7ffa26000640 1 -- 172.21.15.138:0/285521114 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140318a0 con 0x7ffa28075b30 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 -- 172.21.15.138:0/285521114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa28075b30 msgr2=0x7ffa2810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 --2- 172.21.15.138:0/285521114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa28075b30 0x7ffa2810ea90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009900 tx=0x7ffa1402a410 comp rx=0 tx=0).stop 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 -- 172.21.15.138:0/285521114 shutdown_connections 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 --2- 172.21.15.138:0/285521114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa2810efd0 0x7ffa281113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 --2- 172.21.15.138:0/285521114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa28075b30 0x7ffa2810ea90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 --2- 172.21.15.138:0/285521114 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 0x7ffa280755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 -- 172.21.15.138:0/285521114 >> 172.21.15.138:0/285521114 conn(0x7ffa280fe430 msgr2=0x7ffa28100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 -- 172.21.15.138:0/285521114 shutdown_connections 2024-08-20T01:25:13.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.836+0000 7ffa2e200640 1 -- 172.21.15.138:0/285521114 wait complete. 2024-08-20T01:25:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 Processor -- start 2024-08-20T01:25:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 -- start start 2024-08-20T01:25:13.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 0x7ffa2819b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 0x7ffa281a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28114bd0 con 0x7ffa2810efd0 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28114a50 con 0x7ffa28075b30 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.837+0000 7ffa2e200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa28114d50 con 0x7ffa28075210 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 0x7ffa281a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36874/0 (socket says 172.21.15.138:36874) 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa26a00640 1 -- 172.21.15.138:0/3396995916 learned_addr learned my addr 172.21.15.138:0/3396995916 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27400640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 0x7ffa2819b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 -- 172.21.15.138:0/3396995916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 msgr2=0x7ffa2819b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 0x7ffa2819b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 -- 172.21.15.138:0/3396995916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 msgr2=0x7ffa2819b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 -- 172.21.15.138:0/3396995916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa281a11c0 con 0x7ffa2810efd0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa26a00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.838+0000 7ffa27e00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 0x7ffa281a09f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000ec50 tx=0x7ffa2000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.839+0000 7ffa1fe00640 1 -- 172.21.15.138:0/3396995916 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20003b70 con 0x7ffa2810efd0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.839+0000 7ffa1fe00640 1 -- 172.21.15.138:0/3396995916 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa20003d10 con 0x7ffa2810efd0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.839+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa281a14b0 con 0x7ffa2810efd0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.839+0000 7ffa1fe00640 1 -- 172.21.15.138:0/3396995916 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20010830 con 0x7ffa2810efd0 2024-08-20T01:25:13.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.839+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281a7b70 con 0x7ffa2810efd0 2024-08-20T01:25:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 Processor -- start 2024-08-20T01:25:13.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa1fe00640 1 -- 172.21.15.138:0/3396995916 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ffa20003ec0 con 0x7ffa2810efd0 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 -- start start 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c7025759b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 0x55c702584270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c7024eb210 con 0x55c7025755d0 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c7024da710 con 0x55c702575ef0 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.842+0000 7ffa3108e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c7024de750 con 0x55c702581e80 2024-08-20T01:25:13.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36876/0 (socket says 172.21.15.138:36876) 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 -- 172.21.15.138:0/902803958 learned_addr learned my addr 172.21.15.138:0/902803958 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa26a00640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c7025759b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa26a00640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 0x55c702584270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 -- 172.21.15.138:0/902803958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 msgr2=0x55c702584270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 0x55c702584270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 -- 172.21.15.138:0/902803958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 msgr2=0x55c7025759b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c7025759b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 -- 172.21.15.138:0/902803958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7024a6de0 con 0x55c702575ef0 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa26a00640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 0x55c702584270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:25:13.845 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.843+0000 7ffa27400640 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009cf0 tx=0x7ffa18011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62199b30f5d25a63 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa13e00640 1 -- 172.21.15.138:0/902803958 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1801e070 con 0x55c702575ef0 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa1fe00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 0x7ffa0c0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa13e00640 1 -- 172.21.15.138:0/902803958 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa18019ae0 con 0x55c702575ef0 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa13e00640 1 -- 172.21.15.138:0/902803958 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800e970 con 0x55c702575ef0 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa1fe00640 1 -- 172.21.15.138:0/3396995916 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9541+0+0 (secure 0 0 0) 0x7ffa200e4970 con 0x7ffa2810efd0 2024-08-20T01:25:13.849 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa26a00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 0x7ffa0c0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 -- 172.21.15.138:0/902803958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 msgr2=0x55c702581940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009cf0 tx=0x7ffa18011810 comp rx=0 tx=0).stop 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 -- 172.21.15.138:0/902803958 shutdown_connections 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702581e80 0x55c702584270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702581940 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 --2- 172.21.15.138:0/902803958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c7025759b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.844+0000 7ffa3108e740 1 -- 172.21.15.138:0/902803958 >> 172.21.15.138:0/902803958 conn(0x55c7025733b0 msgr2=0x55c7025745a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.845+0000 7ffa3108e740 1 -- 172.21.15.138:0/902803958 shutdown_connections 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.845+0000 7ffa3108e740 1 -- 172.21.15.138:0/902803958 wait complete. 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.845+0000 7ffa26a00640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 0x7ffa0c0b6280 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa14031440 tx=0x7ffa14005e50 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.845+0000 7ffa3108e740 1 -- start start 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.845+0000 7ffa3108e740 1 Processor -- start 2024-08-20T01:25:13.850 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c70257f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55c702413ab0 con 0x55c702581e80 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55c702481140 con 0x55c7025755d0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55c702584b60 con 0x55c702575ef0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa3108e740 10 client.0 _ll_register_callbacks cb 0x55c7024945d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36884/0 (socket says 172.21.15.138:36884) 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 -- 172.21.15.138:0/4262898366 learned_addr learned my addr 172.21.15.138:0/4262898366 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa26a00640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27e00640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c70257f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 -- 172.21.15.138:0/4262898366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 msgr2=0x55c70257f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.851 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 -- 172.21.15.138:0/4262898366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 msgr2=0x55c70257f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c70257f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.846+0000 7ffa27400640 1 -- 172.21.15.138:0/4262898366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c702482170 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa27400640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097b0 tx=0x7ffa18005540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed788a7f53d42ea server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa10c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1801e040 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c7024b1a30 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9541+0+0 (secure 0 0 0) 0x7ffa18004230 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffa18030050 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa3108e740 10 client.27464 fetch_fsmap learned FSMap version 52 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c70223d790 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.847+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c7024b1a30 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7ffa180047e0 con 0x55c7025755d0 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 10 client.27464 fetch_fsmap finished waiting for FSMap version 52 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 10 client.27464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:13.852 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 20 client.27464 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c702591c40 0x55c702594030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 4 client.27464 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7024ed160 con 0x55c702591c40 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa26a00640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c702591c40 0x55c702594030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa26a00640 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c702591c40 0x55c702594030 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5033327961d5a7e server_cookie=af2d939a75b77b0a in_seq=0 out_seq=0 2024-08-20T01:25:13.853 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.848+0000 7ffa10c00640 10 client.27464 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.849+0000 7ffa10c00640 1 -- 172.21.15.138:0/4262898366 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7024ed160 con 0x55c702591c40 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.849+0000 7ffa10c00640 10 client.27464 handle_command_reply: tid=0 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.849+0000 7ffa3108e740 1 client.27464 shutdown 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 msgr2=0x55c70257edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ffa180097b0 tx=0x7ffa18005540 comp rx=0 tx=0).stop 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 shutdown_connections 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c702591c40 0x55c702594030 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c70257f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 --2- 172.21.15.138:0/4262898366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c7025755d0 0x55c70257edc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 >> 172.21.15.138:0/4262898366 conn(0x55c7025733b0 msgr2=0x55c702578fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa10c00640 0 client.27464 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 shutdown_connections 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 1 -- 172.21.15.138:0/4262898366 wait complete. 2024-08-20T01:25:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.850+0000 7ffa3108e740 20 client.27464 trim_cache size 0 max 16384 2024-08-20T01:25:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 Processor -- start 2024-08-20T01:25:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 -- start start 2024-08-20T01:25:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702599770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c702599cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b4730 con 0x55c702581e80 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b2000 con 0x55c702575ef0 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.866+0000 7ffa3108e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281afe70 con 0x55c7025755d0 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702599770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:36304/0 (socket says 172.21.15.138:36304) 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 -- 172.21.15.138:0/3932389765 learned_addr learned my addr 172.21.15.138:0/3932389765 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27e00640 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c702599cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 -- 172.21.15.138:0/3932389765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 msgr2=0x55c702599770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702599770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 -- 172.21.15.138:0/3932389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 msgr2=0x55c702599cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c702599cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 -- 172.21.15.138:0/3932389765 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7022b6620 con 0x55c7025755d0 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa27400640 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa18011850 tx=0x7ffa18009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b47deff09e33fef server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.867+0000 7ffa13400640 1 -- 172.21.15.138:0/3932389765 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1801e070 con 0x55c7025755d0 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa13400640 1 -- 172.21.15.138:0/3932389765 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1800e3b0 con 0x55c7025755d0 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa13400640 1 -- 172.21.15.138:0/3932389765 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18003830 con 0x55c7025755d0 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 -- 172.21.15.138:0/3932389765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 msgr2=0x55c7025723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ffa18011850 tx=0x7ffa18009400 comp rx=0 tx=0).stop 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 -- 172.21.15.138:0/3932389765 shutdown_connections 2024-08-20T01:25:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c702581e80 0x55c702599cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702575ef0 0x55c702599770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 --2- 172.21.15.138:0/3932389765 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c7025755d0 0x55c7025723c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 -- 172.21.15.138:0/3932389765 >> 172.21.15.138:0/3932389765 conn(0x55c7025733b0 msgr2=0x55c702571cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 -- 172.21.15.138:0/3932389765 shutdown_connections 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.868+0000 7ffa3108e740 1 -- 172.21.15.138:0/3932389765 wait complete. 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 -- start start 2024-08-20T01:25:13.871 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 Processor -- start 2024-08-20T01:25:13.873 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c70257c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702581e80 0x55c70257b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b04c0 con 0x55c7025755d0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b1930 con 0x55c702581e80 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281b0b90 con 0x55c702575ef0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.869+0000 7ffa3108e740 10 client.0 _ll_register_callbacks cb 0x7ffa1801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:36308/0 (socket says 172.21.15.138:36308) 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 -- 172.21.15.138:0/193356725 learned_addr learned my addr 172.21.15.138:0/193356725 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa27e00640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702581e80 0x55c70257b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa27400640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c70257c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 -- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702581e80 msgr2=0x55c70257b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.874 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702581e80 0x55c70257b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 -- 172.21.15.138:0/193356725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 msgr2=0x55c70257c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c70257c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 -- 172.21.15.138:0/193356725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c70257bd10 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa26a00640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa14006010 tx=0x7ffa1402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=724d8356b49ef7fe server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa10c00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14043070 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1800e3b0 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.870+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9541+0+0 (secure 0 0 0) 0x7ffa140037f0 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ffa14064050 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 10 client.27433 fetch_fsmap learned FSMap version 52 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x55c702589f20 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1800e3b0 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 52) ==== 8358+0+0 (secure 0 0 0) 0x7ffa140499c0 con 0x55c702575ef0 2024-08-20T01:25:13.875 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 10 client.27433 fetch_fsmap finished waiting for FSMap version 52 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 10 client.27433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 20 client.27433 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70258a680 0x55c7025fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 4 client.27433 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.871+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7024e8c00 con 0x55c70258a680 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.872+0000 7ffa27400640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70258a680 0x55c7025fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.872+0000 7ffa27400640 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70258a680 0x55c7025fe750 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2811e032bb200c0 server_cookie=977badc513ae70df in_seq=0 out_seq=0 2024-08-20T01:25:13.876 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.872+0000 7ffa10c00640 10 client.27433 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa10c00640 1 -- 172.21.15.138:0/193356725 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7024e8c00 con 0x55c70258a680 2024-08-20T01:25:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa10c00640 10 client.27433 handle_command_reply: tid=0 2024-08-20T01:25:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 client.27433 shutdown 2024-08-20T01:25:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 msgr2=0x55c70257afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ffa14006010 tx=0x7ffa1402a410 comp rx=0 tx=0).stop 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 shutdown_connections 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55c70258a680 0x55c7025fe750 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c702581e80 0x55c70257b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55c702575ef0 0x55c70257afa0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 --2- 172.21.15.138:0/193356725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55c7025755d0 0x55c70257c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.976+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 >> 172.21.15.138:0/193356725 conn(0x55c7025733b0 msgr2=0x55c702580c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.977+0000 7ffa10c00640 0 client.27433 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.977+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 shutdown_connections 2024-08-20T01:25:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.977+0000 7ffa3108e740 1 -- 172.21.15.138:0/193356725 wait complete. 2024-08-20T01:25:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.977+0000 7ffa3108e740 20 client.27433 trim_cache size 0 max 16384 2024-08-20T01:25:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:25:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:25:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:25:13.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 msgr2=0x7ffa0c0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 0x7ffa0c0b6280 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffa14031440 tx=0x7ffa14005e50 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 msgr2=0x7ffa281a09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 0x7ffa281a09f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000ec50 tx=0x7ffa2000c6a0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 shutdown_connections 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ffa0c0b3dc0 0x7ffa0c0b6280 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa2810efd0 0x7ffa281a09f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffa28075b30 0x7ffa2819b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 --2- 172.21.15.138:0/3396995916 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ffa28075210 0x7ffa2819b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 >> 172.21.15.138:0/3396995916 conn(0x7ffa280fe430 msgr2=0x7ffa280ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 shutdown_connections 2024-08-20T01:25:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:13.978+0000 7ffa2e200640 1 -- 172.21.15.138:0/3396995916 wait complete. 2024-08-20T01:25:14.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:13 smithi138 ceph-mon[18076]: pgmap v897: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 930 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 352 KiB/s wr, 149 op/s 2024-08-20T01:25:14.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:13 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:25:14.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:13 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:25:14.233 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:13 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:25:14.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:13 smithi142 ceph-mon[21799]: pgmap v897: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 930 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 352 KiB/s wr, 149 op/s 2024-08-20T01:25:14.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:13 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:25:14.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:13 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:25:14.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:13 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:25:14.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 60385 13.54 MB/sec execute 71 sec latency 181.319 ms 2024-08-20T01:25:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:13 smithi167 ceph-mon[21965]: pgmap v897: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 930 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 352 KiB/s wr, 149 op/s 2024-08-20T01:25:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:13 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:25:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:13 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:25:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:13 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:25:15.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 60707 13.56 MB/sec execute 72 sec latency 580.093 ms 2024-08-20T01:25:16.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:15 smithi142 ceph-mon[21799]: pgmap v898: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 928 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-08-20T01:25:16.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:15 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:16.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:15 smithi138 ceph-mon[18076]: pgmap v898: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 928 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-08-20T01:25:16.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:15 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:16.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 60991 13.70 MB/sec execute 73 sec latency 1580.167 ms 2024-08-20T01:25:16.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:15 smithi167 ceph-mon[21965]: pgmap v898: 129 pgs: 10 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 93 active+clean; 928 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 177 KiB/s wr, 53 op/s 2024-08-20T01:25:16.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:15 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:17.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 61527 13.97 MB/sec execute 74 sec latency 1716.622 ms 2024-08-20T01:25:18.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:17 smithi142 ceph-mon[21799]: pgmap v899: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-20T01:25:18.243 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:17 smithi138 ceph-mon[18076]: pgmap v899: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-20T01:25:18.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 62451 14.48 MB/sec execute 75 sec latency 1730.449 ms 2024-08-20T01:25:18.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:17 smithi167 ceph-mon[21965]: pgmap v899: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-08-20T01:25:19.315 INFO:tasks.workunit.client.0.smithi138.stdout: 10 63284 14.95 MB/sec execute 76 sec latency 2730.496 ms 2024-08-20T01:25:20.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:19 smithi142 ceph-mon[21799]: pgmap v900: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-08-20T01:25:20.265 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:19 smithi167 ceph-mon[21965]: pgmap v900: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-08-20T01:25:20.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:19 smithi138 ceph-mon[18076]: pgmap v900: 129 pgs: 129 active+clean; 925 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-08-20T01:25:20.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 64026 15.26 MB/sec execute 77 sec latency 3730.636 ms 2024-08-20T01:25:21.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:21 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:21.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:21 smithi138 ceph-mon[18076]: osdmap e111: 12 total, 12 up, 12 in 2024-08-20T01:25:21.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 64717 15.48 MB/sec execute 78 sec latency 4730.697 ms 2024-08-20T01:25:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:21 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:21.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:21 smithi167 ceph-mon[21965]: osdmap e111: 12 total, 12 up, 12 in 2024-08-20T01:25:21.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:21 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:25:21.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:21 smithi142 ceph-mon[21799]: osdmap e111: 12 total, 12 up, 12 in 2024-08-20T01:25:21.628 INFO:tasks.quiescer.fs.[cephfs]:Total duration (49.9) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-08-20T01:25:21.628 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-08-20T01:25:21.628 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:25:22.282 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:22.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 65112 15.53 MB/sec execute 79 sec latency 5730.764 ms 2024-08-20T01:25:23.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:22 smithi138 ceph-mon[18076]: pgmap v902: 129 pgs: 129 active+clean; 939 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-08-20T01:25:23.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:22 smithi167 ceph-mon[21965]: pgmap v902: 129 pgs: 129 active+clean; 939 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-08-20T01:25:23.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:22 smithi142 ceph-mon[21799]: pgmap v902: 129 pgs: 129 active+clean; 939 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 324 op/s 2024-08-20T01:25:23.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 65925 15.78 MB/sec execute 80 sec latency 6730.819 ms 2024-08-20T01:25:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.793+0000 7fcc44200640 1 -- 172.21.15.138:0/1339630974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1134e0 msgr2=0x7fcc3c1158d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.793+0000 7fcc44200640 1 --2- 172.21.15.138:0/1339630974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1158d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009ca0 tx=0x7fcc3402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 -- 172.21.15.138:0/1339630974 shutdown_connections 2024-08-20T01:25:23.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 --2- 172.21.15.138:0/1339630974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1158d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 --2- 172.21.15.138:0/1339630974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c112fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 --2- 172.21.15.138:0/1339630974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c100ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 -- 172.21.15.138:0/1339630974 >> 172.21.15.138:0/1339630974 conn(0x7fcc3c0fc730 msgr2=0x7fcc3c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 -- 172.21.15.138:0/1339630974 shutdown_connections 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.794+0000 7fcc44200640 1 -- 172.21.15.138:0/1339630974 wait complete. 2024-08-20T01:25:23.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.796+0000 7fcc44200640 1 Processor -- start 2024-08-20T01:25:23.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.796+0000 7fcc44200640 1 -- start start 2024-08-20T01:25:23.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.796+0000 7fcc44200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:23.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.796+0000 7fcc44200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc44200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc41800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc41800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53778/0 (socket says 172.21.15.138:53778) 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc44200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c1190f0 con 0x7fcc3c1010e0 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc44200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c118f70 con 0x7fcc3c1007c0 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc44200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c119270 con 0x7fcc3c1134e0 2024-08-20T01:25:23.797 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc41800640 1 -- 172.21.15.138:0/2952464375 learned_addr learned my addr 172.21.15.138:0/2952464375 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc40e00640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.797+0000 7fcc42200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc41800640 1 -- 172.21.15.138:0/2952464375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1134e0 msgr2=0x7fcc3c1a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc41800640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc41800640 1 -- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 msgr2=0x7fcc3c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc41800640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc41800640 1 -- 172.21.15.138:0/2952464375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c1a56b0 con 0x7fcc3c1007c0 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.798+0000 7fcc40e00640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:23.799 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc41800640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000b4a0 tx=0x7fcc3000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30014070 con 0x7fcc3c1007c0 2024-08-20T01:25:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c1a59a0 con 0x7fcc3c1007c0 2024-08-20T01:25:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c1ac230 con 0x7fcc3c1007c0 2024-08-20T01:25:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc30010ac0 con 0x7fcc3c1007c0 2024-08-20T01:25:23.800 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.799+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc30017c30 con 0x7fcc3c1007c0 2024-08-20T01:25:23.801 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.801+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc1c005290 con 0x7fcc3c1007c0 2024-08-20T01:25:23.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.804+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fcc30017070 con 0x7fcc3c1007c0 2024-08-20T01:25:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.806+0000 7fcc3a000640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 0x7fcc240b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.807+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fcc300e36a0 con 0x7fcc3c1007c0 2024-08-20T01:25:23.807 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.807+0000 7fcc40e00640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 0x7fcc240b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:23.808 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.808+0000 7fcc40e00640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 0x7fcc240b6280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c1074b0 tx=0x7fcc28005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:23.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:23.812+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fcc300aa0d0 con 0x7fcc3c1007c0 2024-08-20T01:25:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.195+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc1c005850 con 0x7fcc3c1007c0 2024-08-20T01:25:24.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.196+0000 7fcc3a000640 1 -- 172.21.15.138:0/2952464375 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7fcc300ae620 con 0x7fcc3c1007c0 2024-08-20T01:25:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:25:24.196 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":54,"btime":"2024-08-20T01:25:19:989006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:25:19.256053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:25:24.196 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 54 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.198+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 msgr2=0x7fcc240b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.198+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 0x7fcc240b6280 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c1074b0 tx=0x7fcc28005f90 comp rx=0 tx=0).stop 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 msgr2=0x7fcc3c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000b4a0 tx=0x7fcc3000b960 comp rx=0 tx=0).stop 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 shutdown_connections 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fcc240b3dc0 0x7fcc240b6280 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fcc3c1134e0 0x7fcc3c1a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:24.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc3c1010e0 0x7fcc3c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:24.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 --2- 172.21.15.138:0/2952464375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcc3c1007c0 0x7fcc3c19f600 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:24.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 >> 172.21.15.138:0/2952464375 conn(0x7fcc3c0fc730 msgr2=0x7fcc3c102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:24.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 shutdown_connections 2024-08-20T01:25:24.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:24.199+0000 7fcc44200640 1 -- 172.21.15.138:0/2952464375 wait complete. 2024-08-20T01:25:24.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 66812 16.43 MB/sec execute 81 sec latency 7730.862 ms 2024-08-20T01:25:25.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:24 smithi138 ceph-mon[18076]: pgmap v903: 129 pgs: 129 active+clean; 957 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-08-20T01:25:25.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:24 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2952464375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:25.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:24 smithi167 ceph-mon[21965]: pgmap v903: 129 pgs: 129 active+clean; 957 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-08-20T01:25:25.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:24 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2952464375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:25.228 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-08-20T01:25:25.229 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-08-20T01:25:25.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:24 smithi142 ceph-mon[21799]: pgmap v903: 129 pgs: 129 active+clean; 957 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 435 op/s 2024-08-20T01:25:25.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:24 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2952464375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:25.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 67328 16.46 MB/sec execute 82 sec latency 8730.912 ms 2024-08-20T01:25:25.857 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:26.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 67956 16.62 MB/sec execute 83 sec latency 9730.973 ms 2024-08-20T01:25:27.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:26 smithi138 ceph-mon[18076]: pgmap v904: 129 pgs: 129 active+clean; 963 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.121+0000 7f851da00640 1 -- 172.21.15.138:0/1013701501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 msgr2=0x7f85181111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.121+0000 7f851da00640 1 --2- 172.21.15.138:0/1013701501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181111b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8510009ca0 tx=0x7f851002a8c0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 -- 172.21.15.138:0/1013701501 shutdown_connections 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 --2- 172.21.15.138:0/1013701501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181111b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 --2- 172.21.15.138:0/1013701501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075c20 0x7f851810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 --2- 172.21.15.138:0/1013701501 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075270 0x7f8518075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 -- 172.21.15.138:0/1013701501 >> 172.21.15.138:0/1013701501 conn(0x7f85180fe280 msgr2=0x7f85181006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 -- 172.21.15.138:0/1013701501 shutdown_connections 2024-08-20T01:25:27.122 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.122+0000 7f851da00640 1 -- 172.21.15.138:0/1013701501 wait complete. 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 Processor -- start 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 -- start start 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075270 0x7f851819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8518114a10 con 0x7f8518075270 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8518114890 con 0x7f851810ed80 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851da00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8518114b90 con 0x7f8518075c20 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f8517e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075270 0x7f851819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f8517400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f8517400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:48380/0 (socket says 172.21.15.138:48380) 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f8517400640 1 -- 172.21.15.138:0/3782756277 learned_addr learned my addr 172.21.15.138:0/3782756277 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.123+0000 7f851d000640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 -- 172.21.15.138:0/3782756277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 msgr2=0x7f85181a07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181a07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 -- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075270 msgr2=0x7f851819aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075270 0x7f851819aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 -- 172.21.15.138:0/3782756277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85181a0f90 con 0x7f8518075c20 2024-08-20T01:25:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8517400640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f850400e9f0 tx=0x7f850400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8514c00640 1 -- 172.21.15.138:0/3782756277 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850400cd80 con 0x7f8518075c20 2024-08-20T01:25:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8514c00640 1 -- 172.21.15.138:0/3782756277 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85040108e0 con 0x7f8518075c20 2024-08-20T01:25:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85181a1280 con 0x7f8518075c20 2024-08-20T01:25:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f8514c00640 1 -- 172.21.15.138:0/3782756277 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504018980 con 0x7f8518075c20 2024-08-20T01:25:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.124+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85181a7940 con 0x7f8518075c20 2024-08-20T01:25:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.127+0000 7f852207d740 1 Processor -- start 2024-08-20T01:25:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.127+0000 7f852207d740 1 -- start start 2024-08-20T01:25:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.127+0000 7f852207d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 0x561760e88bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f852207d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e94b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f852207d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e97470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f852207d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561760dfe210 con 0x561760e887d0 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f852207d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561760ded710 con 0x561760e890f0 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f852207d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561760df1750 con 0x561760e95080 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f851d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 0x561760e88bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e94b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f851d000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 0x561760e88bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:52256/0 (socket says 172.21.15.138:52256) 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f851d000640 1 -- 172.21.15.138:0/519169969 learned_addr learned my addr 172.21.15.138:0/519169969 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8514c00640 1 -- 172.21.15.138:0/3782756277 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8504010430 con 0x7f8518075c20 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 -- 172.21.15.138:0/519169969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 msgr2=0x561760e97470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e97470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 -- 172.21.15.138:0/519169969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 msgr2=0x561760e88bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 0x561760e88bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.129 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.128+0000 7f8517e00640 1 -- 172.21.15.138:0/519169969 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561760db9de0 con 0x561760e890f0 2024-08-20T01:25:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f8517e00640 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e94b40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8508009c20 tx=0x7f8508011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=620abd1e55e2b17b server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f850d600640 1 -- 172.21.15.138:0/519169969 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850801e070 con 0x561760e890f0 2024-08-20T01:25:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f850d600640 1 -- 172.21.15.138:0/519169969 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8508009dd0 con 0x561760e890f0 2024-08-20T01:25:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f850d600640 1 -- 172.21.15.138:0/519169969 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8508018710 con 0x561760e890f0 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f852207d740 1 -- 172.21.15.138:0/519169969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 msgr2=0x561760e94b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.129+0000 7f852207d740 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e94b40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8508009c20 tx=0x7f8508011810 comp rx=0 tx=0).stop 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 -- 172.21.15.138:0/519169969 shutdown_connections 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e97470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e94b40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 --2- 172.21.15.138:0/519169969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e887d0 0x561760e88bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 -- 172.21.15.138:0/519169969 >> 172.21.15.138:0/519169969 conn(0x561760e86600 msgr2=0x561760e877a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f8514c00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 0x7f84fc0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 -- 172.21.15.138:0/519169969 shutdown_connections 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 -- 172.21.15.138:0/519169969 wait complete. 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f8514c00640 1 -- 172.21.15.138:0/3782756277 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f85040e46b0 con 0x7f8518075c20 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f8517e00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 0x7f84fc0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.130+0000 7f852207d740 1 -- start start 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 Processor -- start 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f8517e00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 0x7f84fc0b6200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8508009f50 tx=0x7f8508009770 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e92050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760e92ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x561760d26ab0 con 0x561760e95080 2024-08-20T01:25:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x561760d94140 con 0x561760e887d0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f852207d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x561760e97d60 con 0x561760e890f0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.131+0000 7f8517e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e92050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f852207d740 10 client.0 _ll_register_callbacks cb 0x561760da75d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:48386/0 (socket says 172.21.15.138:48386) 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 -- 172.21.15.138:0/3433234292 learned_addr learned my addr 172.21.15.138:0/3433234292 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517e00640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e92050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53814/0 (socket says 172.21.15.138:53814) 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 -- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 msgr2=0x561760e92050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e92050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 -- 172.21.15.138:0/3433234292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 msgr2=0x561760e92ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f851d000640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760e92ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760e92ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 -- 172.21.15.138:0/3433234292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561760d95170 con 0x561760e890f0 2024-08-20T01:25:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f851d000640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760e92ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:25:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8517400640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85040e1ac0 tx=0x7f8504010720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce8f1539ed3c3522 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8502000640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.132+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040af520 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561760dc4a30 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f85040e42b0 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f85040b3fd0 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 10 client.27448 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561760b50790 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x561760dc4a30 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f85040ae020 con 0x561760e890f0 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 10 client.27448 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 10 client.27448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 20 client.27448 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760ea5040 0x561760ea7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 4 client.27448 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.133+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561760e00160 con 0x561760ea5040 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.134+0000 7f8517e00640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760ea5040 0x561760ea7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.134+0000 7f8517e00640 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760ea5040 0x561760ea7430 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fac07d70fbd9ae0 server_cookie=1d62a6ac79ced328 in_seq=0 out_seq=0 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.134+0000 7f8502000640 10 client.27448 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f8502000640 1 -- 172.21.15.138:0/3433234292 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561760e00160 con 0x561760ea5040 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f8502000640 10 client.27448 handle_command_reply: tid=0 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 client.27448 shutdown 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 msgr2=0x561760e92590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f85040e1ac0 tx=0x7f8504010720 comp rx=0 tx=0).stop 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 shutdown_connections 2024-08-20T01:25:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760ea5040 0x561760ea7430 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760e92ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e890f0 0x561760e92590 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 --2- 172.21.15.138:0/3433234292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e92050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 >> 172.21.15.138:0/3433234292 conn(0x561760e86600 msgr2=0x561760e8c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.135+0000 7f8502000640 0 client.27448 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.136+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 shutdown_connections 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.136+0000 7f852207d740 1 -- 172.21.15.138:0/3433234292 wait complete. 2024-08-20T01:25:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.136+0000 7f852207d740 20 client.27448 trim_cache size 0 max 16384 2024-08-20T01:25:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 Processor -- start 2024-08-20T01:25:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 -- start start 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 0x561760e8c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760eacce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b44f0 con 0x561760e95080 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b1d60 con 0x561760e890f0 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f852207d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85181afbd0 con 0x561760e887d0 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f8517400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f8517400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53820/0 (socket says 172.21.15.138:53820) 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f8517400640 1 -- 172.21.15.138:0/2862282622 learned_addr learned my addr 172.21.15.138:0/2862282622 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f851d000640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760eacce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.151+0000 7f8517e00640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 0x561760e8c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 -- 172.21.15.138:0/2862282622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 msgr2=0x561760e8c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 0x561760e8c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 -- 172.21.15.138:0/2862282622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 msgr2=0x561760eacce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760eacce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 -- 172.21.15.138:0/2862282622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561760bc9620 con 0x561760e890f0 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517e00640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 0x561760e8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f8517400640 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85040a8240 tx=0x7f85040ab150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8949b9b0698aa0a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f850cc00640 1 -- 172.21.15.138:0/2862282622 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040e42b0 con 0x561760e890f0 2024-08-20T01:25:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f850cc00640 1 -- 172.21.15.138:0/2862282622 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504018d70 con 0x561760e890f0 2024-08-20T01:25:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.152+0000 7f850cc00640 1 -- 172.21.15.138:0/2862282622 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85040a6c50 con 0x561760e890f0 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- 172.21.15.138:0/2862282622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 msgr2=0x561760e93d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f85040a8240 tx=0x7f85040ab150 comp rx=0 tx=0).stop 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- 172.21.15.138:0/2862282622 shutdown_connections 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e95080 0x561760eacce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e890f0 0x561760e93d90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.159 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 --2- 172.21.15.138:0/2862282622 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e887d0 0x561760e8c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- 172.21.15.138:0/2862282622 >> 172.21.15.138:0/2862282622 conn(0x561760e86600 msgr2=0x561760e93680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- 172.21.15.138:0/2862282622 shutdown_connections 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- 172.21.15.138:0/2862282622 wait complete. 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.153+0000 7f852207d740 1 -- start start 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 Processor -- start 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e890f0 0x561760e8ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.160 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b0220 con 0x561760e890f0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b1690 con 0x561760e887d0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85181b08f0 con 0x561760e95080 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f852207d740 10 client.0 _ll_register_callbacks cb 0x7f850400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f8517e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f851d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f8517e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:53824/0 (socket says 172.21.15.138:53824) 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f8517400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e890f0 0x561760e8ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f851d000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:48412/0 (socket says 172.21.15.138:48412) 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.154+0000 7f8517e00640 1 -- 172.21.15.138:0/322693226 learned_addr learned my addr 172.21.15.138:0/322693226 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:27.161 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 -- 172.21.15.138:0/322693226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 msgr2=0x561760e8f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 -- 172.21.15.138:0/322693226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e890f0 msgr2=0x561760e8ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e890f0 0x561760e8ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 -- 172.21.15.138:0/322693226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561760f0ecb0 con 0x561760e887d0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f851d000640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8517e00640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8508009b40 tx=0x7f85080183b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5f7b906fd704cd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8502000640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850801e040 con 0x561760e887d0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8504018d70 con 0x561760e887d0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f8508004880 con 0x561760e887d0 2024-08-20T01:25:27.162 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f8508004a80 con 0x561760e887d0 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.155+0000 7f852207d740 10 client.27494 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x561760f0fdb0 con 0x561760e887d0 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504018d70 con 0x561760e887d0 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f8508021030 con 0x561760e887d0 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 10 client.27494 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 10 client.27494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 20 client.27494 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760f103c0 0x561760f127e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 4 client.27494 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561760dfbc00 con 0x561760f103c0 2024-08-20T01:25:27.163 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.156+0000 7f8517400640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760f103c0 0x561760f127e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.157+0000 7f8517400640 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760f103c0 0x561760f127e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124b14d8b55f247f server_cookie=cc7aac58e1aabef6 in_seq=0 out_seq=0 2024-08-20T01:25:27.164 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:27.157+0000 7f8502000640 10 client.27494 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:27.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:26 smithi167 ceph-mon[21965]: pgmap v904: 129 pgs: 129 active+clean; 963 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-08-20T01:25:27.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:26 smithi142 ceph-mon[21799]: pgmap v904: 129 pgs: 129 active+clean; 963 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 5.5 MiB/s wr, 480 op/s 2024-08-20T01:25:27.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 68689 16.86 MB/sec execute 84 sec latency 10731.026 ms 2024-08-20T01:25:28.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 68862 16.80 MB/sec execute 85 sec latency 11731.081 ms 2024-08-20T01:25:29.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:28 smithi138 ceph-mon[18076]: pgmap v905: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 506 op/s 2024-08-20T01:25:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:28 smithi167 ceph-mon[21965]: pgmap v905: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 506 op/s 2024-08-20T01:25:29.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:28 smithi142 ceph-mon[21799]: pgmap v905: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 6.8 MiB/s wr, 506 op/s 2024-08-20T01:25:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.312+0000 7f8502000640 1 -- 172.21.15.138:0/322693226 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561760dfbc00 con 0x561760f103c0 2024-08-20T01:25:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.312+0000 7f8502000640 10 client.27494 handle_command_reply: tid=0 2024-08-20T01:25:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.312+0000 7f852207d740 1 client.27494 shutdown 2024-08-20T01:25:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 msgr2=0x561760e9d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8508009b40 tx=0x7f85080183b0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 shutdown_connections 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x561760f103c0 0x561760f127e0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x561760e95080 0x561760e8f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x561760e890f0 0x561760e8ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 --2- 172.21.15.138:0/322693226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561760e887d0 0x561760e9d750 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8508009b40 tx=0x7f85080183b0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 >> 172.21.15.138:0/322693226 conn(0x561760e86600 msgr2=0x561760e93ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.313+0000 7f8502000640 0 client.27494 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.314+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 shutdown_connections 2024-08-20T01:25:29.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.314+0000 7f852207d740 1 -- 172.21.15.138:0/322693226 wait complete. 2024-08-20T01:25:29.315 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.314+0000 7f852207d740 20 client.27494 trim_cache size 0 max 16384 2024-08-20T01:25:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:25:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:25:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:25:29.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:25:29.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 69886 17.34 MB/sec execute 86 sec latency 12703.262 ms 2024-08-20T01:25:29.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 msgr2=0x7f84fc0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 0x7f84fc0b6200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8508009f50 tx=0x7f8508009770 comp rx=0 tx=0).stop 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 msgr2=0x7f851819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f850400e9f0 tx=0x7f850400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 shutdown_connections 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f84fc0b3d40 0x7f84fc0b6200 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f851810ed80 0x7f85181a07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8518075c20 0x7f851819b430 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 --2- 172.21.15.138:0/3782756277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8518075270 0x7f851819aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.318+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 >> 172.21.15.138:0/3782756277 conn(0x7f85180fe280 msgr2=0x7f8518110fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.320+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 shutdown_connections 2024-08-20T01:25:29.319 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:29.320+0000 7f851da00640 1 -- 172.21.15.138:0/3782756277 wait complete. 2024-08-20T01:25:29.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.939+0000 7f9d1c600640 1 -- 172.21.15.138:0/1119766864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 msgr2=0x7f9d1411e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.939+0000 7f9d1c600640 1 --2- 172.21.15.138:0/1119766864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d1411e1c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c009ca0 tx=0x7f9d0c02a940 comp rx=0 tx=0).stop 2024-08-20T01:25:29.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 -- 172.21.15.138:0/1119766864 shutdown_connections 2024-08-20T01:25:29.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 --2- 172.21.15.138:0/1119766864 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d1411e1c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 --2- 172.21.15.138:0/1119766864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d1410a260 0x7f9d1410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 --2- 172.21.15.138:0/1119766864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d14109e80 0x7f9d14110ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 -- 172.21.15.138:0/1119766864 >> 172.21.15.138:0/1119766864 conn(0x7f9d1406dcc0 msgr2=0x7f9d1406e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:29.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 -- 172.21.15.138:0/1119766864 shutdown_connections 2024-08-20T01:25:29.941 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.940+0000 7f9d1c600640 1 -- 172.21.15.138:0/1119766864 wait complete. 2024-08-20T01:25:29.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.942+0000 7f9d1c600640 1 Processor -- start 2024-08-20T01:25:29.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.943+0000 7f9d1c600640 1 -- start start 2024-08-20T01:25:29.943 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14109e80 0x7f9d141a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 0x7f9d141a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.944 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d14121860 con 0x7f9d14109e80 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141216e0 con 0x7f9d1410a260 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1c600640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141219e0 con 0x7f9d1411bdd0 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d19200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 0x7f9d141a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41432/0 (socket says 172.21.15.138:41432) 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.944+0000 7f9d1a600640 1 -- 172.21.15.138:0/645182762 learned_addr learned my addr 172.21.15.138:0/645182762 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:29.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d19200640 1 -- 172.21.15.138:0/645182762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 msgr2=0x7f9d141ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d19200640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d19200640 1 -- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14109e80 msgr2=0x7f9d141a7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d19200640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14109e80 0x7f9d141a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d19200640 1 -- 172.21.15.138:0/645182762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d141add80 con 0x7f9d1410a260 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.945+0000 7f9d1a600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.946+0000 7f9d19200640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 0x7f9d141a8210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c970 tx=0x7f9d0000ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:29.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.946+0000 7f9d12000640 1 -- 172.21.15.138:0/645182762 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00007bb0 con 0x7f9d1410a260 2024-08-20T01:25:29.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.946+0000 7f9d12000640 1 -- 172.21.15.138:0/645182762 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d00007d50 con 0x7f9d1410a260 2024-08-20T01:25:29.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.946+0000 7f9d12000640 1 -- 172.21.15.138:0/645182762 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d00004e40 con 0x7f9d1410a260 2024-08-20T01:25:29.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.947+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d141ae070 con 0x7f9d1410a260 2024-08-20T01:25:29.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.947+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d141b4730 con 0x7f9d1410a260 2024-08-20T01:25:29.952 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.952+0000 7f9d1f1c9740 1 Processor -- start 2024-08-20T01:25:29.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.953+0000 7f9d1f1c9740 1 -- start start 2024-08-20T01:25:29.954 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b1151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b117ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56151b07e600 con 0x56151b108e40 2024-08-20T01:25:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56151b06db00 con 0x56151b109760 2024-08-20T01:25:29.955 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.955+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56151b071b40 con 0x56151b1156f0 2024-08-20T01:25:29.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.956+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.956+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49790/0 (socket says 172.21.15.138:49790) 2024-08-20T01:25:29.956 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.956+0000 7f9d1a600640 1 -- 172.21.15.138:0/2653371915 learned_addr learned my addr 172.21.15.138:0/2653371915 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:29.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.956+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b1151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.957 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.957+0000 7f9d12000640 1 -- 172.21.15.138:0/645182762 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f9d00004fe0 con 0x7f9d1410a260 2024-08-20T01:25:29.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.957+0000 7f9d1a600640 1 -- 172.21.15.138:0/2653371915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 msgr2=0x56151b117ae0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:25:29.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.957+0000 7f9d1a600640 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b117ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.957+0000 7f9d1a600640 1 -- 172.21.15.138:0/2653371915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 msgr2=0x56151b1151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:29.959 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.959+0000 7f9d12000640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 0x7f9cfc0b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.959+0000 7f9d1a600640 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b1151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.959+0000 7f9d1a600640 1 -- 172.21.15.138:0/2653371915 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151b013ee0 con 0x56151b108e40 2024-08-20T01:25:29.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.959+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b1151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:29.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.959+0000 7f9d12000640 1 -- 172.21.15.138:0/645182762 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f9d000e4290 con 0x7f9d1410a260 2024-08-20T01:25:29.960 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.960+0000 7f9d19200640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 0x7f9cfc0b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.961+0000 7f9d19200640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 0x7f9cfc0b6390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9d000a7df0 tx=0x7f9d000e4590 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.961+0000 7f9d1a600640 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c031d80 tx=0x7f9d0c031db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200aedbddad15571 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.961+0000 7f9d06a00640 1 -- 172.21.15.138:0/2653371915 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c034040 con 0x56151b108e40 2024-08-20T01:25:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.962+0000 7f9d06a00640 1 -- 172.21.15.138:0/2653371915 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0c003e20 con 0x56151b108e40 2024-08-20T01:25:29.961 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.962+0000 7f9d06a00640 1 -- 172.21.15.138:0/2653371915 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c0308a0 con 0x56151b108e40 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.963+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2653371915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 msgr2=0x56151b109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.963+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c031d80 tx=0x7f9d0c031db0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.963+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2653371915 shutdown_connections 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.964+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b117ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.964+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b1151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.963 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.964+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2653371915 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b109220 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.964 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.964+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2653371915 >> 172.21.15.138:0/2653371915 conn(0x56151b106c70 msgr2=0x56151b107e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:29.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.965+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2653371915 shutdown_connections 2024-08-20T01:25:29.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.965+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2653371915 wait complete. 2024-08-20T01:25:29.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.965+0000 7f9d1f1c9740 1 -- start start 2024-08-20T01:25:29.965 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.965+0000 7f9d1f1c9740 1 Processor -- start 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b109760 0x56151b112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b1156f0 0x56151b113110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x56151aed07b0 con 0x56151b109760 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x56151b008dd0 con 0x56151b1156f0 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.966+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56151b118310 con 0x56151b108e40 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1f1c9740 10 client.0 _ll_register_callbacks cb 0x56151ad7a0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:29.967 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41448/0 (socket says 172.21.15.138:41448) 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 -- 172.21.15.138:0/3060526638 learned_addr learned my addr 172.21.15.138:0/3060526638 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b1156f0 0x56151b113110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 -- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b1156f0 msgr2=0x56151b113110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b1156f0 0x56151b113110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 -- 172.21.15.138:0/3060526638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b109760 msgr2=0x56151b112bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:29.968 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.967+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b109760 0x56151b112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.968+0000 7f9d1a600640 1 -- 172.21.15.138:0/3060526638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151addcba0 con 0x56151b108e40 2024-08-20T01:25:29.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.968+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098d0 tx=0x7f9d0c02fd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e519eaa47c93be30 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:29.969 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.969+0000 7f9cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:25:29.970 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.971+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c049040 con 0x56151b108e40 2024-08-20T01:25:29.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.971+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0c003e20 con 0x56151b108e40 2024-08-20T01:25:29.971 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.971+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f9d0c030bc0 con 0x56151b108e40 2024-08-20T01:25:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.972+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9d0c030dc0 con 0x56151b108e40 2024-08-20T01:25:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.972+0000 7f9d1f1c9740 10 client.27463 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:29.972 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.972+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56151b123f50 con 0x56151b108e40 2024-08-20T01:25:29.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0c003e20 con 0x56151b108e40 2024-08-20T01:25:29.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f9d0c04f040 con 0x56151b108e40 2024-08-20T01:25:29.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 10 client.27463 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:29.973 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 10 client.27463 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:29.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 20 client.27463 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:29.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1252a0 0x56151b127690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:29.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 4 client.27463 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:29.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.973+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56151b080550 con 0x56151b1252a0 2024-08-20T01:25:29.974 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.974+0000 7f9d19c00640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1252a0 0x56151b127690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:29.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.975+0000 7f9cfb400640 10 client.27463 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:29.975 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.975+0000 7f9d19c00640 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1252a0 0x56151b127690 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=303d0a89ae0a1d06 server_cookie=917eb072e2ec3b07 in_seq=0 out_seq=0 2024-08-20T01:25:29.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.978+0000 7f9cfb400640 1 -- 172.21.15.138:0/3060526638 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56151b080550 con 0x56151b1252a0 2024-08-20T01:25:29.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.978+0000 7f9cfb400640 10 client.27463 handle_command_reply: tid=0 2024-08-20T01:25:29.978 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.978+0000 7f9d1f1c9740 1 client.27463 shutdown 2024-08-20T01:25:29.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 msgr2=0x56151b112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:29.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9d0c0098d0 tx=0x7f9d0c02fd40 comp rx=0 tx=0).stop 2024-08-20T01:25:29.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 shutdown_connections 2024-08-20T01:25:29.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1252a0 0x56151b127690 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b1156f0 0x56151b113110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b109760 0x56151b112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3060526638 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b112690 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 >> 172.21.15.138:0/3060526638 conn(0x56151b106c70 msgr2=0x56151b10c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.979+0000 7f9cfb400640 0 client.27463 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:29.981 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.982+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 shutdown_connections 2024-08-20T01:25:29.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.982+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3060526638 wait complete. 2024-08-20T01:25:29.982 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:29.982+0000 7f9d1f1c9740 20 client.27463 trim_cache size 0 max 16384 2024-08-20T01:25:30.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 Processor -- start 2024-08-20T01:25:30.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 -- start start 2024-08-20T01:25:30.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.008 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b12ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b1156f0 0x56151b12d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141c1210 con 0x56151b1156f0 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141bea80 con 0x56151b109760 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.008+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141bc8f0 con 0x56151b108e40 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.009+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.009+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:41458/0 (socket says 172.21.15.138:41458) 2024-08-20T01:25:30.009 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.009+0000 7f9d1a600640 1 -- 172.21.15.138:0/2417327547 learned_addr learned my addr 172.21.15.138:0/2417327547 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:30.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b12ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 -- 172.21.15.138:0/2417327547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 msgr2=0x56151b10ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:30.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 -- 172.21.15.138:0/2417327547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b1156f0 msgr2=0x56151b12d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:30.010 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b1156f0 0x56151b12d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 -- 172.21.15.138:0/2417327547 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56151b106aa0 con 0x56151b109760 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d1a600640 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.010+0000 7f9d19c00640 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b12ccc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56151b127c00 tx=0x7f9d08010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=179327f89dfe112a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.011+0000 7f9d06000640 1 -- 172.21.15.138:0/2417327547 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0801b030 con 0x56151b109760 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.011+0000 7f9d06000640 1 -- 172.21.15.138:0/2417327547 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0800ad10 con 0x56151b109760 2024-08-20T01:25:30.011 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.011+0000 7f9d06000640 1 -- 172.21.15.138:0/2417327547 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08003d20 con 0x56151b109760 2024-08-20T01:25:30.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.012+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2417327547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 msgr2=0x56151b12ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:30.012 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.012+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b12ccc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x56151b127c00 tx=0x7f9d08010800 comp rx=0 tx=0).stop 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2417327547 shutdown_connections 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b1156f0 0x56151b12d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b12ccc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/2417327547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b108e40 0x56151b10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2417327547 >> 172.21.15.138:0/2417327547 conn(0x56151b106c70 msgr2=0x56151b1144e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2417327547 shutdown_connections 2024-08-20T01:25:30.013 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.013+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/2417327547 wait complete. 2024-08-20T01:25:30.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.014+0000 7f9d1f1c9740 1 -- start start 2024-08-20T01:25:30.014 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.014+0000 7f9d1f1c9740 1 Processor -- start 2024-08-20T01:25:30.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.015+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b119510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.015 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b119f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141bcf40 con 0x56151b108e40 2024-08-20T01:25:30.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141be3b0 con 0x56151b109760 2024-08-20T01:25:30.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f9d141bd610 con 0x56151b1156f0 2024-08-20T01:25:30.016 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1f1c9740 10 client.0 _ll_register_callbacks cb 0x7f9d0c02ff00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.016+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b119510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d19c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d19c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56208/0 (socket says 172.21.15.138:56208) 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d1a600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b119510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49812/0 (socket says 172.21.15.138:49812) 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d1a600640 1 -- 172.21.15.138:0/3725987988 learned_addr learned my addr 172.21.15.138:0/3725987988 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:30.017 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b119f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d19c00640 1 -- 172.21.15.138:0/3725987988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 msgr2=0x56151b119f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d19c00640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b119f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.017+0000 7f9d19c00640 1 -- 172.21.15.138:0/3725987988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 msgr2=0x56151b119510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.018+0000 7f9d19c00640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b119510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.018+0000 7f9d19c00640 1 -- 172.21.15.138:0/3725987988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56151b1076f0 con 0x56151b109760 2024-08-20T01:25:30.018 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.018+0000 7f9d19c00640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56151b123f20 tx=0x7f9d08003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c00b876969aa630 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:30.019 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.018+0000 7f9cfb400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:30.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.021+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0801b040 con 0x56151b109760 2024-08-20T01:25:30.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.021+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f9d08004220 con 0x56151b109760 2024-08-20T01:25:30.021 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.021+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d0800ad10 con 0x56151b109760 2024-08-20T01:25:30.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.021+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f9d0802e050 con 0x56151b109760 2024-08-20T01:25:30.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.022+0000 7f9d1f1c9740 10 client.27509 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:30.022 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.022+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x56151b18fe00 con 0x56151b109760 2024-08-20T01:25:30.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d0800ad10 con 0x56151b109760 2024-08-20T01:25:30.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f9d08005420 con 0x56151b109760 2024-08-20T01:25:30.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 10 client.27509 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:30.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 10 client.27509 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:30.023 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 20 client.27509 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1905a0 0x56151b1929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 4 client.27509 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.023+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x56151b07bff0 con 0x56151b1905a0 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.024+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1905a0 0x56151b1929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.024+0000 7f9cfb400640 10 client.27509 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:30.024 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.024+0000 7f9d1a600640 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1905a0 0x56151b1929c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3730a25757226fb4 server_cookie=f5987e5e06c26f3a in_seq=0 out_seq=0 2024-08-20T01:25:30.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70324 17.35 MB/sec execute 87 sec latency 644.423 ms 2024-08-20T01:25:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9cfb400640 1 -- 172.21.15.138:0/3725987988 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56151b07bff0 con 0x56151b1905a0 2024-08-20T01:25:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9cfb400640 10 client.27509 handle_command_reply: tid=0 2024-08-20T01:25:30.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 client.27509 shutdown 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 msgr2=0x56151b119a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x56151b123f20 tx=0x7f9d08003820 comp rx=0 tx=0).stop 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 shutdown_connections 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56151b1905a0 0x56151b1929c0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56151b1156f0 0x56151b119f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151b109760 0x56151b119a50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9cfb400640 0 client.27509 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 --2- 172.21.15.138:0/3725987988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56151b108e40 0x56151b119510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 >> 172.21.15.138:0/3725987988 conn(0x56151b106c70 msgr2=0x56151b112580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 shutdown_connections 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 1 -- 172.21.15.138:0/3725987988 wait complete. 2024-08-20T01:25:30.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.623+0000 7f9d1f1c9740 20 client.27509 trim_cache size 0 max 16384 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 54, 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 62, 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "a27ddc43": { 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 62, 2024-08-20T01:25:30.625 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 144.9, 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:30.626 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:25:30.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 msgr2=0x7f9cfc0b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 0x7f9cfc0b6390 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9d000a7df0 tx=0x7f9d000e4590 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 msgr2=0x7f9d141a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 0x7f9d141a8210 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d0000c970 tx=0x7f9d0000ce30 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 shutdown_connections 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f9cfc0b3ed0 0x7f9cfc0b6390 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9d1411bdd0 0x7f9d141ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9d1410a260 0x7f9d141a8210 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 --2- 172.21.15.138:0/645182762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d14109e80 0x7f9d141a7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 >> 172.21.15.138:0/645182762 conn(0x7f9d1406dcc0 msgr2=0x7f9d1411c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 shutdown_connections 2024-08-20T01:25:30.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:30.624+0000 7f9d1c600640 1 -- 172.21.15.138:0/645182762 wait complete. 2024-08-20T01:25:31.022 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:30 smithi138 ceph-mon[18076]: pgmap v906: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 515 op/s 2024-08-20T01:25:31.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:30 smithi167 ceph-mon[21965]: pgmap v906: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 515 op/s 2024-08-20T01:25:31.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:30 smithi142 ceph-mon[21799]: pgmap v906: 129 pgs: 129 active+clean; 983 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 6.8 MiB/s wr, 515 op/s 2024-08-20T01:25:31.316 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 17.16 MB/sec execute 88 sec latency 1644.504 ms 2024-08-20T01:25:31.421 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a27ddc43', quiesce took 9.8 seconds. Will release after: 15.149999999999999 2024-08-20T01:25:32.334 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.96 MB/sec execute 89 sec latency 2662.581 ms 2024-08-20T01:25:33.066 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:32 smithi138 ceph-mon[18076]: pgmap v907: 129 pgs: 129 active+clean; 998 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2024-08-20T01:25:33.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:32 smithi167 ceph-mon[21965]: pgmap v907: 129 pgs: 129 active+clean; 998 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2024-08-20T01:25:33.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:32 smithi142 ceph-mon[21799]: pgmap v907: 129 pgs: 129 active+clean; 998 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.8 MiB/s wr, 652 op/s 2024-08-20T01:25:33.335 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.77 MB/sec execute 90 sec latency 3662.638 ms 2024-08-20T01:25:34.334 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.59 MB/sec execute 91 sec latency 4662.707 ms 2024-08-20T01:25:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:34 smithi167 ceph-mon[21965]: pgmap v908: 129 pgs: 129 active+clean; 1002 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-08-20T01:25:35.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:34 smithi142 ceph-mon[21799]: pgmap v908: 129 pgs: 129 active+clean; 1002 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-08-20T01:25:35.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:34 smithi138 ceph-mon[18076]: pgmap v908: 129 pgs: 129 active+clean; 1002 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 7.2 MiB/s wr, 611 op/s 2024-08-20T01:25:35.334 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.41 MB/sec execute 92 sec latency 5662.749 ms 2024-08-20T01:25:36.335 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.23 MB/sec execute 93 sec latency 6662.850 ms 2024-08-20T01:25:37.098 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:36 smithi138 ceph-mon[18076]: pgmap v909: 129 pgs: 129 active+clean; 1004 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-08-20T01:25:37.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:36 smithi167 ceph-mon[21965]: pgmap v909: 129 pgs: 129 active+clean; 1004 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-08-20T01:25:37.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:36 smithi142 ceph-mon[21799]: pgmap v909: 129 pgs: 129 active+clean; 1004 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.0 MiB/s wr, 533 op/s 2024-08-20T01:25:37.339 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 16.06 MB/sec execute 94 sec latency 7667.721 ms 2024-08-20T01:25:38.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.89 MB/sec execute 95 sec latency 8674.423 ms 2024-08-20T01:25:38.846 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:38 smithi138 ceph-mon[18076]: pgmap v910: 129 pgs: 129 active+clean; 1013 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 551 op/s 2024-08-20T01:25:39.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:38 smithi167 ceph-mon[21965]: pgmap v910: 129 pgs: 129 active+clean; 1013 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 551 op/s 2024-08-20T01:25:39.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:38 smithi142 ceph-mon[21799]: pgmap v910: 129 pgs: 129 active+clean; 1013 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 7.3 MiB/s wr, 551 op/s 2024-08-20T01:25:39.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.72 MB/sec execute 96 sec latency 9674.481 ms 2024-08-20T01:25:40.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.56 MB/sec execute 97 sec latency 10674.577 ms 2024-08-20T01:25:41.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:40 smithi167 ceph-mon[21965]: pgmap v911: 129 pgs: 129 active+clean; 1017 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-08-20T01:25:41.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:40 smithi142 ceph-mon[21799]: pgmap v911: 129 pgs: 129 active+clean; 1017 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-08-20T01:25:41.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:40 smithi138 ceph-mon[18076]: pgmap v911: 129 pgs: 129 active+clean; 1017 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.9 MiB/s wr, 433 op/s 2024-08-20T01:25:41.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.40 MB/sec execute 98 sec latency 11674.630 ms 2024-08-20T01:25:42.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.25 MB/sec execute 99 sec latency 12674.676 ms 2024-08-20T01:25:42.445 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.444+0000 7fa47ec00640 1 -- 172.21.15.138:0/1050845939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa47806a7a0 msgr2=0x7fa47806ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.444+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1050845939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa47806a7a0 0x7fa47806ab80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009860 tx=0x7fa46c02a340 comp rx=0 tx=0).stop 2024-08-20T01:25:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 -- 172.21.15.138:0/1050845939 shutdown_connections 2024-08-20T01:25:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1050845939 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa478105970 0x7fa47810f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1050845939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478104fd0 0x7fa478105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.446 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1050845939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa47806a7a0 0x7fa47806ab80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 -- 172.21.15.138:0/1050845939 >> 172.21.15.138:0/1050845939 conn(0x7fa478077270 msgr2=0x7fa478077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 -- 172.21.15.138:0/1050845939 shutdown_connections 2024-08-20T01:25:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.445+0000 7fa47ec00640 1 -- 172.21.15.138:0/1050845939 wait complete. 2024-08-20T01:25:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 Processor -- start 2024-08-20T01:25:42.448 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 -- start start 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47806a7a0 0x7fa47819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478104fd0 0x7fa47819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112800 con 0x7fa478105970 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112680 con 0x7fa478104fd0 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa478112980 con 0x7fa47806a7a0 2024-08-20T01:25:42.449 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47e200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33962/0 (socket says 172.21.15.138:33962) 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47e200640 1 -- 172.21.15.138:0/1630764747 learned_addr learned my addr 172.21.15.138:0/1630764747 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47ce00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478104fd0 0x7fa47819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.446+0000 7fa47d800640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47806a7a0 0x7fa47819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 -- 172.21.15.138:0/1630764747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47806a7a0 msgr2=0x7fa47819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47806a7a0 0x7fa47819af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 -- 172.21.15.138:0/1630764747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478104fd0 msgr2=0x7fa47819b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478104fd0 0x7fa47819b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.455 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 -- 172.21.15.138:0/1630764747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4781a1000 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47e200640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa47000eaa0 tx=0x7fa47000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa476000640 1 -- 172.21.15.138:0/1630764747 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47000cdb0 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa476000640 1 -- 172.21.15.138:0/1630764747 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa470005020 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4781a12f0 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa476000640 1 -- 172.21.15.138:0/1630764747 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700116d0 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.447+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4781a79b0 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.450+0000 7fa483046740 1 Processor -- start 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa476000640 1 -- 172.21.15.138:0/1630764747 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa4700051d0 con 0x7fa478105970 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 -- start start 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.456 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 0x55811c876e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c8773c0 0x55811c8797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.457 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55811c7e0210 con 0x55811c86ab10 2024-08-20T01:25:42.465 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55811c7cf710 con 0x55811c86b430 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.452+0000 7fa483046740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55811c7d3750 con 0x55811c8773c0 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 0x55811c876e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33968/0 (socket says 172.21.15.138:33968) 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47ce00640 1 -- 172.21.15.138:0/2711218391 learned_addr learned my addr 172.21.15.138:0/2711218391 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 -- 172.21.15.138:0/2711218391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c8773c0 msgr2=0x55811c8797b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c8773c0 0x55811c8797b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 -- 172.21.15.138:0/2711218391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 msgr2=0x55811c86aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47d800640 1 -- 172.21.15.138:0/2711218391 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811c79bde0 con 0x55811c86b430 2024-08-20T01:25:42.466 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa47ce00640 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.453+0000 7fa476000640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 0x7fa4600b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.454+0000 7fa476000640 1 -- 172.21.15.138:0/1630764747 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fa4700e4560 con 0x7fa478105970 2024-08-20T01:25:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.454+0000 7fa47ce00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 0x7fa4600b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.467 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.454+0000 7fa47ce00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 0x7fa4600b6200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa464012450 tx=0x7fa464009260 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.476+0000 7fa47d800640 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 0x55811c876e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa46c035040 tx=0x7fa46c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bc546b75a37abdd server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa46aa00640 1 -- 172.21.15.138:0/2711218391 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c041020 con 0x55811c86b430 2024-08-20T01:25:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa46aa00640 1 -- 172.21.15.138:0/2711218391 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c030e50 con 0x55811c86b430 2024-08-20T01:25:42.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa46aa00640 1 -- 172.21.15.138:0/2711218391 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c037040 con 0x55811c86b430 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 -- 172.21.15.138:0/2711218391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 msgr2=0x55811c876e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 0x55811c876e80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa46c035040 tx=0x7fa46c005d30 comp rx=0 tx=0).stop 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 -- 172.21.15.138:0/2711218391 shutdown_connections 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c8773c0 0x55811c8797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c86b430 0x55811c876e80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 --2- 172.21.15.138:0/2711218391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c86aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.477+0000 7fa483046740 1 -- 172.21.15.138:0/2711218391 >> 172.21.15.138:0/2711218391 conn(0x55811c868940 msgr2=0x55811c869ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.478+0000 7fa483046740 1 -- 172.21.15.138:0/2711218391 shutdown_connections 2024-08-20T01:25:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.478+0000 7fa483046740 1 -- 172.21.15.138:0/2711218391 wait complete. 2024-08-20T01:25:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.478+0000 7fa483046740 1 -- start start 2024-08-20T01:25:42.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.478+0000 7fa483046740 1 Processor -- start 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86b430 0x55811c874010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c874550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55811c708ab0 con 0x55811c86b430 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55811c776140 con 0x55811c8773c0 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55811c87a040 con 0x55811c86ab10 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa483046740 10 client.0 _ll_register_callbacks cb 0x55811c7895d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47ce00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86b430 0x55811c874010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c874550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47d800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45806/0 (socket says 172.21.15.138:45806) 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47d800640 1 -- 172.21.15.138:0/2088604685 learned_addr learned my addr 172.21.15.138:0/2088604685 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 -- 172.21.15.138:0/2088604685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 msgr2=0x55811c876bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 -- 172.21.15.138:0/2088604685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86b430 msgr2=0x55811c874010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86b430 0x55811c874010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47e200640 1 -- 172.21.15.138:0/2088604685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811c777170 con 0x55811c8773c0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.479+0000 7fa47d800640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa47e200640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c874550 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4700e1910 tx=0x7fa4700aaa30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3316de8f78ab4f0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa45f400640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47001f040 con 0x55811c8773c0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c030e50 con 0x55811c8773c0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fa4700e4180 con 0x55811c8773c0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa4700e4380 con 0x55811c8773c0 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa483046740 10 client.27521 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:42.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55811c532790 con 0x55811c8773c0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.480+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c030e50 con 0x55811c8773c0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa47000f040 con 0x55811c8773c0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 10 client.27521 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 10 client.27521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 20 client.27521 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c886fd0 0x55811c8893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 4 client.27521 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55811c7e2160 con 0x55811c886fd0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.481+0000 7fa47d800640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c886fd0 0x55811c8893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.482+0000 7fa47d800640 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c886fd0 0x55811c8893e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59851e7cff3da77 server_cookie=5e146ccfa59a5f1 in_seq=0 out_seq=0 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.482+0000 7fa45f400640 10 client.27521 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:42.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.482+0000 7fa45f400640 1 -- 172.21.15.138:0/2088604685 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55811c7e2160 con 0x55811c886fd0 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.482+0000 7fa45f400640 10 client.27521 handle_command_reply: tid=0 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 client.27521 shutdown 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 msgr2=0x55811c874550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c874550 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa4700e1910 tx=0x7fa4700aaa30 comp rx=0 tx=0).stop 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 shutdown_connections 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c886fd0 0x55811c8893e0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c874550 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86b430 0x55811c874010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa45f400640 0 client.27521 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 --2- 172.21.15.138:0/2088604685 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86ab10 0x55811c876bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 >> 172.21.15.138:0/2088604685 conn(0x55811c868940 msgr2=0x55811c86e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 shutdown_connections 2024-08-20T01:25:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 1 -- 172.21.15.138:0/2088604685 wait complete. 2024-08-20T01:25:42.495 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.483+0000 7fa483046740 20 client.27521 trim_cache size 0 max 16384 2024-08-20T01:25:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 Processor -- start 2024-08-20T01:25:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 -- start start 2024-08-20T01:25:42.499 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c869ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c88ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781b44c0 con 0x55811c86ab10 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781b1d30 con 0x55811c8773c0 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa483046740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781afba0 con 0x55811c86b430 2024-08-20T01:25:42.500 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa47d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa47d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33988/0 (socket says 172.21.15.138:33988) 2024-08-20T01:25:42.501 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa47d800640 1 -- 172.21.15.138:0/3749273525 learned_addr learned my addr 172.21.15.138:0/3749273525 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa47e200640 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c88ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.498+0000 7fa47ce00640 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c869ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 -- 172.21.15.138:0/3749273525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 msgr2=0x55811c869ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c869ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 -- 172.21.15.138:0/3749273525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 msgr2=0x55811c88ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c88ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 -- 172.21.15.138:0/3749273525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55811c5ab620 con 0x55811c86ab10 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa47d800640 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa46c02a860 tx=0x7fa46c048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6689bb129e8c3c45 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa46a000640 1 -- 172.21.15.138:0/3749273525 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c041040 con 0x55811c86ab10 2024-08-20T01:25:42.502 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa46a000640 1 -- 172.21.15.138:0/3749273525 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c0054f0 con 0x55811c86ab10 2024-08-20T01:25:42.503 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.499+0000 7fa46a000640 1 -- 172.21.15.138:0/3749273525 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c037040 con 0x55811c86ab10 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- 172.21.15.138:0/3749273525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 msgr2=0x55811c869990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa46c02a860 tx=0x7fa46c048040 comp rx=0 tx=0).stop 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- 172.21.15.138:0/3749273525 shutdown_connections 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c88ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c869ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 --2- 172.21.15.138:0/3749273525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c869990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- 172.21.15.138:0/3749273525 >> 172.21.15.138:0/3749273525 conn(0x55811c868940 msgr2=0x55811c873fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- 172.21.15.138:0/3749273525 shutdown_connections 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- 172.21.15.138:0/3749273525 wait complete. 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 -- start start 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.500+0000 7fa483046740 1 Processor -- start 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c87a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c87ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781b01f0 con 0x55811c86ab10 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781b1660 con 0x55811c8773c0 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa4781b08c0 con 0x55811c86b430 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa483046740 10 client.0 _ll_register_callbacks cb 0x7fa4700a6c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c87a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47e200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:38624/0 (socket says 172.21.15.138:38624) 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47ce00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c87ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47e200640 1 -- 172.21.15.138:0/2885621713 learned_addr learned my addr 172.21.15.138:0/2885621713 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47d800640 1 -- 172.21.15.138:0/2885621713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 msgr2=0x55811c87ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47d800640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c87ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47d800640 1 -- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 msgr2=0x55811c8efcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.501+0000 7fa47d800640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa47d800640 1 -- 172.21.15.138:0/2885621713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55811c8f01f0 con 0x55811c86ab10 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa47e200640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa47d800640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c87a6d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa46c030850 tx=0x7fa46c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa5092767649b5f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa45f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:25:42.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c037040 con 0x55811c86ab10 2024-08-20T01:25:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fa46c0037f0 con 0x55811c86ab10 2024-08-20T01:25:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c0054f0 con 0x55811c86ab10 2024-08-20T01:25:42.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa46c041040 con 0x55811c86ab10 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa483046740 10 client.17487 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.502+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55811c8f1410 con 0x55811c86ab10 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c0054f0 con 0x55811c86ab10 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa46c03d630 con 0x55811c86ab10 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 10 client.17487 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 10 client.17487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 20 client.17487 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c8f1790 0x55811c8f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 4 client.17487 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55811c7ddc00 con 0x55811c8f1790 2024-08-20T01:25:42.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.503+0000 7fa47ce00640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c8f1790 0x55811c8f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.504+0000 7fa47ce00640 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c8f1790 0x55811c8f3bb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d69789b9934b28ba server_cookie=88defb4b2b0e5c9e in_seq=0 out_seq=0 2024-08-20T01:25:42.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.504+0000 7fa45f400640 10 client.17487 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.580+0000 7fa45f400640 1 -- 172.21.15.138:0/2885621713 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55811c7ddc00 con 0x55811c8f1790 2024-08-20T01:25:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.580+0000 7fa45f400640 10 client.17487 handle_command_reply: tid=0 2024-08-20T01:25:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.580+0000 7fa483046740 1 client.17487 shutdown 2024-08-20T01:25:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 msgr2=0x55811c87a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c87a6d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa46c030850 tx=0x7fa46c000bc0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 shutdown_connections 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55811c8f1790 0x55811c8f3bb0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55811c8773c0 0x55811c8efcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55811c86b430 0x55811c87ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 --2- 172.21.15.138:0/2885621713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55811c86ab10 0x55811c87a6d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 >> 172.21.15.138:0/2885621713 conn(0x55811c868940 msgr2=0x55811c869990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa45f400640 0 client.17487 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 shutdown_connections 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 1 -- 172.21.15.138:0/2885621713 wait complete. 2024-08-20T01:25:42.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.581+0000 7fa483046740 20 client.17487 trim_cache size 0 max 16384 2024-08-20T01:25:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:25:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:25:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:25:42.584 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 msgr2=0x7fa4600b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 0x7fa4600b6200 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa464012450 tx=0x7fa464009260 comp rx=0 tx=0).stop 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 msgr2=0x7fa4781a0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa47000eaa0 tx=0x7fa47000ef60 comp rx=0 tx=0).stop 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 shutdown_connections 2024-08-20T01:25:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa4600b3d40 0x7fa4600b6200 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa478105970 0x7fa4781a0830 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa478104fd0 0x7fa47819b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 --2- 172.21.15.138:0/1630764747 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa47806a7a0 0x7fa47819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 >> 172.21.15.138:0/1630764747 conn(0x7fa478077270 msgr2=0x7fa4781081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.582+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 shutdown_connections 2024-08-20T01:25:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:42.583+0000 7fa47ec00640 1 -- 172.21.15.138:0/1630764747 wait complete. 2024-08-20T01:25:43.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:42 smithi167 ceph-mon[21965]: pgmap v912: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 560 op/s 2024-08-20T01:25:43.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:42 smithi142 ceph-mon[21799]: pgmap v912: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 560 op/s 2024-08-20T01:25:43.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:42 smithi138 ceph-mon[18076]: pgmap v912: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 10 MiB/s wr, 560 op/s 2024-08-20T01:25:43.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 15.10 MB/sec execute 100 sec latency 13674.727 ms 2024-08-20T01:25:44.346 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.95 MB/sec execute 101 sec latency 14674.778 ms 2024-08-20T01:25:45.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:44 smithi167 ceph-mon[21965]: pgmap v913: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 364 op/s 2024-08-20T01:25:45.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:44 smithi142 ceph-mon[21799]: pgmap v913: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 364 op/s 2024-08-20T01:25:45.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:44 smithi138 ceph-mon[18076]: pgmap v913: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.4 MiB/s wr, 364 op/s 2024-08-20T01:25:45.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.80 MB/sec execute 102 sec latency 15674.830 ms 2024-08-20T01:25:46.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.66 MB/sec execute 103 sec latency 16674.897 ms 2024-08-20T01:25:46.572 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:25:46.981 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:47.116 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:46 smithi138 ceph-mon[18076]: pgmap v914: 129 pgs: 129 active+clean; 1020 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 310 op/s 2024-08-20T01:25:47.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:46 smithi167 ceph-mon[21965]: pgmap v914: 129 pgs: 129 active+clean; 1020 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 310 op/s 2024-08-20T01:25:47.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:46 smithi142 ceph-mon[21799]: pgmap v914: 129 pgs: 129 active+clean; 1020 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.7 MiB/s wr, 310 op/s 2024-08-20T01:25:47.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.52 MB/sec execute 104 sec latency 17674.963 ms 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- 172.21.15.138:0/65318260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0077d20 msgr2=0x7f47a0113150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 --2- 172.21.15.138:0/65318260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0077d20 0x7f47a0113150 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f478c009860 tx=0x7f478c02a340 comp rx=0 tx=0).stop 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- 172.21.15.138:0/65318260 shutdown_connections 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 --2- 172.21.15.138:0/65318260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0113690 0x7f47a0115a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 --2- 172.21.15.138:0/65318260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0077d20 0x7f47a0113150 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:47.686 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 --2- 172.21.15.138:0/65318260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077400 0x7f47a00777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- 172.21.15.138:0/65318260 >> 172.21.15.138:0/65318260 conn(0x7f47a0100730 msgr2=0x7f47a0102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- 172.21.15.138:0/65318260 shutdown_connections 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- 172.21.15.138:0/65318260 wait complete. 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 Processor -- start 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.649+0000 7f47a8000640 1 -- start start 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 0x7f47a019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 0x7f47a019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0113690 0x7f47a01a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0119070 con 0x7f47a0077400 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0118ef0 con 0x7f47a0113690 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f47a01191f0 con 0x7f47a0077d20 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 0x7f47a019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 0x7f47a019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:47.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a4c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 0x7f47a019fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45842/0 (socket says 172.21.15.138:45842) 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a4c00640 1 -- 172.21.15.138:0/2297055380 learned_addr learned my addr 172.21.15.138:0/2297055380 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a6000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0113690 0x7f47a01a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 -- 172.21.15.138:0/2297055380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 msgr2=0x7f47a019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 0x7f47a019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 -- 172.21.15.138:0/2297055380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0113690 msgr2=0x7f47a01a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0113690 0x7f47a01a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 -- 172.21.15.138:0/2297055380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a01a5670 con 0x7f47a0077400 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a5600640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 0x7f47a019f5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f479400e9f0 tx=0x7f479400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479400ccc0 con 0x7f47a0077400 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47940108e0 con 0x7f47a0077400 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794018880 con 0x7f47a0077400 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a01a5960 con 0x7f47a0077400 2024-08-20T01:25:47.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a01ac020 con 0x7f47a0077400 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.655+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00777e0 con 0x7f47a0077400 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.659+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4794010430 con 0x7f47a0077400 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.664+0000 7f479e000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 0x7f47880b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.664+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f47940e4bf0 con 0x7f47a0077400 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.664+0000 7f47a4c00640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 0x7f47880b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.664+0000 7f47a4c00640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 0x7f47880b6200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f47a0079b00 tx=0x7f478c030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:47.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:47.664+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f47940ab4b0 con 0x7f47a0077400 2024-08-20T01:25:48.029 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.028+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47a01a06f0 con 0x7f47a0077400 2024-08-20T01:25:48.030 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.030+0000 7f479e000640 1 -- 172.21.15.138:0/2297055380 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7f47940afa00 con 0x7f47a0077400 2024-08-20T01:25:48.031 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:25:48.031 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":54,"btime":"2024-08-20T01:25:19:989006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:25:19.256053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:25:48.031 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 54 2024-08-20T01:25:48.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 msgr2=0x7f47880b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:48.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 0x7f47880b6200 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f47a0079b00 tx=0x7f478c030340 comp rx=0 tx=0).stop 2024-08-20T01:25:48.033 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 msgr2=0x7f47a019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 0x7f47a019f5c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f479400e9f0 tx=0x7f479400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 shutdown_connections 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f47880b3d40 0x7f47880b6200 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47a0113690 0x7f47a01a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f47a0077d20 0x7f47a019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:48.034 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 --2- 172.21.15.138:0/2297055380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a0077400 0x7f47a019f5c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:48.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 >> 172.21.15.138:0/2297055380 conn(0x7f47a0100730 msgr2=0x7f47a0102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:48.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 shutdown_connections 2024-08-20T01:25:48.035 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:48.033+0000 7f47a8000640 1 -- 172.21.15.138:0/2297055380 wait complete. 2024-08-20T01:25:48.176 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:47 smithi138 ceph-mon[18076]: pgmap v915: 129 pgs: 129 active+clean; 1012 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 322 op/s 2024-08-20T01:25:48.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:47 smithi167 ceph-mon[21965]: pgmap v915: 129 pgs: 129 active+clean; 1012 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 322 op/s 2024-08-20T01:25:48.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:47 smithi142 ceph-mon[21799]: pgmap v915: 129 pgs: 129 active+clean; 1012 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 8.4 MiB/s wr, 322 op/s 2024-08-20T01:25:48.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.38 MB/sec execute 105 sec latency 18675.024 ms 2024-08-20T01:25:48.965 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:48 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2297055380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:49.161 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id a27ddc43 --release --await' 2024-08-20T01:25:49.162 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id a27ddc43 --release --await 2024-08-20T01:25:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:48 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2297055380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:49.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:48 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2297055380' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:49.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.24 MB/sec execute 106 sec latency 19675.075 ms 2024-08-20T01:25:49.542 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:50.069 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:49 smithi138 ceph-mon[18076]: pgmap v916: 129 pgs: 129 active+clean; 1011 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 6.5 MiB/s wr, 254 op/s 2024-08-20T01:25:50.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:49 smithi167 ceph-mon[21965]: pgmap v916: 129 pgs: 129 active+clean; 1011 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 6.5 MiB/s wr, 254 op/s 2024-08-20T01:25:50.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:49 smithi142 ceph-mon[21799]: pgmap v916: 129 pgs: 129 active+clean; 1011 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 734 KiB/s rd, 6.5 MiB/s wr, 254 op/s 2024-08-20T01:25:50.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70330 14.11 MB/sec execute 107 sec latency 20675.201 ms 2024-08-20T01:25:50.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 -- 172.21.15.138:0/3228795537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac108e40 msgr2=0x7f82ac10f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.512 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 --2- 172.21.15.138:0/3228795537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac108e40 0x7f82ac10f640 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009ca0 tx=0x7f82a402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 -- 172.21.15.138:0/3228795537 shutdown_connections 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 --2- 172.21.15.138:0/3228795537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac108e40 0x7f82ac10f640 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 --2- 172.21.15.138:0/3228795537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac1084c0 0x7f82ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 --2- 172.21.15.138:0/3228795537 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac102ab0 0x7f82ac102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 -- 172.21.15.138:0/3228795537 >> 172.21.15.138:0/3228795537 conn(0x7f82ac0fe500 msgr2=0x7f82ac100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 -- 172.21.15.138:0/3228795537 shutdown_connections 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.451+0000 7f82b0e00640 1 -- 172.21.15.138:0/3228795537 wait complete. 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.452+0000 7f82b0e00640 1 Processor -- start 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.452+0000 7f82b0e00640 1 -- start start 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 0x7f82ac199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 0x7f82ac19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.513 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112b00 con 0x7f82ac102ab0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112980 con 0x7f82ac108e40 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82b0e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac112c80 con 0x7f82ac1084c0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40900/0 (socket says 172.21.15.138:40900) 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82a9e00640 1 -- 172.21.15.138:0/1509016349 learned_addr learned my addr 172.21.15.138:0/1509016349 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82a9400640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 0x7f82ac199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.453+0000 7f82aa800640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 0x7f82ac19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 -- 172.21.15.138:0/1509016349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 msgr2=0x7f82ac199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 0x7f82ac199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 -- 172.21.15.138:0/1509016349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 msgr2=0x7f82ac19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 0x7f82ac19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 -- 172.21.15.138:0/1509016349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82ac19efe0 con 0x7f82ac102ab0 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82aa800640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 0x7f82ac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:50.514 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9400640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 0x7f82ac199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a9e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9f0 tx=0x7f829c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a2a00640 1 -- 172.21.15.138:0/1509016349 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c00cd80 con 0x7f82ac102ab0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a2a00640 1 -- 172.21.15.138:0/1509016349 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0054d0 con 0x7f82ac102ab0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82ac19f2d0 con 0x7f82ac102ab0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82a2a00640 1 -- 172.21.15.138:0/1509016349 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c010690 con 0x7f82ac102ab0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.454+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82ac1a5990 con 0x7f82ac102ab0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.457+0000 7f82b3a8b740 1 Processor -- start 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.457+0000 7f82b3a8b740 1 -- start start 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25aab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aab610 0x55cd25abb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd25a4c670 con 0x55cd25abb9b0 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd25a20d70 con 0x55cd25aab610 2024-08-20T01:25:50.515 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd25a10270 con 0x55cd25aaacf0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a2a00640 1 -- 172.21.15.138:0/1509016349 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f829c004e90 con 0x7f82ac102ab0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40908/0 (socket says 172.21.15.138:40908) 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 -- 172.21.15.138:0/1915823779 learned_addr learned my addr 172.21.15.138:0/1915823779 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82aa800640 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25aab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9e00640 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aab610 0x55cd25abb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 -- 172.21.15.138:0/1915823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 msgr2=0x55cd25aab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25aab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 -- 172.21.15.138:0/1915823779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aab610 msgr2=0x55cd25abb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aab610 0x55cd25abb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.458+0000 7f82a9400640 1 -- 172.21.15.138:0/1915823779 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd259ac870 con 0x55cd25abb9b0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.459+0000 7f82a9400640 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f829800bce0 tx=0x7f829800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc66608098fd5adb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.516 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.459+0000 7f8297400640 1 -- 172.21.15.138:0/1915823779 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8298011440 con 0x55cd25abb9b0 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.459+0000 7f8297400640 1 -- 172.21.15.138:0/1915823779 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8298011a90 con 0x55cd25abb9b0 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.459+0000 7f8297400640 1 -- 172.21.15.138:0/1915823779 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829801ba40 con 0x55cd25abb9b0 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- 172.21.15.138:0/1915823779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 msgr2=0x55cd25abdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f829800bce0 tx=0x7f829800f730 comp rx=0 tx=0).stop 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- 172.21.15.138:0/1915823779 shutdown_connections 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25abb9b0 0x55cd25abdda0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aab610 0x55cd25abb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/1915823779 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25aab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- 172.21.15.138:0/1915823779 >> 172.21.15.138:0/1915823779 conn(0x55cd25aa8b20 msgr2=0x55cd25aa9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- 172.21.15.138:0/1915823779 shutdown_connections 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- 172.21.15.138:0/1915823779 wait complete. 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82a2a00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 0x7f82900b6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82a9400640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 0x7f82900b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82a2a00640 1 -- 172.21.15.138:0/1509016349 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f829c0e4930 con 0x7f82ac102ab0 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.460+0000 7f82b3a8b740 1 -- start start 2024-08-20T01:25:50.517 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 Processor -- start 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82a9400640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 0x7f82900b6190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f829801b720 tx=0x7f8298005fd0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 0x55cd25abb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55cd25996ff0 con 0x55cd25aab610 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55cd25a142b0 con 0x55cd25aaacf0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55cd25872bb0 con 0x55cd25abb9b0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82b3a8b740 10 client.0 _ll_register_callbacks cb 0x55cd2571bd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.461+0000 7f82a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82aa800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40916/0 (socket says 172.21.15.138:40916) 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9400640 1 -- 172.21.15.138:0/2135114015 learned_addr learned my addr 172.21.15.138:0/2135114015 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:50.518 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 0x55cd25abb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82aa800640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45304/0 (socket says 172.21.15.138:45304) 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 -- 172.21.15.138:0/2135114015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 msgr2=0x55cd25ab8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 -- 172.21.15.138:0/2135114015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 msgr2=0x55cd25ab8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 -- 172.21.15.138:0/2135114015 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd2576fb00 con 0x55cd25aaacf0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82aa800640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f82a9e00640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 0x55cd25abb1c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f829c0e1c80 tx=0x7f829c0aa970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4634958f33bcb6f2 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.462+0000 7f828be00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0ab840 con 0x55cd25aaacf0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8298011a90 con 0x55cd25aaacf0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f829c0e1f00 con 0x55cd25aaacf0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f829c0e2100 con 0x55cd25aaacf0 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 10 client.27545 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:50.519 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cd257e6960 con 0x55cd25aaacf0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8298011a90 con 0x55cd25aaacf0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f829c0b3d60 con 0x55cd25aaacf0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 10 client.27545 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 10 client.27545 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 20 client.27545 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25acb0f0 0x55cd25acd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 4 client.27545 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.463+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd25a22cc0 con 0x55cd25acb0f0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.464+0000 7f82a9400640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25acb0f0 0x55cd25acd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.464+0000 7f828be00640 10 client.27545 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.464+0000 7f82a9400640 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25acb0f0 0x55cd25acd540 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b993b74b3b8169df server_cookie=d264e79e73b4779e in_seq=0 out_seq=0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f828be00640 1 -- 172.21.15.138:0/2135114015 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd25a22cc0 con 0x55cd25acb0f0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f828be00640 10 client.27545 handle_command_reply: tid=0 2024-08-20T01:25:50.520 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 client.27545 shutdown 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 msgr2=0x55cd25abb1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 0x55cd25abb1c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f829c0e1c80 tx=0x7f829c0aa970 comp rx=0 tx=0).stop 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 shutdown_connections 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25acb0f0 0x55cd25acd540 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25abb9b0 0x55cd25ab8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f828be00640 0 client.27545 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/2135114015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25aaacf0 0x55cd25abb1c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.465+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 >> 172.21.15.138:0/2135114015 conn(0x55cd25aa8b20 msgr2=0x55cd25aae6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.466+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 shutdown_connections 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.466+0000 7f82b3a8b740 1 -- 172.21.15.138:0/2135114015 wait complete. 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.466+0000 7f82b3a8b740 20 client.27545 trim_cache size 0 max 16384 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.483+0000 7f82b3a8b740 1 Processor -- start 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.483+0000 7f82b3a8b740 1 -- start start 2024-08-20T01:25:50.521 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25aa7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1b02d0 con 0x55cd25aab610 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1adb40 con 0x55cd25abb9b0 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1ab9b0 con 0x55cd25aaacf0 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45312/0 (socket says 172.21.15.138:45312) 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 -- 172.21.15.138:0/3636922644 learned_addr learned my addr 172.21.15.138:0/3636922644 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82aa800640 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25aa7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 -- 172.21.15.138:0/3636922644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 msgr2=0x55cd25aa7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25aa7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 -- 172.21.15.138:0/3636922644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 msgr2=0x55cd25ab85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.522 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 -- 172.21.15.138:0/3636922644 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd25aba320 con 0x55cd25aaacf0 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82aa800640 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25aa7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.484+0000 7f82a9e00640 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9c0 tx=0x7f829c0aa9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d379675c40fedac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f8296a00640 1 -- 172.21.15.138:0/3636922644 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0aaf10 con 0x55cd25aaacf0 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f8296a00640 1 -- 172.21.15.138:0/3636922644 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0b41f0 con 0x55cd25aaacf0 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f8296a00640 1 -- 172.21.15.138:0/3636922644 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0ab140 con 0x55cd25aaacf0 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 -- 172.21.15.138:0/3636922644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 msgr2=0x55cd25ac7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9c0 tx=0x7f829c0aa9e0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 -- 172.21.15.138:0/3636922644 shutdown_connections 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25aa7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aab610 0x55cd25ab85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/3636922644 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aaacf0 0x55cd25ac7630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.485+0000 7f82b3a8b740 1 -- 172.21.15.138:0/3636922644 >> 172.21.15.138:0/3636922644 conn(0x55cd25aa8b20 msgr2=0x55cd25aa7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.486+0000 7f82b3a8b740 1 -- 172.21.15.138:0/3636922644 shutdown_connections 2024-08-20T01:25:50.523 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.486+0000 7f82b3a8b740 1 -- 172.21.15.138:0/3636922644 wait complete. 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.486+0000 7f82b3a8b740 1 -- start start 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.486+0000 7f82b3a8b740 1 Processor -- start 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 0x55cd25ac37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25ac3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1ac000 con 0x55cd25aaacf0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1ad470 con 0x55cd25abb9b0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f82ac1ac6d0 con 0x55cd25aab610 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82b3a8b740 10 client.0 _ll_register_callbacks cb 0x7f829c004990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:40932/0 (socket says 172.21.15.138:40932) 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 -- 172.21.15.138:0/275623032 learned_addr learned my addr 172.21.15.138:0/275623032 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9400640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 0x55cd25ac37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82aa800640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25ac3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.524 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 -- 172.21.15.138:0/275623032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 msgr2=0x55cd25ac37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 0x55cd25ac37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 -- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 msgr2=0x55cd25ac3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25ac3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9e00640 1 -- 172.21.15.138:0/275623032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd25abb530 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.487+0000 7f82a9400640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 0x55cd25ac37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f82a9e00640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c6a0 tx=0x7f829c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b36e64775b791ec server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f828be00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829c0aaf10 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f829c0e1f00 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f829c0b41f0 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f829c0e2100 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f82b3a8b740 10 client.17511 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55cd25b36060 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.488+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f829c0b41f0 con 0x55cd25aaacf0 2024-08-20T01:25:50.525 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f829c0a9fb0 con 0x55cd25aaacf0 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 10 client.17511 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 10 client.17511 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 20 client.17511 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25b364f0 0x55cd25b38910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 4 client.17511 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a27ddc43", "release": true, "await": true}] 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a27ddc43", "release": true, "await": true}) -- 0x55cd25a1e760 con 0x55cd25b364f0 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82a9400640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25b364f0 0x55cd25b38910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f82a9400640 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25b364f0 0x55cd25b38910 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc29a78255025bd server_cookie=a80449adc2f3ce2f in_seq=0 out_seq=0 2024-08-20T01:25:50.526 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.489+0000 7f828be00640 10 client.17511 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:50.527 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.526+0000 7f828be00640 1 -- 172.21.15.138:0/275623032 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55cd25a1e760 con 0x55cd25b364f0 2024-08-20T01:25:50.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.526+0000 7f828be00640 10 client.17511 handle_command_reply: tid=0 2024-08-20T01:25:50.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.526+0000 7f82b3a8b740 1 client.17511 shutdown 2024-08-20T01:25:50.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 msgr2=0x55cd25ad36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.528 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c6a0 tx=0x7f829c009310 comp rx=0 tx=0).stop 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 shutdown_connections 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55cd25b364f0 0x55cd25b38910 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cd25abb9b0 0x55cd25ac3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f828be00640 0 client.17511 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55cd25aab610 0x55cd25ac37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 --2- 172.21.15.138:0/275623032 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55cd25aaacf0 0x55cd25ad36a0 secure :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f829c00c6a0 tx=0x7f829c009310 comp rx=0 tx=0).stop 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 >> 172.21.15.138:0/275623032 conn(0x55cd25aa8b20 msgr2=0x55cd25ab76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.527+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 shutdown_connections 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.528+0000 7f82b3a8b740 1 -- 172.21.15.138:0/275623032 wait complete. 2024-08-20T01:25:50.529 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.528+0000 7f82b3a8b740 20 client.17511 trim_cache size 0 max 16384 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 54, 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 64, 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "a27ddc43": { 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 64, 2024-08-20T01:25:50.530 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 144.9, 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:50.531 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 msgr2=0x7f82900b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 0x7f82900b6190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f829801b720 tx=0x7f8298005fd0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 msgr2=0x7f82ac198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f829c00e9f0 tx=0x7f829c00edf0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 shutdown_connections 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82900b3cd0 0x7f82900b6190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f82ac108e40 0x7f82ac19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.532 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82ac1084c0 0x7f82ac199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 --2- 172.21.15.138:0/1509016349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82ac102ab0 0x7f82ac198f30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:50.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 >> 172.21.15.138:0/1509016349 conn(0x7f82ac0fe500 msgr2=0x7f82ac1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:50.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.529+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 shutdown_connections 2024-08-20T01:25:50.533 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:50.530+0000 7f82b0e00640 1 -- 172.21.15.138:0/1509016349 wait complete. 2024-08-20T01:25:51.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 70886 14.23 MB/sec execute 108 sec latency 20871.698 ms 2024-08-20T01:25:51.420 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a27ddc43', total seconds elapsed: 29.8 2024-08-20T01:25:51.420 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.95 2024-08-20T01:25:51.421 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:25:52.062 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:52.282 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:51 smithi138 ceph-mon[18076]: pgmap v917: 129 pgs: 129 active+clean; 1003 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-08-20T01:25:52.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 71731 14.49 MB/sec execute 109 sec latency 107.180 ms 2024-08-20T01:25:52.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:51 smithi167 ceph-mon[21965]: pgmap v917: 129 pgs: 129 active+clean; 1003 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-08-20T01:25:52.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:51 smithi142 ceph-mon[21799]: pgmap v917: 129 pgs: 129 active+clean; 1003 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 259 op/s 2024-08-20T01:25:53.283 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 -- 172.21.15.138:0/3029155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6010a6b0 msgr2=0x7f4f6010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 --2- 172.21.15.138:0/3029155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6010a6b0 0x7f4f6010aaf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4f48009900 tx=0x7f4f4802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 -- 172.21.15.138:0/3029155578 shutdown_connections 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 --2- 172.21.15.138:0/3029155578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010b030 0x7f4f60111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 --2- 172.21.15.138:0/3029155578 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f6010a6b0 0x7f4f6010aaf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 --2- 172.21.15.138:0/3029155578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f60104ca0 0x7f4f60105080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 -- 172.21.15.138:0/3029155578 >> 172.21.15.138:0/3029155578 conn(0x7f4f60100730 msgr2=0x7f4f60102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 -- 172.21.15.138:0/3029155578 shutdown_connections 2024-08-20T01:25:53.284 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.282+0000 7f4f66c00640 1 -- 172.21.15.138:0/3029155578 wait complete. 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.283+0000 7f4f66c00640 1 Processor -- start 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.283+0000 7f4f66c00640 1 -- start start 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010a6b0 0x7f4f60074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f6010b030 0x7f4f6006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60114d00 con 0x7f4f6010b030 2024-08-20T01:25:53.285 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60114b80 con 0x7f4f6010a6b0 2024-08-20T01:25:53.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f66c00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4f60114e80 con 0x7f4f60104ca0 2024-08-20T01:25:53.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:53.286 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f5fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45326/0 (socket says 172.21.15.138:45326) 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f5fe00640 1 -- 172.21.15.138:0/342507057 learned_addr learned my addr 172.21.15.138:0/342507057 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.284+0000 7f4f57e00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010a6b0 0x7f4f60074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 -- 172.21.15.138:0/342507057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010a6b0 msgr2=0x7f4f60074a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010a6b0 0x7f4f60074a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 -- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f6010b030 msgr2=0x7f4f6006d840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f6010b030 0x7f4f6006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 -- 172.21.15.138:0/342507057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f6006e100 con 0x7f4f60104ca0 2024-08-20T01:25:53.287 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.285+0000 7f4f5fe00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000e9f0 tx=0x7f4f5000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.286+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000cd80 con 0x7f4f60104ca0 2024-08-20T01:25:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.286+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f500108e0 con 0x7f4f60104ca0 2024-08-20T01:25:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.286+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50018860 con 0x7f4f60104ca0 2024-08-20T01:25:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.286+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f6006e3f0 con 0x7f4f60104ca0 2024-08-20T01:25:53.288 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.287+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f60073300 con 0x7f4f60104ca0 2024-08-20T01:25:53.289 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.288+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f60105f50 con 0x7f4f60104ca0 2024-08-20T01:25:53.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.291+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4f50010430 con 0x7f4f60104ca0 2024-08-20T01:25:53.295 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.294+0000 7f4f5d600640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 0x7f4f440b6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:53.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.294+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f4f500e4630 con 0x7f4f60104ca0 2024-08-20T01:25:53.296 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.295+0000 7f4f57e00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 0x7f4f440b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:53.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.297+0000 7f4f57e00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 0x7f4f440b6350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f48005ab0 tx=0x7f4f48005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:53.298 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.297+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4f500ab5a0 con 0x7f4f60104ca0 2024-08-20T01:25:53.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 72596 14.89 MB/sec execute 110 sec latency 156.340 ms 2024-08-20T01:25:53.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.713+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f6010c880 con 0x7f4f60104ca0 2024-08-20T01:25:53.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.714+0000 7f4f5d600640 1 -- 172.21.15.138:0/342507057 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7f4f500aaea0 con 0x7f4f60104ca0 2024-08-20T01:25:53.715 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:25:53.716 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 54 2024-08-20T01:25:53.716 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":54,"btime":"2024-08-20T01:25:19:989006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:25:19.256053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 msgr2=0x7f4f440b6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 0x7f4f440b6350 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4f48005ab0 tx=0x7f4f48005a20 comp rx=0 tx=0).stop 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 msgr2=0x7f4f600744c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000e9f0 tx=0x7f4f5000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 shutdown_connections 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4f440b3e90 0x7f4f440b6350 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4f6010b030 0x7f4f6006d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6010a6b0 0x7f4f60074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 --2- 172.21.15.138:0/342507057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4f60104ca0 0x7f4f600744c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:53.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.719+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 >> 172.21.15.138:0/342507057 conn(0x7f4f60100730 msgr2=0x7f4f60102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:53.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.720+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 shutdown_connections 2024-08-20T01:25:53.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:53.720+0000 7f4f66c00640 1 -- 172.21.15.138:0/342507057 wait complete. 2024-08-20T01:25:54.007 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:54 smithi138 ceph-mon[18076]: pgmap v918: 129 pgs: 129 active+clean; 1006 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-08-20T01:25:54.007 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:54 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/342507057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:54.347 INFO:tasks.workunit.client.0.smithi138.stdout: 10 73283 15.05 MB/sec execute 111 sec latency 59.241 ms 2024-08-20T01:25:54.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:54 smithi167 ceph-mon[21965]: pgmap v918: 129 pgs: 129 active+clean; 1006 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-08-20T01:25:54.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:54 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/342507057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:54.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:54 smithi142 ceph-mon[21799]: pgmap v918: 129 pgs: 129 active+clean; 1006 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-08-20T01:25:54.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:54 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/342507057' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:25:54.588 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0' 2024-08-20T01:25:54.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db / --timeout 90 --expiration 144.95 --await-for 45.0 2024-08-20T01:25:55.256 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:25:55.348 INFO:tasks.workunit.client.0.smithi138.stdout: 10 73926 15.23 MB/sec execute 112 sec latency 59.205 ms 2024-08-20T01:25:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.746+0000 7f2565400640 1 -- 172.21.15.138:0/3006907625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 msgr2=0x7f256006a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.746+0000 7f2565400640 1 --2- 172.21.15.138:0/3006907625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f256006a530 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2550013f10 tx=0x7f2550032980 comp rx=0 tx=0).stop 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 -- 172.21.15.138:0/3006907625 shutdown_connections 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 --2- 172.21.15.138:0/3006907625 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 0x7f2560111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 --2- 172.21.15.138:0/3006907625 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f256006aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 --2- 172.21.15.138:0/3006907625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f256006a530 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 -- 172.21.15.138:0/3006907625 >> 172.21.15.138:0/3006907625 conn(0x7f2560077270 msgr2=0x7f2560077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 -- 172.21.15.138:0/3006907625 shutdown_connections 2024-08-20T01:25:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.747+0000 7f2565400640 1 -- 172.21.15.138:0/3006907625 wait complete. 2024-08-20T01:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.748+0000 7f2565400640 1 Processor -- start 2024-08-20T01:25:55.749 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 -- start start 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f25601975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 0x7f2560199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25601063c0 con 0x7f256010ef30 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2560106240 con 0x7f25601033c0 2024-08-20T01:25:55.750 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f2565400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2560106540 con 0x7f256006aa70 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f255f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f25601975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45336/0 (socket says 172.21.15.138:45336) 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.749+0000 7f255fe00640 1 -- 172.21.15.138:0/1094335975 learned_addr learned my addr 172.21.15.138:0/1094335975 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f2564a00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 0x7f2560199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 -- 172.21.15.138:0/1094335975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 msgr2=0x7f25601975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f25601975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 -- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 msgr2=0x7f2560199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 0x7f2560199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 -- 172.21.15.138:0/1094335975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2560199a40 con 0x7f256006aa70 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.750+0000 7f255fe00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2550032ef0 tx=0x7f25500053c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.752 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.751+0000 7f255cc00640 1 -- 172.21.15.138:0/1094335975 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550004c10 con 0x7f256006aa70 2024-08-20T01:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.751+0000 7f255cc00640 1 -- 172.21.15.138:0/1094335975 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2550046070 con 0x7f256006aa70 2024-08-20T01:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.751+0000 7f255cc00640 1 -- 172.21.15.138:0/1094335975 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2550038970 con 0x7f256006aa70 2024-08-20T01:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.751+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2560199cd0 con 0x7f256006aa70 2024-08-20T01:25:55.753 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.751+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601a3630 con 0x7f256006aa70 2024-08-20T01:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.756+0000 7f255cc00640 1 -- 172.21.15.138:0/1094335975 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f255003f070 con 0x7f256006aa70 2024-08-20T01:25:55.757 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.756+0000 7f2569854740 1 Processor -- start 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 -- start start 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fb1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fc0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562794f27210 con 0x562794fbe080 2024-08-20T01:25:55.758 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562794f16710 con 0x562794fb17d0 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.757+0000 7f2569854740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562794f1a750 con 0x562794fb20f0 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.758+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.758+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45338/0 (socket says 172.21.15.138:45338) 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.758+0000 7f255fe00640 1 -- 172.21.15.138:0/1822312843 learned_addr learned my addr 172.21.15.138:0/1822312843 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.758+0000 7f2564a00640 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fb1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.759 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.758+0000 7f255cc00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 0x7f25440b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255fe00640 1 -- 172.21.15.138:0/1822312843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 msgr2=0x562794fb1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255fe00640 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fb1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255fe00640 1 -- 172.21.15.138:0/1822312843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 msgr2=0x562794fc0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255fe00640 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fc0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255cc00640 1 -- 172.21.15.138:0/1094335975 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f255010d000 con 0x7f256006aa70 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f2564a00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 0x7f25440b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.760 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.759+0000 7f255fe00640 1 -- 172.21.15.138:0/1822312843 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562794ee2de0 con 0x562794fb20f0 2024-08-20T01:25:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.760+0000 7f255fe00640 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f255010a430 tx=0x7f2550038470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6075c11423d8b6f8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.760+0000 7f2555200640 1 -- 172.21.15.138:0/1822312843 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500d3750 con 0x562794fb20f0 2024-08-20T01:25:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.760+0000 7f2555200640 1 -- 172.21.15.138:0/1822312843 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25500d3490 con 0x562794fb20f0 2024-08-20T01:25:55.761 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.760+0000 7f2555200640 1 -- 172.21.15.138:0/1822312843 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500cf110 con 0x562794fb20f0 2024-08-20T01:25:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 -- 172.21.15.138:0/1822312843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 msgr2=0x562794fbdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2564a00640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 0x7f25440b62b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2560075480 tx=0x7f2558004a40 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f255010a430 tx=0x7f2550038470 comp rx=0 tx=0).stop 2024-08-20T01:25:55.762 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 -- 172.21.15.138:0/1822312843 shutdown_connections 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fc0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbdb40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 --2- 172.21.15.138:0/1822312843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fb1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 -- 172.21.15.138:0/1822312843 >> 172.21.15.138:0/1822312843 conn(0x562794faf600 msgr2=0x562794fb07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 -- 172.21.15.138:0/1822312843 shutdown_connections 2024-08-20T01:25:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.761+0000 7f2569854740 1 -- 172.21.15.138:0/1822312843 wait complete. 2024-08-20T01:25:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.763+0000 7f2569854740 1 -- start start 2024-08-20T01:25:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.763+0000 7f2569854740 1 Processor -- start 2024-08-20T01:25:55.765 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fbdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fbb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x562794e4fab0 con 0x562794fbe080 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562794ebd140 con 0x562794fb17d0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x562794fc0b20 con 0x562794fb20f0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2569854740 10 client.0 _ll_register_callbacks cb 0x562794ed05d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f255f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f255f400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45348/0 (socket says 172.21.15.138:45348) 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f255f400640 1 -- 172.21.15.138:0/781579593 learned_addr learned my addr 172.21.15.138:0/781579593 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f2564a00640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fbb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.765+0000 7f255fe00640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fbdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 -- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 msgr2=0x562794fbdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fbdb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 -- 172.21.15.138:0/781579593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 msgr2=0x562794fbb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fbb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 -- 172.21.15.138:0/781579593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562794ebe170 con 0x562794fb20f0 2024-08-20T01:25:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f255f400640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f254c011660 tx=0x7f254c011b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c916c20bc5e35894 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.766+0000 7f2549600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:25:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.768+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c004220 con 0x562794fb20f0 2024-08-20T01:25:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.769+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f254c0043c0 con 0x562794fb20f0 2024-08-20T01:25:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.769+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562794eeda30 con 0x562794fb20f0 2024-08-20T01:25:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.769+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f254c011e60 con 0x562794fb20f0 2024-08-20T01:25:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.769+0000 7f2569854740 10 client.27526 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.769+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562794c79790 con 0x562794fb20f0 2024-08-20T01:25:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562794eeda30 con 0x562794fb20f0 2024-08-20T01:25:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f254c0047a0 con 0x562794fb20f0 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 10 client.27526 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 10 client.27526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 20 client.27526 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562794fcdd20 0x562794fd0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 4 client.27526 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.770+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562794f29160 con 0x562794fcdd20 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.771+0000 7f255fe00640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562794fcdd20 0x562794fd0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.772+0000 7f255fe00640 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562794fcdd20 0x562794fd0110 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a184057abef9f7c6 server_cookie=c9fc89b91d7cda27 in_seq=0 out_seq=0 2024-08-20T01:25:55.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.772+0000 7f2549600640 10 client.27526 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.774+0000 7f2549600640 1 -- 172.21.15.138:0/781579593 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562794f29160 con 0x562794fcdd20 2024-08-20T01:25:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.774+0000 7f2549600640 10 client.27526 handle_command_reply: tid=0 2024-08-20T01:25:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.774+0000 7f2569854740 1 client.27526 shutdown 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.776+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 msgr2=0x562794fbaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.776+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f254c011660 tx=0x7f254c011b20 comp rx=0 tx=0).stop 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 shutdown_connections 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562794fcdd20 0x562794fd0110 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fbb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb20f0 0x562794fbaf80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 --2- 172.21.15.138:0/781579593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562794fbdb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 >> 172.21.15.138:0/781579593 conn(0x562794faf600 msgr2=0x562794fb51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2549600640 0 client.27526 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 shutdown_connections 2024-08-20T01:25:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 1 -- 172.21.15.138:0/781579593 wait complete. 2024-08-20T01:25:55.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.777+0000 7f2569854740 20 client.27526 trim_cache size 0 max 16384 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 Processor -- start 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 -- start start 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb20f0 0x562794fb09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fd59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25601b0170 con 0x562794fbe080 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ad9e0 con 0x562794fb20f0 2024-08-20T01:25:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f2569854740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ab850 con 0x562794fb17d0 2024-08-20T01:25:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f255f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb20f0 0x562794fb09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f255fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45358/0 (socket says 172.21.15.138:45358) 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.807+0000 7f255fe00640 1 -- 172.21.15.138:0/4081487419 learned_addr learned my addr 172.21.15.138:0/4081487419 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 -- 172.21.15.138:0/4081487419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb20f0 msgr2=0x562794fb09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb20f0 0x562794fb09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 -- 172.21.15.138:0/4081487419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 msgr2=0x562794fd59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fd59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 -- 172.21.15.138:0/4081487419 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562794cf2620 con 0x562794fb17d0 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f255010a9b0 tx=0x7f2550110b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6268b43e06ad5b0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f254be00640 1 -- 172.21.15.138:0/4081487419 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500d30f0 con 0x562794fb17d0 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f254be00640 1 -- 172.21.15.138:0/4081487419 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25500bb0b0 con 0x562794fb17d0 2024-08-20T01:25:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f254be00640 1 -- 172.21.15.138:0/4081487419 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25500d32d0 con 0x562794fb17d0 2024-08-20T01:25:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- 172.21.15.138:0/4081487419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 msgr2=0x562794faed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f255010a9b0 tx=0x7f2550110b50 comp rx=0 tx=0).stop 2024-08-20T01:25:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- 172.21.15.138:0/4081487419 shutdown_connections 2024-08-20T01:25:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fbe080 0x562794fd59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb20f0 0x562794fb09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- 172.21.15.138:0/4081487419 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fb17d0 0x562794faed80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- 172.21.15.138:0/4081487419 >> 172.21.15.138:0/4081487419 conn(0x562794faf600 msgr2=0x562794fb02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:55.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- 172.21.15.138:0/4081487419 shutdown_connections 2024-08-20T01:25:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- 172.21.15.138:0/4081487419 wait complete. 2024-08-20T01:25:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- start start 2024-08-20T01:25:55.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 Processor -- start 2024-08-20T01:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 0x5627950362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 0x562794fc5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f255fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56872/0 (socket says 172.21.15.138:56872) 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f25601abea0 con 0x562794fb20f0 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ad310 con 0x562794fb17d0 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.809+0000 7f2569854740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ac570 con 0x562794fbe080 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.814+0000 7f2569854740 10 client.0 _ll_register_callbacks cb 0x7f254c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.814+0000 7f2564a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 0x562794fc5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.814+0000 7f2564a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 0x562794fc5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45372/0 (socket says 172.21.15.138:45372) 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.814+0000 7f2564a00640 1 -- 172.21.15.138:0/2579569643 learned_addr learned my addr 172.21.15.138:0/2579569643 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:55.815 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.814+0000 7f255f400640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 0x5627950362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255f400640 1 -- 172.21.15.138:0/2579569643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 msgr2=0x562794fc5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255f400640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 0x562794fc5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255f400640 1 -- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 msgr2=0x562795035d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255f400640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:55.816 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255f400640 1 -- 172.21.15.138:0/2579569643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562794fc6590 con 0x562794fb20f0 2024-08-20T01:25:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.815+0000 7f255fe00640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.816+0000 7f255f400640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 0x5627950362d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f254c009c40 tx=0x7f254c00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc63bb757dd7dde server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:55.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.816+0000 7f2549600640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.816+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c034060 con 0x562794fb20f0 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f254c009c70 con 0x562794fb20f0 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562794f24c00 con 0x562794fb20f0 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f254c028040 con 0x562794fb20f0 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2569854740 10 client.17523 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:55.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x562794fb8760 con 0x562794fb20f0 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.817+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562794f24c00 con 0x562794fb20f0 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f254c02f6c0 con 0x562794fb20f0 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 10 client.17523 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 20 client.17523 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:55.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562795049ec0 0x56279504c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 4 client.17523 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:25:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562794f22340 con 0x562795049ec0 2024-08-20T01:25:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.818+0000 7f255fe00640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562795049ec0 0x56279504c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.819+0000 7f255fe00640 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562795049ec0 0x56279504c2e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18338e31e0ab0aee server_cookie=19af61e9abcd67a4 in_seq=0 out_seq=0 2024-08-20T01:25:55.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:55.819+0000 7f2549600640 10 client.17523 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:56.120 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:56 smithi138 ceph-mon[18076]: pgmap v919: 129 pgs: 129 active+clean; 1016 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-08-20T01:25:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.139+0000 7f2549600640 1 -- 172.21.15.138:0/2579569643 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562794f22340 con 0x562795049ec0 2024-08-20T01:25:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.139+0000 7f2549600640 10 client.17523 handle_command_reply: tid=0 2024-08-20T01:25:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.139+0000 7f2569854740 1 client.17523 shutdown 2024-08-20T01:25:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.139+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 msgr2=0x5627950362d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 0x5627950362d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f254c009c40 tx=0x7f254c00faf0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 shutdown_connections 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x562795049ec0 0x56279504c2e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562794fbe080 0x562794fc5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562794fb20f0 0x5627950362d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 --2- 172.21.15.138:0/2579569643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562794fb17d0 0x562795035d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 >> 172.21.15.138:0/2579569643 conn(0x562794faf600 msgr2=0x562794fbc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2549600640 0 client.17523 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.140+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 shutdown_connections 2024-08-20T01:25:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.141+0000 7f2569854740 1 -- 172.21.15.138:0/2579569643 wait complete. 2024-08-20T01:25:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.142+0000 7f2569854740 20 client.17523 trim_cache size 0 max 16384 2024-08-20T01:25:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:25:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:25:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:25:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:25:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.144+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 msgr2=0x7f25440b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.144+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 0x7f25440b62b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2560075480 tx=0x7f2558004a40 comp rx=0 tx=0).stop 2024-08-20T01:25:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.144+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 msgr2=0x7f25601970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.144+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2550032ef0 tx=0x7f25500053c0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 shutdown_connections 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f25440b3df0 0x7f25440b62b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f256010ef30 0x7f2560199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f25601033c0 0x7f25601975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 --2- 172.21.15.138:0/1094335975 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f256006aa70 0x7f25601970a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 >> 172.21.15.138:0/1094335975 conn(0x7f2560077270 msgr2=0x7f25601008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.145+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 shutdown_connections 2024-08-20T01:25:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:25:56.146+0000 7f2565400640 1 -- 172.21.15.138:0/1094335975 wait complete. 2024-08-20T01:25:56.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 74616 15.36 MB/sec execute 113 sec latency 216.608 ms 2024-08-20T01:25:56.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:56 smithi167 ceph-mon[21965]: pgmap v919: 129 pgs: 129 active+clean; 1016 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-08-20T01:25:56.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:56 smithi142 ceph-mon[21799]: pgmap v919: 129 pgs: 129 active+clean; 1016 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 -- 172.21.15.138:0/3727315471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 msgr2=0x7fb09c113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/3727315471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c113160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb088009900 tx=0x7fb08802a3f0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 -- 172.21.15.138:0/3727315471 shutdown_connections 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/3727315471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c1136a0 0x7fb09c115a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/3727315471 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c113160 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/3727315471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c077400 0x7fb09c0777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.716+0000 7fb0a2a00640 1 -- 172.21.15.138:0/3727315471 >> 172.21.15.138:0/3727315471 conn(0x7fb09c100720 msgr2=0x7fb09c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.717+0000 7fb0a2a00640 1 -- 172.21.15.138:0/3727315471 shutdown_connections 2024-08-20T01:25:56.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.717+0000 7fb0a2a00640 1 -- 172.21.15.138:0/3727315471 wait complete. 2024-08-20T01:25:56.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.718+0000 7fb0a2a00640 1 Processor -- start 2024-08-20T01:25:56.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.718+0000 7fb0a2a00640 1 -- start start 2024-08-20T01:25:56.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c077400 0x7fb09c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 0x7fb09c1a4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1190d0 con 0x7fb09c077400 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c118f50 con 0x7fb09c1136a0 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.719+0000 7fb0a2a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c119250 con 0x7fb09c077d20 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.720+0000 7fb09b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.720+0000 7fb09b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45392/0 (socket says 172.21.15.138:45392) 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.720+0000 7fb09be00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c077400 0x7fb09c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.720+0000 7fb09b400640 1 -- 172.21.15.138:0/1262438487 learned_addr learned my addr 172.21.15.138:0/1262438487 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.720+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 0x7fb09c1a4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 -- 172.21.15.138:0/1262438487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 msgr2=0x7fb09c1a4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 0x7fb09c1a4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 -- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c077400 msgr2=0x7fb09c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c077400 0x7fb09c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 -- 172.21.15.138:0/1262438487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09c1a55f0 con 0x7fb09c077d20 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 0x7fb09c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:25:56.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.721+0000 7fb09b400640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb088005ab0 tx=0x7fb088003c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.723+0000 7fb098c00640 1 -- 172.21.15.138:0/1262438487 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08803f070 con 0x7fb09c077d20 2024-08-20T01:25:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.723+0000 7fb098c00640 1 -- 172.21.15.138:0/1262438487 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb088031d60 con 0x7fb09c077d20 2024-08-20T01:25:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.723+0000 7fb098c00640 1 -- 172.21.15.138:0/1262438487 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088035030 con 0x7fb09c077d20 2024-08-20T01:25:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.723+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09c1a5880 con 0x7fb09c077d20 2024-08-20T01:25:56.724 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.723+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09c1abfa0 con 0x7fb09c077d20 2024-08-20T01:25:56.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.728+0000 7fb0a546f740 1 Processor -- start 2024-08-20T01:25:56.730 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.728+0000 7fb0a546f740 1 -- start start 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30a9a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30a9cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556d30a03600 con 0x556d30a8e760 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556d309f2b00 con 0x556d30a8de40 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a546f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556d309f6b40 con 0x556d30a9a6f0 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a0a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56886/0 (socket says 172.21.15.138:56886) 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb0a0a00640 1 -- 172.21.15.138:0/4132872716 learned_addr learned my addr 172.21.15.138:0/4132872716 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:56.731 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.729+0000 7fb09be00640 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30a9a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb098c00640 1 -- 172.21.15.138:0/1262438487 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb088042070 con 0x7fb09c077d20 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb0a0a00640 1 -- 172.21.15.138:0/4132872716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 msgr2=0x556d30a9cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30a9cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb0a0a00640 1 -- 172.21.15.138:0/4132872716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 msgr2=0x556d30a9a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30a9a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.730+0000 7fb0a0a00640 1 -- 172.21.15.138:0/4132872716 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d30998ee0 con 0x556d30a8de40 2024-08-20T01:25:56.732 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.731+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb094010740 tx=0x7fb094010c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a980f5bdfd2de4d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.731+0000 7fb091200640 1 -- 172.21.15.138:0/4132872716 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09400ead0 con 0x556d30a8de40 2024-08-20T01:25:56.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.732+0000 7fb091200640 1 -- 172.21.15.138:0/4132872716 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094010e50 con 0x556d30a8de40 2024-08-20T01:25:56.733 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.732+0000 7fb091200640 1 -- 172.21.15.138:0/4132872716 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09401c670 con 0x556d30a8de40 2024-08-20T01:25:56.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.732+0000 7fb098c00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 0x7fb0800b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.732+0000 7fb098c00640 1 -- 172.21.15.138:0/1262438487 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fb088104a40 con 0x7fb09c077d20 2024-08-20T01:25:56.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb09be00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 0x7fb0800b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 -- 172.21.15.138:0/4132872716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 msgr2=0x556d30a8e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.734 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb094010740 tx=0x7fb094010c00 comp rx=0 tx=0).stop 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 -- 172.21.15.138:0/4132872716 shutdown_connections 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30a9cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30a9a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 --2- 172.21.15.138:0/4132872716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30a8e220 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb0a546f740 1 -- 172.21.15.138:0/4132872716 >> 172.21.15.138:0/4132872716 conn(0x556d30a8bc70 msgr2=0x556d30a8ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.733+0000 7fb09be00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 0x7fb0800b63d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x556d30a8cb50 tx=0x7fb08c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.735 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.734+0000 7fb0a546f740 1 -- 172.21.15.138:0/4132872716 shutdown_connections 2024-08-20T01:25:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.734+0000 7fb0a546f740 1 -- 172.21.15.138:0/4132872716 wait complete. 2024-08-20T01:25:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.734+0000 7fb0a546f740 1 -- start start 2024-08-20T01:25:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.735+0000 7fb0a546f740 1 Processor -- start 2024-08-20T01:25:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.735+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 0x556d30a976a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.735+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a9a6f0 0x556d30a98120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a546f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x556d308557b0 con 0x556d30a8de40 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a546f740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x556d3098ddd0 con 0x556d30a9a6f0 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a546f740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x556d30aae380 con 0x556d30a8e760 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a546f740 10 client.0 _ll_register_callbacks cb 0x556d306ff0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb09b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb09b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45412/0 (socket says 172.21.15.138:45412) 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb09b400640 1 -- 172.21.15.138:0/3476795369 learned_addr learned my addr 172.21.15.138:0/3476795369 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb09be00640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 0x556d30a976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.736+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a9a6f0 0x556d30a98120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 -- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a9a6f0 msgr2=0x556d30a98120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a9a6f0 0x556d30a98120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 -- 172.21.15.138:0/3476795369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 msgr2=0x556d30a976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 0x556d30a976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 -- 172.21.15.138:0/3476795369 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30761ba0 con 0x556d30a8e760 2024-08-20T01:25:56.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09be00640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 0x556d30a976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:25:56.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb09b400640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb08802a9b0 tx=0x7fb0880c72a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73967b6feaabbd2f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.737+0000 7fb085600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088049040 con 0x556d30a8e760 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fb088102010 con 0x556d30a8e760 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb094010e50 con 0x556d30a8e760 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb0880cae10 con 0x556d30a8e760 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb0a546f740 10 client.27541 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.738+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556d30ab9f20 con 0x556d30a8e760 2024-08-20T01:25:56.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.739+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094010e50 con 0x556d30a8e760 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.739+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fb0880ec310 con 0x556d30a8e760 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.739+0000 7fb0a546f740 10 client.27541 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.739+0000 7fb0a546f740 10 client.27541 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.740+0000 7fb0a546f740 20 client.27541 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.740+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30abb3d0 0x556d30abd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.740+0000 7fb0a546f740 4 client.27541 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.740+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d30a05550 con 0x556d30abb3d0 2024-08-20T01:25:56.741 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.740+0000 7fb09be00640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30abb3d0 0x556d30abd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.741+0000 7fb09be00640 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30abb3d0 0x556d30abd800 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d085b4a7b21b554c server_cookie=dd7d729cb7ecec11 in_seq=0 out_seq=0 2024-08-20T01:25:56.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.741+0000 7fb085600640 10 client.27541 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:56.744 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.743+0000 7fb085600640 1 -- 172.21.15.138:0/3476795369 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d30a05550 con 0x556d30abb3d0 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.743+0000 7fb085600640 10 client.27541 handle_command_reply: tid=0 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.743+0000 7fb0a546f740 1 client.27541 shutdown 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 msgr2=0x556d30a97be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb08802a9b0 tx=0x7fb0880c72a0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 shutdown_connections 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30abb3d0 0x556d30abd800 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.745 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb085600640 0 client.27541 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a9a6f0 0x556d30a98120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8e760 0x556d30a97be0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3476795369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8de40 0x556d30a976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.744+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 >> 172.21.15.138:0/3476795369 conn(0x556d30a8bc70 msgr2=0x556d30a91810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.745+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 shutdown_connections 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.745+0000 7fb0a546f740 1 -- 172.21.15.138:0/3476795369 wait complete. 2024-08-20T01:25:56.746 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.745+0000 7fb0a546f740 20 client.27541 trim_cache size 0 max 16384 2024-08-20T01:25:56.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.776+0000 7fb0a546f740 1 Processor -- start 2024-08-20T01:25:56.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.776+0000 7fb0a546f740 1 -- start start 2024-08-20T01:25:56.777 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.776+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 0x556d30a91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 0x556d30ac30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 0x556d30ac3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.778 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb09be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 0x556d30a91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb09be00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 0x556d30a91eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:45428/0 (socket says 172.21.15.138:45428) 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb09be00640 1 -- 172.21.15.138:0/1573307666 learned_addr learned my addr 172.21.15.138:0/1573307666 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb09b400640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 0x556d30ac30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.777+0000 7fb0a546f740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1b01a0 con 0x556d30a9a6f0 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.778+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ada10 con 0x556d30a8e760 2024-08-20T01:25:56.779 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.778+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1a0670 con 0x556d30a8de40 2024-08-20T01:25:56.780 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.778+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 0x556d30ac3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb0a0a00640 1 -- 172.21.15.138:0/1573307666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 msgr2=0x556d30a91eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 0x556d30a91eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb0a0a00640 1 -- 172.21.15.138:0/1573307666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 msgr2=0x556d30ac30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 0x556d30ac30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb0a0a00640 1 -- 172.21.15.138:0/1573307666 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d30a86250 con 0x556d30a9a6f0 2024-08-20T01:25:56.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.779+0000 7fb09b400640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 0x556d30ac30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:25:56.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.780+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 0x556d30ac3610 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb094006010 tx=0x7fb094010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07904f5f08919e9 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.782 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.780+0000 7fb087e00640 1 -- 172.21.15.138:0/1573307666 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094004ce0 con 0x556d30a9a6f0 2024-08-20T01:25:56.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.781+0000 7fb087e00640 1 -- 172.21.15.138:0/1573307666 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09400ed80 con 0x556d30a9a6f0 2024-08-20T01:25:56.783 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.782+0000 7fb087e00640 1 -- 172.21.15.138:0/1573307666 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094005990 con 0x556d30a9a6f0 2024-08-20T01:25:56.784 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.783+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 msgr2=0x556d30ac3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.783+0000 7fb0a546f740 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 0x556d30ac3610 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb094006010 tx=0x7fb094010440 comp rx=0 tx=0).stop 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.784+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 shutdown_connections 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.784+0000 7fb0a546f740 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a9a6f0 0x556d30ac3610 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.784+0000 7fb0a546f740 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8e760 0x556d30ac30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.784+0000 7fb0a546f740 1 --2- 172.21.15.138:0/1573307666 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a8de40 0x556d30a91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.785 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.784+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 >> 172.21.15.138:0/1573307666 conn(0x556d30a8bc70 msgr2=0x556d30ab6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:56.786 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.785+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 shutdown_connections 2024-08-20T01:25:56.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.785+0000 7fb0a546f740 1 -- 172.21.15.138:0/1573307666 wait complete. 2024-08-20T01:25:56.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.786+0000 7fb0a546f740 1 -- start start 2024-08-20T01:25:56.787 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.786+0000 7fb0a546f740 1 Processor -- start 2024-08-20T01:25:56.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.788+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.788+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30aafdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.788+0000 7fb0a546f740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30b250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.789 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.788+0000 7fb09be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.788+0000 7fb09be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56914/0 (socket says 172.21.15.138:56914) 2024-08-20T01:25:56.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb09be00640 1 -- 172.21.15.138:0/3755247464 learned_addr learned my addr 172.21.15.138:0/3755247464 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:25:56.790 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb0a0a00640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30b250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb09b400640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30aafdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c074de0 con 0x556d30a8e760 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ad340 con 0x556d30a8de40 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb09c1ac5a0 con 0x556d30a9a6f0 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.789+0000 7fb0a546f740 10 client.0 _ll_register_callbacks cb 0x7fb088003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:25:56.791 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.790+0000 7fb09be00640 1 -- 172.21.15.138:0/3755247464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 msgr2=0x556d30b250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.790+0000 7fb09be00640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30b250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.790+0000 7fb09be00640 1 -- 172.21.15.138:0/3755247464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 msgr2=0x556d30aafdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:25:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.790+0000 7fb09be00640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30aafdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.790+0000 7fb09be00640 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30b255e0 con 0x556d30a8de40 2024-08-20T01:25:56.792 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.791+0000 7fb09b400640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30aafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.791+0000 7fb09be00640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004420 tx=0x7fb08c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da218e253973b794 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.791+0000 7fb085600640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.791+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c003d40 con 0x556d30a8de40 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.792+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb09400ed80 con 0x556d30a8de40 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.792+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7fb08c003ee0 con 0x556d30a8de40 2024-08-20T01:25:56.793 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.792+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fb08c038050 con 0x556d30a8de40 2024-08-20T01:25:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.792+0000 7fb0a546f740 10 client.27590 fetch_fsmap learned FSMap version 54 2024-08-20T01:25:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.792+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x556d30b26790 con 0x556d30a8de40 2024-08-20T01:25:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.793+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb09400ed80 con 0x556d30a8de40 2024-08-20T01:25:56.794 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.793+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fb08c02a420 con 0x556d30a8de40 2024-08-20T01:25:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.793+0000 7fb0a546f740 10 client.27590 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:25:56.795 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.793+0000 7fb0a546f740 10 client.27590 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.793+0000 7fb0a546f740 20 client.27590 populate_metadata read hostname 'smithi138' 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.794+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30b26fc0 0x556d30b293e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.794+0000 7fb0a546f740 4 client.27590 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}] 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.794+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.95, "await_for": 45.0}) -- 0x556d30a00ff0 con 0x556d30b26fc0 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.794+0000 7fb09b400640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30b26fc0 0x556d30b293e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.795+0000 7fb09b400640 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30b26fc0 0x556d30b293e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bafcbdcf6246987 server_cookie=95a8b0b69532dd4f in_seq=0 out_seq=0 2024-08-20T01:25:56.796 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:56.795+0000 7fb085600640 10 client.27590 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:57.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75019 15.47 MB/sec execute 114 sec latency 539.402 ms 2024-08-20T01:25:57.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.876+0000 7fb085600640 1 -- 172.21.15.138:0/3755247464 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556d30a00ff0 con 0x556d30b26fc0 2024-08-20T01:25:57.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.876+0000 7fb085600640 10 client.27590 handle_command_reply: tid=0 2024-08-20T01:25:57.877 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.876+0000 7fb0a546f740 1 client.27590 shutdown 2024-08-20T01:25:57.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.877+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 msgr2=0x556d30aaf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:57.878 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.877+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb08c004420 tx=0x7fb08c004730 comp rx=0 tx=0).stop 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 shutdown_connections 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x556d30b26fc0 0x556d30b293e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x556d30a9a6f0 0x556d30b250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x556d30a8e760 0x556d30aafdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 --2- 172.21.15.138:0/3755247464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d30a8de40 0x556d30aaf890 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 >> 172.21.15.138:0/3755247464 conn(0x556d30a8bc70 msgr2=0x556d30a8a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:57.879 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb085600640 0 client.27590 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:25:57.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 shutdown_connections 2024-08-20T01:25:57.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.878+0000 7fb0a546f740 1 -- 172.21.15.138:0/3755247464 wait complete. 2024-08-20T01:25:57.880 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.879+0000 7fb0a546f740 20 client.27590 trim_cache size 0 max 16384 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 54, 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 66, 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "e9ccf1ae": { 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 66, 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:25:57.881 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 144.9, 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "QUIESCED", 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:57.882 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 msgr2=0x7fb0800b63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 0x7fb0800b63d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x556d30a8cb50 tx=0x7fb08c005f90 comp rx=0 tx=0).stop 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 msgr2=0x7fb09c19fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:25:57.883 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb088005ab0 tx=0x7fb088003c80 comp rx=0 tx=0).stop 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 shutdown_connections 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb0800b3f10 0x7fb0800b63d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb09c1136a0 0x7fb09c1a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb09c077d20 0x7fb09c19fa80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 --2- 172.21.15.138:0/1262438487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb09c077400 0x7fb09c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.880+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 >> 172.21.15.138:0/1262438487 conn(0x7fb09c100720 msgr2=0x7fb09c102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.881+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 shutdown_connections 2024-08-20T01:25:57.884 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:25:57.881+0000 7fb0a2a00640 1 -- 172.21.15.138:0/1262438487 wait complete. 2024-08-20T01:25:58.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:25:58 smithi138 ceph-mon[18076]: pgmap v920: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-08-20T01:25:58.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 15.35 MB/sec execute 115 sec latency 1539.465 ms 2024-08-20T01:25:58.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:25:58 smithi167 ceph-mon[21965]: pgmap v920: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-08-20T01:25:58.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:25:58 smithi142 ceph-mon[21799]: pgmap v920: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 359 op/s 2024-08-20T01:25:59.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 15.21 MB/sec execute 116 sec latency 2539.548 ms 2024-08-20T01:26:00.182 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e9ccf1ae', quiesce took 8.8 seconds. Will release after: 16.15 2024-08-20T01:26:00.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:00 smithi138 ceph-mon[18076]: pgmap v921: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-08-20T01:26:00.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 15.08 MB/sec execute 117 sec latency 3539.628 ms 2024-08-20T01:26:00.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:00 smithi167 ceph-mon[21965]: pgmap v921: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-08-20T01:26:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:00 smithi142 ceph-mon[21799]: pgmap v921: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.9 MiB/s wr, 348 op/s 2024-08-20T01:26:01.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.96 MB/sec execute 118 sec latency 4539.678 ms 2024-08-20T01:26:02.235 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:02 smithi138 ceph-mon[18076]: pgmap v922: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-08-20T01:26:02.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.83 MB/sec execute 119 sec latency 5539.772 ms 2024-08-20T01:26:02.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:02 smithi167 ceph-mon[21965]: pgmap v922: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-08-20T01:26:02.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:02 smithi142 ceph-mon[21799]: pgmap v922: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.3 MiB/s wr, 448 op/s 2024-08-20T01:26:03.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.71 MB/sec execute 120 sec latency 6539.821 ms 2024-08-20T01:26:04.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:04 smithi138 ceph-mon[18076]: pgmap v923: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.8 MiB/s wr, 462 op/s 2024-08-20T01:26:04.349 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.59 MB/sec execute 121 sec latency 7539.880 ms 2024-08-20T01:26:04.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:04 smithi167 ceph-mon[21965]: pgmap v923: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.8 MiB/s wr, 462 op/s 2024-08-20T01:26:04.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:04 smithi142 ceph-mon[21799]: pgmap v923: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.8 MiB/s wr, 462 op/s 2024-08-20T01:26:05.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.47 MB/sec execute 122 sec latency 8539.934 ms 2024-08-20T01:26:06.126 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:06 smithi138 ceph-mon[18076]: pgmap v924: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-08-20T01:26:06.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.35 MB/sec execute 123 sec latency 9539.987 ms 2024-08-20T01:26:06.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:06 smithi167 ceph-mon[21965]: pgmap v924: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-08-20T01:26:06.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:06 smithi142 ceph-mon[21799]: pgmap v924: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.2 MiB/s wr, 464 op/s 2024-08-20T01:26:07.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.23 MB/sec execute 124 sec latency 10540.042 ms 2024-08-20T01:26:08.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.12 MB/sec execute 125 sec latency 11540.157 ms 2024-08-20T01:26:08.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:08 smithi167 ceph-mon[21965]: pgmap v925: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-08-20T01:26:08.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:08 smithi142 ceph-mon[21799]: pgmap v925: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-08-20T01:26:08.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:08 smithi138 ceph-mon[18076]: pgmap v925: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.4 MiB/s wr, 467 op/s 2024-08-20T01:26:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 -- 172.21.15.138:0/1285437978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 msgr2=0x7f12b0104fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 --2- 172.21.15.138:0/1285437978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b0104fd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009ca0 tx=0x7f12a402a8b0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 -- 172.21.15.138:0/1285437978 shutdown_connections 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 --2- 172.21.15.138:0/1285437978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010af70 0x7f12b0111770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 --2- 172.21.15.138:0/1285437978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010a5f0 0x7f12b010aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 --2- 172.21.15.138:0/1285437978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b0104fd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 -- 172.21.15.138:0/1285437978 >> 172.21.15.138:0/1285437978 conn(0x7f12b01005d0 msgr2=0x7f12b01029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 -- 172.21.15.138:0/1285437978 shutdown_connections 2024-08-20T01:26:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.284+0000 7f12b8000640 1 -- 172.21.15.138:0/1285437978 wait complete. 2024-08-20T01:26:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 Processor -- start 2024-08-20T01:26:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 -- start start 2024-08-20T01:26:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010a5f0 0x7f12b019b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.287 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 0x7f12b01a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0114c80 con 0x7f12b010af70 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0114b00 con 0x7f12b0104bf0 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.285+0000 7f12b8000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12b0114e00 con 0x7f12b010a5f0 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 0x7f12b01a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37836/0 (socket says 172.21.15.138:37836) 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 -- 172.21.15.138:0/2039996781 learned_addr learned my addr 172.21.15.138:0/2039996781 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b4c00640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010a5f0 0x7f12b019b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 -- 172.21.15.138:0/2039996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010a5f0 msgr2=0x7f12b019b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010a5f0 0x7f12b019b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 -- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 msgr2=0x7f12b01a09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 0x7f12b01a09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.288 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 -- 172.21.15.138:0/2039996781 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b01a11c0 con 0x7f12b0104bf0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b6000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 0x7f12b01a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.286+0000 7f12b5600640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005e30 tx=0x7f12a4003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.287+0000 7f12ae000640 1 -- 172.21.15.138:0/2039996781 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4004040 con 0x7f12b0104bf0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.287+0000 7f12ae000640 1 -- 172.21.15.138:0/2039996781 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a4040070 con 0x7f12b0104bf0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.287+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b01a1450 con 0x7f12b0104bf0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.287+0000 7f12ae000640 1 -- 172.21.15.138:0/2039996781 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a4031b30 con 0x7f12b0104bf0 2024-08-20T01:26:09.289 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.287+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b01a7b70 con 0x7f12b0104bf0 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 Processor -- start 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 -- start start 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f085258ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0852591e0 0x55f08525b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0851c2210 con 0x55f08524d250 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f0851b1710 con 0x55f0852591e0 2024-08-20T01:26:09.291 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12bb00b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f0851b5750 con 0x55f08524c930 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12ae000640 1 -- 172.21.15.138:0/2039996781 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f12a4031cf0 con 0x7f12b0104bf0 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12b6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12b6000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:35838/0 (socket says 172.21.15.138:35838) 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.290+0000 7f12b6000640 1 -- 172.21.15.138:0/2829238416 learned_addr learned my addr 172.21.15.138:0/2829238416 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b4c00640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0852591e0 0x55f08525b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b5600640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f085258ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 -- 172.21.15.138:0/2829238416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0852591e0 msgr2=0x55f08525b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0852591e0 0x55f08525b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 -- 172.21.15.138:0/2829238416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 msgr2=0x55f085258ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f085258ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 -- 172.21.15.138:0/2829238416 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f08517dde0 con 0x55f08524c930 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b5600640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f085258ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:26:09.292 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12b6000640 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010480 tx=0x7f12a8010940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=887292ff925993dc server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12a2600640 1 -- 172.21.15.138:0/2829238416 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8011760 con 0x55f08524c930 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.291+0000 7f12a2600640 1 -- 172.21.15.138:0/2829238416 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8011db0 con 0x55f08524c930 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12a2600640 1 -- 172.21.15.138:0/2829238416 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a801ce40 con 0x55f08524c930 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12ae000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 0x7f12980b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 -- 172.21.15.138:0/2829238416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 msgr2=0x55f08524cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12a8010480 tx=0x7f12a8010940 comp rx=0 tx=0).stop 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 -- 172.21.15.138:0/2829238416 shutdown_connections 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f0852591e0 0x55f08525b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f085258ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2829238416 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524c930 0x55f08524cd10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.297 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12bb00b740 1 -- 172.21.15.138:0/2829238416 >> 172.21.15.138:0/2829238416 conn(0x55f08524a760 msgr2=0x55f08524b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12ae000640 1 -- 172.21.15.138:0/2039996781 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f12a4105410 con 0x7f12b0104bf0 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.292+0000 7f12b5600640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 0x7f12980b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.293+0000 7f12bb00b740 1 -- 172.21.15.138:0/2829238416 shutdown_connections 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.293+0000 7f12bb00b740 1 -- 172.21.15.138:0/2829238416 wait complete. 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.293+0000 7f12b5600640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 0x7f12980b6280 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f08524fab0 tx=0x7f12a402a8b0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.293+0000 7f12bb00b740 1 -- start start 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.293+0000 7f12bb00b740 1 Processor -- start 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f085258a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524d250 0x55f085255e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55f0850eaab0 con 0x55f0852591e0 2024-08-20T01:26:09.298 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55f085158140 con 0x55f08524c930 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55f08525bec0 con 0x55f08524d250 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12bb00b740 10 client.0 _ll_register_callbacks cb 0x55f08516b5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50516/0 (socket says 172.21.15.138:50516) 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 -- 172.21.15.138:0/2957159738 learned_addr learned my addr 172.21.15.138:0/2957159738 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b4c00640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524d250 0x55f085255e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f085258a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 -- 172.21.15.138:0/2957159738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524d250 msgr2=0x55f085255e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524d250 0x55f085255e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 -- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 msgr2=0x55f085258a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.294+0000 7f12b6000640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f085258a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f12b6000640 1 -- 172.21.15.138:0/2957159738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f085159170 con 0x55f0852591e0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f12b6000640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12a801cb80 tx=0x7f12a8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29833d7328624a1f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f1296a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8005320 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f12a80054c0 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a8011db0 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f12a8034050 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f12bb00b740 10 client.17547 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.295+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f084f14790 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8011db0 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f12a8011a50 con 0x55f0852591e0 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 10 client.17547 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 10 client.17547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 20 client.17547 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f085268ce0 0x55f08526b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 4 client.17547 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:26:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0851c4160 con 0x55f085268ce0 2024-08-20T01:26:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.296+0000 7f12b4c00640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f085268ce0 0x55f08526b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.297+0000 7f12b4c00640 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f085268ce0 0x55f08526b0d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd84846a384bcf5 server_cookie=38c60b79c11f7ccd in_seq=0 out_seq=0 2024-08-20T01:26:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.297+0000 7f1296a00640 10 client.17547 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.297+0000 7f1296a00640 1 -- 172.21.15.138:0/2957159738 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0851c4160 con 0x55f085268ce0 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.297+0000 7f1296a00640 10 client.17547 handle_command_reply: tid=0 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 client.17547 shutdown 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 msgr2=0x55f085256410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f12a801cb80 tx=0x7f12a8013040 comp rx=0 tx=0).stop 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 shutdown_connections 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f085268ce0 0x55f08526b0d0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f0852591e0 0x55f085256410 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f08524d250 0x55f085255e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2957159738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f085258a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f1296a00640 0 client.17547 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 >> 172.21.15.138:0/2957159738 conn(0x55f08524a760 msgr2=0x55f085250300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 shutdown_connections 2024-08-20T01:26:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 1 -- 172.21.15.138:0/2957159738 wait complete. 2024-08-20T01:26:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.298+0000 7f12bb00b740 20 client.17547 trim_cache size 0 max 16384 2024-08-20T01:26:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 Processor -- start 2024-08-20T01:26:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 -- start start 2024-08-20T01:26:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.313 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f0852708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f085270de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b4620 con 0x55f08524d250 2024-08-20T01:26:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b1e90 con 0x55f08524c930 2024-08-20T01:26:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.313+0000 7f12bb00b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01afd00 con 0x55f0852591e0 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37852/0 (socket says 172.21.15.138:37852) 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 -- 172.21.15.138:0/2271214830 learned_addr learned my addr 172.21.15.138:0/2271214830 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b4c00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f0852708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f085270de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 -- 172.21.15.138:0/2271214830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 msgr2=0x55f085270de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f085270de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 -- 172.21.15.138:0/2271214830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 msgr2=0x55f0852708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f0852708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 -- 172.21.15.138:0/2271214830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f085258dc0 con 0x55f08524c930 2024-08-20T01:26:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.314+0000 7f12b6000640 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bfd0 tx=0x7f12a8011560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a52826ad496d095 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12a1c00640 1 -- 172.21.15.138:0/2271214830 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800bc00 con 0x55f08524c930 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12a1c00640 1 -- 172.21.15.138:0/2271214830 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8004cd0 con 0x55f08524c930 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12a1c00640 1 -- 172.21.15.138:0/2271214830 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a801e110 con 0x55f08524c930 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 -- 172.21.15.138:0/2271214830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 msgr2=0x55f08524bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bfd0 tx=0x7f12a8011560 comp rx=0 tx=0).stop 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 -- 172.21.15.138:0/2271214830 shutdown_connections 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f085270de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524d250 0x55f0852708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 --2- 172.21.15.138:0/2271214830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524c930 0x55f08524bec0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 -- 172.21.15.138:0/2271214830 >> 172.21.15.138:0/2271214830 conn(0x55f08524a760 msgr2=0x55f08524b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 -- 172.21.15.138:0/2271214830 shutdown_connections 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.315+0000 7f12bb00b740 1 -- 172.21.15.138:0/2271214830 wait complete. 2024-08-20T01:26:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 -- start start 2024-08-20T01:26:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 Processor -- start 2024-08-20T01:26:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 0x55f08525ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f08525f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.320 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0350 con 0x55f08524c930 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b17c0 con 0x55f08524d250 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12b01b0a20 con 0x55f0852591e0 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.316+0000 7f12bb00b740 10 client.0 _ll_register_callbacks cb 0x7f129c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50534/0 (socket says 172.21.15.138:50534) 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 -- 172.21.15.138:0/1900688341 learned_addr learned my addr 172.21.15.138:0/1900688341 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f08525f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b4c00640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 0x55f08525ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 -- 172.21.15.138:0/1900688341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 msgr2=0x55f08525f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f08525f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 -- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 msgr2=0x55f08525ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 0x55f08525ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.321 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 -- 172.21.15.138:0/1900688341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f084f8d620 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b4c00640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 0x55f08525ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f12b6000640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bb10 tx=0x7f12a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7b377392fec36d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f1296a00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a803e020 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.317+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9453+0+0 (secure 0 0 0) 0x7f12a801c480 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a8004cd0 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f12a8049050 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 10 client.17553 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55f0852d3660 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8004cd0 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7f12a8011c80 con 0x55f08524c930 2024-08-20T01:26:09.322 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 10 client.17553 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 10 client.17553 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 20 client.17553 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f0852d3c70 0x55f0852d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 4 client.17553 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.318+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0851bfc00 con 0x55f0852d3c70 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.319+0000 7f12b4c00640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f0852d3c70 0x55f0852d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.319+0000 7f12b4c00640 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f0852d3c70 0x55f0852d6090 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2d322165ba5ffc server_cookie=af695fa9206be2f8 in_seq=0 out_seq=0 2024-08-20T01:26:09.323 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.319+0000 7f1296a00640 10 client.17553 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:09.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 14.01 MB/sec execute 126 sec latency 12540.235 ms 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f1296a00640 1 -- 172.21.15.138:0/1900688341 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f0851bfc00 con 0x55f0852d3c70 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f1296a00640 10 client.17553 handle_command_reply: tid=0 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 client.17553 shutdown 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 msgr2=0x55f085261760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f12a800bb10 tx=0x7f12a8004600 comp rx=0 tx=0).stop 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 shutdown_connections 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55f0852d3c70 0x55f0852d6090 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55f0852591e0 0x55f08525f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f08524d250 0x55f08525ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 --2- 172.21.15.138:0/1900688341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55f08524c930 0x55f085261760 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 >> 172.21.15.138:0/1900688341 conn(0x55f08524a760 msgr2=0x55f085249760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.389 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.388+0000 7f1296a00640 0 client.17553 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 shutdown_connections 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12bb00b740 1 -- 172.21.15.138:0/1900688341 wait complete. 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12bb00b740 20 client.17553 trim_cache size 0 max 16384 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:26:09.390 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 msgr2=0x7f12980b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 0x7f12980b6280 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55f08524fab0 tx=0x7f12a402a8b0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 msgr2=0x7f12b019b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005e30 tx=0x7f12a4003970 comp rx=0 tx=0).stop 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 shutdown_connections 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12980b3dc0 0x7f12980b6280 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.391 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12b010af70 0x7f12b01a09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f12b010a5f0 0x7f12b019b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 --2- 172.21.15.138:0/2039996781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12b0104bf0 0x7f12b019b120 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.389+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 >> 172.21.15.138:0/2039996781 conn(0x7f12b01005d0 msgr2=0x7f12b01029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.390+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 shutdown_connections 2024-08-20T01:26:09.392 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:09.390+0000 7f12b8000640 1 -- 172.21.15.138:0/2039996781 wait complete. 2024-08-20T01:26:10.350 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.90 MB/sec execute 127 sec latency 13540.303 ms 2024-08-20T01:26:10.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:10 smithi167 ceph-mon[21965]: pgmap v926: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-08-20T01:26:10.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:10 smithi142 ceph-mon[21799]: pgmap v926: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-08-20T01:26:10.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:10 smithi138 ceph-mon[18076]: pgmap v926: 129 pgs: 129 active+clean; 1.0 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 276 op/s 2024-08-20T01:26:11.352 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.79 MB/sec execute 128 sec latency 14542.607 ms 2024-08-20T01:26:11.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:11 smithi167 ceph-mon[21965]: osdmap e112: 12 total, 12 up, 12 in 2024-08-20T01:26:11.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:11 smithi142 ceph-mon[21799]: osdmap e112: 12 total, 12 up, 12 in 2024-08-20T01:26:11.544 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:11 smithi138 ceph-mon[18076]: osdmap e112: 12 total, 12 up, 12 in 2024-08-20T01:26:12.352 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.68 MB/sec execute 129 sec latency 15542.663 ms 2024-08-20T01:26:12.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:12 smithi167 ceph-mon[21965]: pgmap v928: 129 pgs: 6 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 102 active+clean; 996 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.3 MiB/s wr, 233 op/s 2024-08-20T01:26:12.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:12 smithi142 ceph-mon[21799]: pgmap v928: 129 pgs: 6 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 102 active+clean; 996 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.3 MiB/s wr, 233 op/s 2024-08-20T01:26:12.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:12 smithi138 ceph-mon[18076]: pgmap v928: 129 pgs: 6 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 102 active+clean; 996 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 875 KiB/s rd, 5.3 MiB/s wr, 233 op/s 2024-08-20T01:26:13.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.58 MB/sec execute 130 sec latency 16542.778 ms 2024-08-20T01:26:14.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.47 MB/sec execute 131 sec latency 17542.855 ms 2024-08-20T01:26:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:14 smithi167 ceph-mon[21965]: pgmap v929: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-08-20T01:26:14.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:14 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:26:14.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:14 smithi142 ceph-mon[21799]: pgmap v929: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-08-20T01:26:14.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:14 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:26:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:14 smithi138 ceph-mon[18076]: pgmap v929: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 4.7 MiB/s wr, 212 op/s 2024-08-20T01:26:14.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:14 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:26:15.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.37 MB/sec execute 132 sec latency 18542.941 ms 2024-08-20T01:26:15.444 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:26:15.444 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:26:15.444 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:15 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:26:15.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:26:15.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:26:15.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:15 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:26:15.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:26:15.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:26:15.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:15 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:26:16.333 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:26:16.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.27 MB/sec execute 133 sec latency 19542.984 ms 2024-08-20T01:26:16.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:16 smithi167 ceph-mon[21965]: pgmap v930: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-08-20T01:26:16.722 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:26:16.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:16 smithi142 ceph-mon[21799]: pgmap v930: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-08-20T01:26:16.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:16 smithi138 ceph-mon[18076]: pgmap v930: 129 pgs: 10 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 86 active+clean; 971 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 3.8 MiB/s wr, 191 op/s 2024-08-20T01:26:17.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.17 MB/sec execute 134 sec latency 20543.029 ms 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 -- 172.21.15.138:0/2091222442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 msgr2=0x7f8254113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 --2- 172.21.15.138:0/2091222442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f8254113170 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8240009900 tx=0x7f824002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 -- 172.21.15.138:0/2091222442 shutdown_connections 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 --2- 172.21.15.138:0/2091222442 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 0x7f8254115aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 --2- 172.21.15.138:0/2091222442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f8254113170 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.422 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 --2- 172.21.15.138:0/2091222442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f82540777e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 -- 172.21.15.138:0/2091222442 >> 172.21.15.138:0/2091222442 conn(0x7f82541006b0 msgr2=0x7f8254102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:17.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.421+0000 7f8258a00640 1 -- 172.21.15.138:0/2091222442 shutdown_connections 2024-08-20T01:26:17.423 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.422+0000 7f8258a00640 1 -- 172.21.15.138:0/2091222442 wait complete. 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.422+0000 7f8258a00640 1 Processor -- start 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 -- start start 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 0x7f82541a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f82541190f0 con 0x7f8254077d20 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f8254118f70 con 0x7f8254077400 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8258a00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f8254119270 con 0x7f82541136b0 2024-08-20T01:26:17.424 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8251e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8251400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8251e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:37884/0 (socket says 172.21.15.138:37884) 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8251e00640 1 -- 172.21.15.138:0/1149466937 learned_addr learned my addr 172.21.15.138:0/1149466937 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.423+0000 7f8251400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50552/0 (socket says 172.21.15.138:50552) 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8252800640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 0x7f82541a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 -- 172.21.15.138:0/1149466937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 msgr2=0x7f82541a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 0x7f82541a4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 -- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 msgr2=0x7f825419fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 -- 172.21.15.138:0/1149466937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541a5690 con 0x7f8254077400 2024-08-20T01:26:17.425 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251400640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f8251e00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f824400d950 tx=0x7f824400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.424+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244017030 con 0x7f8254077400 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.425+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541a5980 con 0x7f8254077400 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.425+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8244004020 con 0x7f8254077400 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.425+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82440047a0 con 0x7f8254077400 2024-08-20T01:26:17.426 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.425+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541ac210 con 0x7f8254077400 2024-08-20T01:26:17.427 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.426+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82540777e0 con 0x7f8254077400 2024-08-20T01:26:17.428 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.428+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8244009a30 con 0x7f8254077400 2024-08-20T01:26:17.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.429+0000 7f824aa00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 0x7f82380b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:17.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.429+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9541+0+0 (secure 0 0 0) 0x7f82440e3e10 con 0x7f8254077400 2024-08-20T01:26:17.429 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.429+0000 7f8251400640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 0x7f82380b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:17.430 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.430+0000 7f8251400640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 0x7f82380b6280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8240005ab0 tx=0x7f8240005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:17.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.433+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f82440e7030 con 0x7f8254077400 2024-08-20T01:26:17.651 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.650+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82541ac470 con 0x7f8254077400 2024-08-20T01:26:17.652 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.652+0000 7f824aa00640 1 -- 172.21.15.138:0/1149466937 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+11936 (secure 0 0 0) 0x7f82440aacc0 con 0x7f8254077400 2024-08-20T01:26:17.652 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:26:17.652 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":54,"btime":"2024-08-20T01:25:19:989006+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":54,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:25:19.256053+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24704,24743,24706]},"id":1}]} 2024-08-20T01:26:17.652 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 54 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.654+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 msgr2=0x7f82380b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.654+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 0x7f82380b6280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8240005ab0 tx=0x7f8240005a20 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.654+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 msgr2=0x7f825419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.654+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f824400d950 tx=0x7f824400de10 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 shutdown_connections 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f82380b3dc0 0x7f82380b6280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f82541136b0 0x7f82541a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8254077d20 0x7f825419fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 --2- 172.21.15.138:0/1149466937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8254077400 0x7f825419f5e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:17.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 >> 172.21.15.138:0/1149466937 conn(0x7f82541006b0 msgr2=0x7f8254102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:17.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 shutdown_connections 2024-08-20T01:26:17.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:17.655+0000 7f8258a00640 1 -- 172.21.15.138:0/1149466937 wait complete. 2024-08-20T01:26:18.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 13.07 MB/sec execute 135 sec latency 21543.131 ms 2024-08-20T01:26:18.529 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --set-id e9ccf1ae --release --await' 2024-08-20T01:26:18.529 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --set-id e9ccf1ae --release --await 2024-08-20T01:26:18.714 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:18 smithi167 ceph-mon[21965]: pgmap v931: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-08-20T01:26:18.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:18 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1149466937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:26:18.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:18 smithi142 ceph-mon[21799]: pgmap v931: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-08-20T01:26:18.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:18 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1149466937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:26:18.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:18 smithi138 ceph-mon[18076]: pgmap v931: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 81 op/s 2024-08-20T01:26:18.792 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:18 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1149466937' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:26:18.941 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:26:19.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75074 12.98 MB/sec execute 136 sec latency 22543.197 ms 2024-08-20T01:26:19.607 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 -- 172.21.15.138:0/2965598262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0113810 msgr2=0x7fa2a0115c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 --2- 172.21.15.138:0/2965598262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0113810 0x7fa2a0115c00 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa298009be0 tx=0x7fa29802a6c0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 -- 172.21.15.138:0/2965598262 shutdown_connections 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 --2- 172.21.15.138:0/2965598262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0113810 0x7fa2a0115c00 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 --2- 172.21.15.138:0/2965598262 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0075b90 0x7fa2a01132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 --2- 172.21.15.138:0/2965598262 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a0075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 -- 172.21.15.138:0/2965598262 >> 172.21.15.138:0/2965598262 conn(0x7fa2a00fe500 msgr2=0x7fa2a0100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 -- 172.21.15.138:0/2965598262 shutdown_connections 2024-08-20T01:26:19.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.607+0000 7fa2a5200640 1 -- 172.21.15.138:0/2965598262 wait complete. 2024-08-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 Processor -- start 2024-08-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 -- start start 2024-08-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0113810 0x7fa2a01a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0119070 con 0x7fa2a0113810 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0118ef0 con 0x7fa2a0075b90 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.608+0000 7fa2a5200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01191f0 con 0x7fa2a0075270 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa29de00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56680/0 (socket says 172.21.15.138:56680) 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 -- 172.21.15.138:0/1849690151 learned_addr learned my addr 172.21.15.138:0/1849690151 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa29e800640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0113810 0x7fa2a01a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 -- 172.21.15.138:0/1849690151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 msgr2=0x7fa2a019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 -- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0113810 msgr2=0x7fa2a01a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0113810 0x7fa2a01a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.610 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 -- 172.21.15.138:0/1849690151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01a56b0 con 0x7fa2a0075b90 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa29de00640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.609+0000 7fa295400640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9c0 tx=0x7fa28800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.610+0000 7fa297400640 1 -- 172.21.15.138:0/1849690151 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa28800cde0 con 0x7fa2a0075b90 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.610+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01a59a0 con 0x7fa2a0075b90 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.610+0000 7fa297400640 1 -- 172.21.15.138:0/1849690151 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2880035f0 con 0x7fa2a0075b90 2024-08-20T01:26:19.611 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.610+0000 7fa297400640 1 -- 172.21.15.138:0/1849690151 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa288010690 con 0x7fa2a0075b90 2024-08-20T01:26:19.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.610+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01ac060 con 0x7fa2a0075b90 2024-08-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.613+0000 7fa2a7dd4740 1 Processor -- start 2024-08-20T01:26:19.613 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.613+0000 7fa2a7dd4740 1 -- start start 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa297400640 1 -- 172.21.15.138:0/1849690151 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa2880037a0 con 0x7fa2a0075b90 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc1610 0x5619c6ccd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 0x5619c6ccf990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5619c6c62670 con 0x5619c6cc0cf0 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5619c6c36d70 con 0x5619c6ccd5a0 2024-08-20T01:26:19.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5619c6c26270 con 0x5619c6cc1610 2024-08-20T01:26:19.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa295400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 0x5619c6ccf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43458/0 (socket says 172.21.15.138:43458) 2024-08-20T01:26:19.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 -- 172.21.15.138:0/3431418572 learned_addr learned my addr 172.21.15.138:0/3431418572 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:19.615 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29de00640 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc1610 0x5619c6ccd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 -- 172.21.15.138:0/3431418572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc1610 msgr2=0x5619c6ccd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc1610 0x5619c6ccd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 -- 172.21.15.138:0/3431418572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 msgr2=0x5619c6ccf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 0x5619c6ccf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa29e800640 1 -- 172.21.15.138:0/3431418572 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c6bc2870 con 0x5619c6cc0cf0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.614+0000 7fa295400640 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 0x5619c6ccf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.615+0000 7fa29e800640 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa2a006c040 tx=0x7fa298005ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0360c5b37f578f4 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.615+0000 7fa28f400640 1 -- 172.21.15.138:0/3431418572 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298041030 con 0x5619c6cc0cf0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.615+0000 7fa28f400640 1 -- 172.21.15.138:0/3431418572 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298003e20 con 0x5619c6cc0cf0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.615+0000 7fa28f400640 1 -- 172.21.15.138:0/3431418572 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298036040 con 0x5619c6cc0cf0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.615+0000 7fa297400640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa297400640 1 -- 172.21.15.138:0/1849690151 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9541+0+0 (secure 0 0 0) 0x7fa2880e46e0 con 0x7fa2a0075b90 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa29de00640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/3431418572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 msgr2=0x5619c6cc10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fa2a006c040 tx=0x7fa298005ec0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/3431418572 shutdown_connections 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6ccd5a0 0x5619c6ccf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc1610 0x5619c6ccd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/3431418572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cc10d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/3431418572 >> 172.21.15.138:0/3431418572 conn(0x5619c6cbeb20 msgr2=0x5619c6cbfcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/3431418572 shutdown_connections 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.616+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/3431418572 wait complete. 2024-08-20T01:26:19.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.617+0000 7fa29de00640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 0x7fa2840b6200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5619c6cbfa00 tx=0x7fa290005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.617+0000 7fa2a7dd4740 1 -- start start 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.617+0000 7fa2a7dd4740 1 Processor -- start 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6cca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.620 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5619c6bacff0 con 0x5619c6cc0cf0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5619c6c2a2b0 con 0x5619c6cc1610 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5619c6a88bb0 con 0x5619c6ccd5a0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa2a7dd4740 10 client.0 _ll_register_callbacks cb 0x5619c6931d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa29de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa29de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43468/0 (socket says 172.21.15.138:43468) 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa29de00640 1 -- 172.21.15.138:0/2894810082 learned_addr learned my addr 172.21.15.138:0/2894810082 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa29e800640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6cca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 -- 172.21.15.138:0/2894810082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 msgr2=0x5619c6cca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6cca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 -- 172.21.15.138:0/2894810082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 msgr2=0x5619c6cccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.618+0000 7fa295400640 1 -- 172.21.15.138:0/2894810082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c6985b00 con 0x5619c6cc1610 2024-08-20T01:26:19.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa295400640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cca1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa2880e19d0 tx=0x7fa2880a6d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a659e6ca17f035 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa283e00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2880aa600 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa298003e20 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9541+0+0 (secure 0 0 0) 0x7fa2880b3cc0 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa2880aa9a0 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa2a7dd4740 10 client.27626 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5619c69fc960 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.619+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2880e4070 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa2880e4220 con 0x5619c6cc1610 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 10 client.27626 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 10 client.27626 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:26:19.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 20 client.27626 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:19.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6cdcb70 0x5619c6cdeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 4 client.27626 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:26:19.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619c6c38cc0 con 0x5619c6cdcb70 2024-08-20T01:26:19.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.620+0000 7fa29de00640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6cdcb70 0x5619c6cdeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.621+0000 7fa29de00640 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6cdcb70 0x5619c6cdeff0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc745df405c51b9 server_cookie=85d012e927d50727 in_seq=0 out_seq=0 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.621+0000 7fa283e00640 10 client.27626 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.621+0000 7fa283e00640 1 -- 172.21.15.138:0/2894810082 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619c6c38cc0 con 0x5619c6cdcb70 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.621+0000 7fa283e00640 10 client.27626 handle_command_reply: tid=0 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.621+0000 7fa2a7dd4740 1 client.27626 shutdown 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 msgr2=0x5619c6cca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cca1c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa2880e19d0 tx=0x7fa2880a6d50 comp rx=0 tx=0).stop 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 shutdown_connections 2024-08-20T01:26:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6cdcb70 0x5619c6cdeff0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6cca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cca1c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2894810082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa283e00640 0 client.27626 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 >> 172.21.15.138:0/2894810082 conn(0x5619c6cbeb20 msgr2=0x5619c6cc4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 shutdown_connections 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2894810082 wait complete. 2024-08-20T01:26:19.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.622+0000 7fa2a7dd4740 20 client.27626 trim_cache size 0 max 16384 2024-08-20T01:26:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.639+0000 7fa2a7dd4740 1 Processor -- start 2024-08-20T01:26:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 -- start start 2024-08-20T01:26:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.640 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6ccbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b8c20 con 0x5619c6ccd5a0 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b6490 con 0x5619c6cc1610 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b4300 con 0x5619c6cc0cf0 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa29e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43474/0 (socket says 172.21.15.138:43474) 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa29e800640 1 -- 172.21.15.138:0/2559435260 learned_addr learned my addr 172.21.15.138:0/2559435260 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa29de00640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.641 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.640+0000 7fa295400640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6ccbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 -- 172.21.15.138:0/2559435260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 msgr2=0x5619c6cd9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.642 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 -- 172.21.15.138:0/2559435260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 msgr2=0x5619c6ccbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6ccbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 -- 172.21.15.138:0/2559435260 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619c6cdf670 con 0x5619c6ccd5a0 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29de00640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa29e800640 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa298009be0 tx=0x7fa298030d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1c5afde2c61c29 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa28ea00640 1 -- 172.21.15.138:0/2559435260 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29804f070 con 0x5619c6ccd5a0 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa28ea00640 1 -- 172.21.15.138:0/2559435260 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298030970 con 0x5619c6ccd5a0 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.641+0000 7fa28ea00640 1 -- 172.21.15.138:0/2559435260 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa298036050 con 0x5619c6ccd5a0 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2559435260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 msgr2=0x5619c6ce4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa298009be0 tx=0x7fa298030d90 comp rx=0 tx=0).stop 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2559435260 shutdown_connections 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6ccd5a0 0x5619c6ce4900 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6ccbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/2559435260 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2559435260 >> 172.21.15.138:0/2559435260 conn(0x5619c6cbeb20 msgr2=0x5619c6ccb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.643 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2559435260 shutdown_connections 2024-08-20T01:26:19.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.642+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/2559435260 wait complete. 2024-08-20T01:26:19.644 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 -- start start 2024-08-20T01:26:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 Processor -- start 2024-08-20T01:26:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cd2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6d40980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.645 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b4950 con 0x5619c6cc0cf0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b5dc0 con 0x5619c6cc1610 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a01b5020 con 0x5619c6ccd5a0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa2a7dd4740 10 client.0 _ll_register_callbacks cb 0x7fa288003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa29de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.643+0000 7fa29de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43482/0 (socket says 172.21.15.138:43482) 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 -- 172.21.15.138:0/1484242201 learned_addr learned my addr 172.21.15.138:0/1484242201 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29e800640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6d40980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa295400640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cd2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 -- 172.21.15.138:0/1484242201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 msgr2=0x5619c6d40980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6d40980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 -- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 msgr2=0x5619c6cd2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cd2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 -- 172.21.15.138:0/1484242201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619c6cbf5a0 con 0x5619c6cc0cf0 2024-08-20T01:26:19.646 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa29de00640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290009460 tx=0x7fa2900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2b0768ed3b0eb5a server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa283e00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.644+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29002e030 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa298030970 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9541+0+0 (secure 0 0 0) 0x7fa290018030 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa290004020 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 10 client.17577 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x5619c6d48350 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa298030970 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa290005140 con 0x5619c6cc0cf0 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 10 client.17577 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 10 client.17577 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 20 client.17577 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6d48aa0 0x5619c6d4aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 4 client.17577 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e9ccf1ae", "release": true, "await": true}] 2024-08-20T01:26:19.647 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.645+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e9ccf1ae", "release": true, "await": true}) -- 0x5619c6c34760 con 0x5619c6d48aa0 2024-08-20T01:26:19.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.646+0000 7fa295400640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6d48aa0 0x5619c6d4aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:19.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.646+0000 7fa295400640 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6d48aa0 0x5619c6d4aec0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f6c6b357034b608 server_cookie=164282d7782e7147 in_seq=0 out_seq=0 2024-08-20T01:26:19.648 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.646+0000 7fa283e00640 10 client.17577 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa283e00640 1 -- 172.21.15.138:0/1484242201 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5619c6c34760 con 0x5619c6d48aa0 2024-08-20T01:26:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa283e00640 10 client.17577 handle_command_reply: tid=0 2024-08-20T01:26:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 client.17577 shutdown 2024-08-20T01:26:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 msgr2=0x5619c6cd1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.687 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa290009460 tx=0x7fa2900046c0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 shutdown_connections 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5619c6d48aa0 0x5619c6d4aec0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5619c6ccd5a0 0x5619c6d40980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619c6cc1610 0x5619c6cd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 --2- 172.21.15.138:0/1484242201 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5619c6cc0cf0 0x5619c6cd1b80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa283e00640 0 client.17577 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.687+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 >> 172.21.15.138:0/1484242201 conn(0x5619c6cbeb20 msgr2=0x5619c6ccc010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 shutdown_connections 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a7dd4740 1 -- 172.21.15.138:0/1484242201 wait complete. 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a7dd4740 20 client.17577 trim_cache size 0 max 16384 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 54, 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 68, 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": { 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "e9ccf1ae": { 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "version": 68, 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "age_ref": 0.0, 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:26:19.689 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "timeout": 90.0, 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "expiration": 144.9, 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "members": { 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "file:/": { 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "excluded": false, 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "state": { 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RELEASED", 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: "age": 0.0 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:26:19.690 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 msgr2=0x7fa2840b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 0x7fa2840b6200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x5619c6cbfa00 tx=0x7fa290005f90 comp rx=0 tx=0).stop 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 msgr2=0x7fa2a019fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:19.691 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.688+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa28800e9c0 tx=0x7fa28800ee80 comp rx=0 tx=0).stop 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 shutdown_connections 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa2840b3d40 0x7fa2840b6200 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2a0113810 0x7fa2a01a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2a0075b90 0x7fa2a019fb40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 --2- 172.21.15.138:0/1849690151 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa2a0075270 0x7fa2a019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 >> 172.21.15.138:0/1849690151 conn(0x7fa2a00fe500 msgr2=0x7fa2a00ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 shutdown_connections 2024-08-20T01:26:19.692 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:26:19.689+0000 7fa2a5200640 1 -- 172.21.15.138:0/1849690151 wait complete. 2024-08-20T01:26:20.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 75502 13.05 MB/sec execute 137 sec latency 23004.298 ms 2024-08-20T01:26:20.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:20 smithi138 ceph-mon[18076]: pgmap v932: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:26:20.715 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:20 smithi167 ceph-mon[21965]: pgmap v932: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:26:20.743 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:20 smithi142 ceph-mon[21799]: pgmap v932: 129 pgs: 129 active+clean; 970 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.1 MiB/s wr, 73 op/s 2024-08-20T01:26:20.853 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e9ccf1ae', total seconds elapsed: 29.4 2024-08-20T01:26:21.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 76116 13.21 MB/sec execute 138 sec latency 164.857 ms 2024-08-20T01:26:22.073 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:21 smithi138 ceph-mon[18076]: osdmap e113: 12 total, 12 up, 12 in 2024-08-20T01:26:22.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:21 smithi167 ceph-mon[21965]: osdmap e113: 12 total, 12 up, 12 in 2024-08-20T01:26:22.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:21 smithi142 ceph-mon[21799]: osdmap e113: 12 total, 12 up, 12 in 2024-08-20T01:26:22.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 76995 13.40 MB/sec execute 139 sec latency 61.199 ms 2024-08-20T01:26:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 -- 172.21.15.138:0/4290740408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48810a610 msgr2=0x7fe48810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.550 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 --2- 172.21.15.138:0/4290740408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48810a610 0x7fe48810aa50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe474009900 tx=0x7fe47402a3f0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 -- 172.21.15.138:0/4290740408 shutdown_connections 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 --2- 172.21.15.138:0/4290740408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 0x7fe488111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 --2- 172.21.15.138:0/4290740408 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe48810a610 0x7fe48810aa50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 --2- 172.21.15.138:0/4290740408 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe488104c10 0x7fe488104ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 -- 172.21.15.138:0/4290740408 >> 172.21.15.138:0/4290740408 conn(0x7fe4881006b0 msgr2=0x7fe488102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 -- 172.21.15.138:0/4290740408 shutdown_connections 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.550+0000 7fe48e400640 1 -- 172.21.15.138:0/4290740408 wait complete. 2024-08-20T01:26:22.551 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.551+0000 7fe48e400640 1 Processor -- start 2024-08-20T01:26:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.552+0000 7fe48e400640 1 -- start start 2024-08-20T01:26:22.552 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 0x7fe48819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 0x7fe4881a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe488114c30 con 0x7fe48810af90 2024-08-20T01:26:22.553 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe488114ab0 con 0x7fe488104c10 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe48e400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe488114db0 con 0x7fe48810a610 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 0x7fe4881a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56718/0 (socket says 172.21.15.138:56718) 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 -- 172.21.15.138:0/853078756 learned_addr learned my addr 172.21.15.138:0/853078756 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe486a00640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 0x7fe48819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 -- 172.21.15.138:0/853078756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 msgr2=0x7fe48819b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 0x7fe48819b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 -- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 msgr2=0x7fe4881a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 0x7fe4881a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.554 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.553+0000 7fe487400640 1 -- 172.21.15.138:0/853078756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4881a12b0 con 0x7fe488104c10 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe486a00640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 0x7fe48819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe487400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d950 tx=0x7fe47800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe47fe00640 1 -- 172.21.15.138:0/853078756 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe478017070 con 0x7fe488104c10 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe47fe00640 1 -- 172.21.15.138:0/853078756 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe478009c30 con 0x7fe488104c10 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe47fe00640 1 -- 172.21.15.138:0/853078756 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47800f620 con 0x7fe488104c10 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4881a15a0 con 0x7fe488104c10 2024-08-20T01:26:22.555 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.554+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4881a7c60 con 0x7fe488104c10 2024-08-20T01:26:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.558+0000 7fe491482740 1 Processor -- start 2024-08-20T01:26:22.558 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.558+0000 7fe491482740 1 -- start start 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe47fe00640 1 -- 172.21.15.138:0/853078756 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fe47800f7c0 con 0x7fe488104c10 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaed1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaed4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ecae3b210 con 0x563ecaed1e80 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ecae2a710 con 0x563ecaec5ef0 2024-08-20T01:26:22.559 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe491482740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ecae2e750 con 0x563ecaec55d0 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe487400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaed1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.559+0000 7fe487e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39684/0 (socket says 172.21.15.138:39684) 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 -- 172.21.15.138:0/2430138363 learned_addr learned my addr 172.21.15.138:0/2430138363 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe486a00640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaed4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.560 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 -- 172.21.15.138:0/2430138363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 msgr2=0x563ecaed1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaed1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 -- 172.21.15.138:0/2430138363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 msgr2=0x563ecaed4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaed4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487e00640 1 -- 172.21.15.138:0/2430138363 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecadf6de0 con 0x563ecaec55d0 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe486a00640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaed4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.560+0000 7fe487400640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaed1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:26:22.561 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.561+0000 7fe487e00640 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe480007ec0 tx=0x7fe48000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae428c1d6cf258f7 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.561+0000 7fe47cc00640 1 -- 172.21.15.138:0/2430138363 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480010840 con 0x563ecaec55d0 2024-08-20T01:26:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.561+0000 7fe47cc00640 1 -- 172.21.15.138:0/2430138363 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4800183f0 con 0x563ecaec55d0 2024-08-20T01:26:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.561+0000 7fe47cc00640 1 -- 172.21.15.138:0/2430138363 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe480017600 con 0x563ecaec55d0 2024-08-20T01:26:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.562+0000 7fe47fe00640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 0x7fe46c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.562 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.562+0000 7fe491482740 1 -- 172.21.15.138:0/2430138363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 msgr2=0x563ecaec59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.562+0000 7fe491482740 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe480007ec0 tx=0x7fe48000f790 comp rx=0 tx=0).stop 2024-08-20T01:26:22.563 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.563+0000 7fe47fe00640 1 -- 172.21.15.138:0/853078756 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fe4780e4070 con 0x7fe488104c10 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe487400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 0x7fe46c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe491482740 1 -- 172.21.15.138:0/2430138363 shutdown_connections 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe491482740 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaed4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe491482740 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaed1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe491482740 1 --2- 172.21.15.138:0/2430138363 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec55d0 0x563ecaec59b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.564 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe491482740 1 -- 172.21.15.138:0/2430138363 >> 172.21.15.138:0/2430138363 conn(0x563ecaec33b0 msgr2=0x563ecaec45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.564+0000 7fe487400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 0x7fe46c0b6200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x563ecaec8750 tx=0x7fe478005880 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.565+0000 7fe491482740 1 -- 172.21.15.138:0/2430138363 shutdown_connections 2024-08-20T01:26:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.565+0000 7fe491482740 1 -- 172.21.15.138:0/2430138363 wait complete. 2024-08-20T01:26:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.565+0000 7fe491482740 1 -- start start 2024-08-20T01:26:22.565 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 Processor -- start 2024-08-20T01:26:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaecedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.566 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaecf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x563ecad63ab0 con 0x563ecaed1e80 2024-08-20T01:26:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x563ecadd1140 con 0x563ecaec55d0 2024-08-20T01:26:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x563ecaed4b60 con 0x563ecaec5ef0 2024-08-20T01:26:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.566+0000 7fe491482740 10 client.0 _ll_register_callbacks cb 0x563ecade45d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:22.567 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39696/0 (socket says 172.21.15.138:39696) 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 -- 172.21.15.138:0/2618842739 learned_addr learned my addr 172.21.15.138:0/2618842739 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe487e00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaecedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe487e00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaecf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 -- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 msgr2=0x563ecaecedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaecedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 -- 172.21.15.138:0/2618842739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 msgr2=0x563ecaecf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.567+0000 7fe486a00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaecf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.568 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.568+0000 7fe486a00640 1 -- 172.21.15.138:0/2618842739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecadd2170 con 0x563ecaec5ef0 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.568+0000 7fe487e00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaecf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.568+0000 7fe486a00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe474005ab0 tx=0x7fe4740052f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec365ee01a5b91 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.568+0000 7fe471600640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035030 con 0x563ecaec5ef0 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fe4740048c0 con 0x563ecaec5ef0 2024-08-20T01:26:22.569 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563ecae01a30 con 0x563ecaec5ef0 2024-08-20T01:26:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe474004ac0 con 0x563ecaec5ef0 2024-08-20T01:26:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe491482740 10 client.27601 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.569+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x563ecab8d790 con 0x563ecaec5ef0 2024-08-20T01:26:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.570+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563ecae01a30 con 0x563ecaec5ef0 2024-08-20T01:26:22.570 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.570+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fe4740314a0 con 0x563ecaec5ef0 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 10 client.27601 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 10 client.27601 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 20 client.27601 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaeea570 0x563ecaeec990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 4 client.27601 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.571+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ecae3d160 con 0x563ecaeea570 2024-08-20T01:26:22.572 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.572+0000 7fe487e00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaeea570 0x563ecaeec990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.573+0000 7fe487e00640 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaeea570 0x563ecaeec990 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f0bd57727a9538c server_cookie=e783dc58e5e2fc57 in_seq=0 out_seq=0 2024-08-20T01:26:22.573 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.573+0000 7fe471600640 10 client.27601 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe471600640 1 -- 172.21.15.138:0/2618842739 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563ecae3d160 con 0x563ecaeea570 2024-08-20T01:26:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe471600640 10 client.27601 handle_command_reply: tid=0 2024-08-20T01:26:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 client.27601 shutdown 2024-08-20T01:26:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 msgr2=0x563ecaecf300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.581 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe474005ab0 tx=0x7fe4740052f0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 shutdown_connections 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaeea570 0x563ecaeec990 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaed1e80 0x563ecaecf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaec5ef0 0x563ecaecf300 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 --2- 172.21.15.138:0/2618842739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaecedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.581+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 >> 172.21.15.138:0/2618842739 conn(0x563ecaec33b0 msgr2=0x563ecaec8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.582+0000 7fe471600640 0 client.27601 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.582+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 shutdown_connections 2024-08-20T01:26:22.582 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.582+0000 7fe491482740 1 -- 172.21.15.138:0/2618842739 wait complete. 2024-08-20T01:26:22.583 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.582+0000 7fe491482740 20 client.27601 trim_cache size 0 max 16384 2024-08-20T01:26:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.606+0000 7fe491482740 1 Processor -- start 2024-08-20T01:26:22.606 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 -- start start 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaed0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaef27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881b0350 con 0x563ecaec5ef0 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881adbc0 con 0x563ecaec55d0 2024-08-20T01:26:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.607+0000 7fe491482740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881aba30 con 0x563ecaed1e80 2024-08-20T01:26:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.608+0000 7fe486a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe487e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaed0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43540/0 (socket says 172.21.15.138:43540) 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 -- 172.21.15.138:0/2820510059 learned_addr learned my addr 172.21.15.138:0/2820510059 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe487e00640 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaef27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 -- 172.21.15.138:0/2820510059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 msgr2=0x563ecaef27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaef27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 -- 172.21.15.138:0/2820510059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 msgr2=0x563ecaed0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaed0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.609+0000 7fe486a00640 1 -- 172.21.15.138:0/2820510059 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecac06620 con 0x563ecaec5ef0 2024-08-20T01:26:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.610+0000 7fe487e00640 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaed0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:26:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.610+0000 7fe486a00640 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe4740056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a956b02e4f43742 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.611+0000 7fe473e00640 1 -- 172.21.15.138:0/2820510059 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035030 con 0x563ecaec5ef0 2024-08-20T01:26:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.611+0000 7fe473e00640 1 -- 172.21.15.138:0/2820510059 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe474055070 con 0x563ecaec5ef0 2024-08-20T01:26:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.611+0000 7fe473e00640 1 -- 172.21.15.138:0/2820510059 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740046c0 con 0x563ecaec5ef0 2024-08-20T01:26:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 -- 172.21.15.138:0/2820510059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 msgr2=0x563ecaef2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe47402a910 tx=0x7fe4740056f0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 -- 172.21.15.138:0/2820510059 shutdown_connections 2024-08-20T01:26:22.613 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaef27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec5ef0 0x563ecaef2260 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.613+0000 7fe491482740 1 --2- 172.21.15.138:0/2820510059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec55d0 0x563ecaed0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.614+0000 7fe491482740 1 -- 172.21.15.138:0/2820510059 >> 172.21.15.138:0/2820510059 conn(0x563ecaec33b0 msgr2=0x563ecaec8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.614+0000 7fe491482740 1 -- 172.21.15.138:0/2820510059 shutdown_connections 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.614+0000 7fe491482740 1 -- 172.21.15.138:0/2820510059 wait complete. 2024-08-20T01:26:22.614 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.614+0000 7fe491482740 1 -- start start 2024-08-20T01:26:22.615 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.615+0000 7fe491482740 1 Processor -- start 2024-08-20T01:26:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.616+0000 7fe491482740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 0x563ecaf4d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.616 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe491482740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaf4d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe487e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 0x563ecaf4d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe487e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 0x563ecaf4d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:43548/0 (socket says 172.21.15.138:43548) 2024-08-20T01:26:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe491482740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaf4dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe487e00640 1 -- 172.21.15.138:0/3426932776 learned_addr learned my addr 172.21.15.138:0/3426932776 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:22.617 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.617+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881ac080 con 0x563ecaec55d0 2024-08-20T01:26:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.618+0000 7fe486a00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaf4d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.618 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.618+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881ad4f0 con 0x563ecaec5ef0 2024-08-20T01:26:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.618+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fe4881ac750 con 0x563ecaed1e80 2024-08-20T01:26:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.619+0000 7fe487e00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaf4dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.619+0000 7fe486a00640 1 -- 172.21.15.138:0/3426932776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 msgr2=0x563ecaf4dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.619+0000 7fe491482740 10 client.0 _ll_register_callbacks cb 0x7fe474009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:22.619 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe486a00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaf4dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe486a00640 1 -- 172.21.15.138:0/3426932776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 msgr2=0x563ecaf4d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe486a00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 0x563ecaf4d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe486a00640 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecaee16e0 con 0x563ecaec5ef0 2024-08-20T01:26:22.620 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe487e00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaf4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:26:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.620+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe474055070 con 0x563ecaec5ef0 2024-08-20T01:26:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.621+0000 7fe486a00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaf4d730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe474031f00 tx=0x7fe474004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b5c89c2433eb771 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.621+0000 7fe471600640 10 client.27650 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:26:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.623+0000 7fe471600640 1 -- 172.21.15.138:0/3426932776 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474035040 con 0x563ecaec5ef0 2024-08-20T01:26:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.623+0000 7fe471600640 1 -- 172.21.15.138:0/3426932776 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fe4740037f0 con 0x563ecaec5ef0 2024-08-20T01:26:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.623+0000 7fe471600640 1 -- 172.21.15.138:0/3426932776 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fe474003a30 con 0x563ecaec5ef0 2024-08-20T01:26:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.623+0000 7fe491482740 10 client.27650 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:22.623 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.623+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=54}) -- 0x563ecaee2910 con 0x563ecaec5ef0 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe471600640 1 -- 172.21.15.138:0/3426932776 <== mon.2 v2:172.21.15.142:3300/0 4 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fe474003bd0 con 0x563ecaec5ef0 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 10 client.27650 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 10 client.27650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 20 client.27650 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaee2da0 0x563ecaf68250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:22.624 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 4 client.27650 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:26:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ecae38c00 con 0x563ecaee2da0 2024-08-20T01:26:22.625 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.624+0000 7fe487e00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaee2da0 0x563ecaf68250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.625+0000 7fe487e00640 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaee2da0 0x563ecaf68250 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a513b514cd6471e server_cookie=7d51e685a2a4fd87 in_seq=0 out_seq=0 2024-08-20T01:26:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.625+0000 7fe471600640 10 client.27650 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.971+0000 7fe471600640 1 -- 172.21.15.138:0/3426932776 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ecae38c00 con 0x563ecaee2da0 2024-08-20T01:26:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.972+0000 7fe471600640 10 client.27650 handle_command_reply: tid=0 2024-08-20T01:26:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.972+0000 7fe491482740 1 client.27650 shutdown 2024-08-20T01:26:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.972+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 msgr2=0x563ecaf4d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.972+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaf4d730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe474031f00 tx=0x7fe474004f10 comp rx=0 tx=0).stop 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 shutdown_connections 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x563ecaee2da0 0x563ecaf68250 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x563ecaed1e80 0x563ecaf4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563ecaec5ef0 0x563ecaf4d730 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 --2- 172.21.15.138:0/3426932776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x563ecaec55d0 0x563ecaf4d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 >> 172.21.15.138:0/3426932776 conn(0x563ecaec33b0 msgr2=0x563ecaec40e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe471600640 0 client.27650 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:22.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 shutdown_connections 2024-08-20T01:26:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.973+0000 7fe491482740 1 -- 172.21.15.138:0/3426932776 wait complete. 2024-08-20T01:26:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.974+0000 7fe491482740 20 client.27650 trim_cache size 0 max 16384 2024-08-20T01:26:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:26:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:26:22.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:26:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:26:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 msgr2=0x7fe46c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 0x7fe46c0b6200 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x563ecaec8750 tx=0x7fe478005880 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 msgr2=0x7fe48819b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe47800d950 tx=0x7fe47800de10 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 shutdown_connections 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fe46c0b3d40 0x7fe46c0b6200 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe48810af90 0x7fe4881a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe48810a610 0x7fe48819b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 --2- 172.21.15.138:0/853078756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe488104c10 0x7fe48819b210 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.975+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 >> 172.21.15.138:0/853078756 conn(0x7fe4881006b0 msgr2=0x7fe488107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.976+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 shutdown_connections 2024-08-20T01:26:22.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:22.976+0000 7fe48e400640 1 -- 172.21.15.138:0/853078756 wait complete. 2024-08-20T01:26:23.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:22 smithi167 ceph-mon[21965]: pgmap v934: 129 pgs: 129 active+clean; 972 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 425 KiB/s wr, 46 op/s 2024-08-20T01:26:23.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:22 smithi142 ceph-mon[21799]: pgmap v934: 129 pgs: 129 active+clean; 972 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 425 KiB/s wr, 46 op/s 2024-08-20T01:26:23.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:22 smithi138 ceph-mon[18076]: pgmap v934: 129 pgs: 129 active+clean; 972 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 425 KiB/s wr, 46 op/s 2024-08-20T01:26:23.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 77512 13.50 MB/sec execute 140 sec latency 236.559 ms 2024-08-20T01:26:24.353 INFO:tasks.workunit.client.0.smithi138.stdout: 10 78426 13.82 MB/sec execute 141 sec latency 53.928 ms 2024-08-20T01:26:25.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:24 smithi167 ceph-mon[21965]: pgmap v935: 129 pgs: 129 active+clean; 973 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 368 KiB/s wr, 41 op/s 2024-08-20T01:26:25.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:24 smithi142 ceph-mon[21799]: pgmap v935: 129 pgs: 129 active+clean; 973 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 368 KiB/s wr, 41 op/s 2024-08-20T01:26:25.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:24 smithi138 ceph-mon[18076]: pgmap v935: 129 pgs: 129 active+clean; 973 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 368 KiB/s wr, 41 op/s 2024-08-20T01:26:25.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 79104 13.97 MB/sec execute 142 sec latency 420.128 ms 2024-08-20T01:26:26.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 79667 14.05 MB/sec execute 143 sec latency 1420.312 ms 2024-08-20T01:26:27.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:26 smithi167 ceph-mon[21965]: pgmap v936: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-08-20T01:26:27.231 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:26 smithi138 ceph-mon[18076]: pgmap v936: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-08-20T01:26:27.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:26 smithi142 ceph-mon[21799]: pgmap v936: 129 pgs: 129 active+clean; 987 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 1.7 MiB/s wr, 132 op/s 2024-08-20T01:26:27.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 80206 14.21 MB/sec execute 144 sec latency 2420.396 ms 2024-08-20T01:26:28.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 80773 14.30 MB/sec execute 145 sec latency 3420.437 ms 2024-08-20T01:26:29.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:28 smithi167 ceph-mon[21965]: pgmap v937: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-08-20T01:26:29.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:28 smithi142 ceph-mon[21799]: pgmap v937: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-08-20T01:26:29.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:28 smithi138 ceph-mon[18076]: pgmap v937: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.3 MiB/s wr, 379 op/s 2024-08-20T01:26:29.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 81202 14.35 MB/sec execute 146 sec latency 4420.490 ms 2024-08-20T01:26:30.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 81312 14.31 MB/sec execute 147 sec latency 5420.539 ms 2024-08-20T01:26:31.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:30 smithi167 ceph-mon[21965]: pgmap v938: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-20T01:26:31.229 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:30 smithi138 ceph-mon[18076]: pgmap v938: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-20T01:26:31.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:30 smithi142 ceph-mon[21799]: pgmap v938: 129 pgs: 129 active+clean; 1005 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.4 MiB/s wr, 391 op/s 2024-08-20T01:26:31.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 81448 14.24 MB/sec execute 148 sec latency 6357.940 ms 2024-08-20T01:26:32.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 81895 14.35 MB/sec execute 149 sec latency 5672.522 ms 2024-08-20T01:26:33.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:32 smithi167 ceph-mon[21965]: pgmap v939: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-08-20T01:26:33.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:32 smithi142 ceph-mon[21799]: pgmap v939: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-08-20T01:26:33.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:32 smithi138 ceph-mon[18076]: pgmap v939: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 6.6 MiB/s wr, 520 op/s 2024-08-20T01:26:33.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 82247 14.34 MB/sec execute 150 sec latency 6126.854 ms 2024-08-20T01:26:34.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 82834 14.49 MB/sec execute 151 sec latency 6869.601 ms 2024-08-20T01:26:35.197 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:34 smithi138 ceph-mon[18076]: pgmap v940: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-08-20T01:26:35.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:34 smithi167 ceph-mon[21965]: pgmap v940: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-08-20T01:26:35.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:34 smithi142 ceph-mon[21799]: pgmap v940: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 MiB/s rd, 6.3 MiB/s wr, 531 op/s 2024-08-20T01:26:35.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 83353 14.56 MB/sec execute 152 sec latency 6976.688 ms 2024-08-20T01:26:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 -- 172.21.15.138:0/1329134308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2406a530 msgr2=0x7fdd2406a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 --2- 172.21.15.138:0/1329134308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2406a530 0x7fdd2406a970 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009900 tx=0x7fdd1002a400 comp rx=0 tx=0).stop 2024-08-20T01:26:36.169 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 -- 172.21.15.138:0/1329134308 shutdown_connections 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 --2- 172.21.15.138:0/1329134308 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd24102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 --2- 172.21.15.138:0/1329134308 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd2406a530 0x7fdd2406a970 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 --2- 172.21.15.138:0/1329134308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd24107400 0x7fdd241077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 -- 172.21.15.138:0/1329134308 >> 172.21.15.138:0/1329134308 conn(0x7fdd24077270 msgr2=0x7fdd24077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.168+0000 7fdd23200640 1 -- 172.21.15.138:0/1329134308 shutdown_connections 2024-08-20T01:26:36.170 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.169+0000 7fdd23200640 1 -- 172.21.15.138:0/1329134308 wait complete. 2024-08-20T01:26:36.171 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.170+0000 7fdd23200640 1 Processor -- start 2024-08-20T01:26:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.171+0000 7fdd23200640 1 -- start start 2024-08-20T01:26:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2406a530 0x7fdd2419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd24107400 0x7fdd241a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24118de0 con 0x7fdd24107400 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24118c60 con 0x7fdd2406a530 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd23200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24118f60 con 0x7fdd2406afc0 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd21400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd21400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59312/0 (socket says 172.21.15.138:59312) 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.172+0000 7fdd21400640 1 -- 172.21.15.138:0/3214613721 learned_addr learned my addr 172.21.15.138:0/3214613721 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 -- 172.21.15.138:0/3214613721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2406a530 msgr2=0x7fdd2419f3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2406a530 0x7fdd2419f3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 -- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd24107400 msgr2=0x7fdd241a4c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd24107400 0x7fdd241a4c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 -- 172.21.15.138:0/3214613721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd241a5440 con 0x7fdd2406afc0 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.173+0000 7fdd21400640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd10004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.174+0000 7fdd1aa00640 1 -- 172.21.15.138:0/3214613721 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1003f030 con 0x7fdd2406afc0 2024-08-20T01:26:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.174+0000 7fdd1aa00640 1 -- 172.21.15.138:0/3214613721 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd10003aa0 con 0x7fdd2406afc0 2024-08-20T01:26:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.174+0000 7fdd1aa00640 1 -- 172.21.15.138:0/3214613721 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10035070 con 0x7fdd2406afc0 2024-08-20T01:26:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.174+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd241a56d0 con 0x7fdd2406afc0 2024-08-20T01:26:36.177 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.174+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd24107ee0 con 0x7fdd2406afc0 2024-08-20T01:26:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.180+0000 7fdd2bedb740 1 Processor -- start 2024-08-20T01:26:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.180+0000 7fdd2bedb740 1 -- start start 2024-08-20T01:26:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.180+0000 7fdd1aa00640 1 -- 172.21.15.138:0/3214613721 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fdd10042030 con 0x7fdd2406afc0 2024-08-20T01:26:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.180+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.180+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df7728fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd22800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd22800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59326/0 (socket says 172.21.15.138:59326) 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd22800640 1 -- 172.21.15.138:0/671567378 learned_addr learned my addr 172.21.15.138:0/671567378 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 0x55df772927b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df771f9210 con 0x55df77284430 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55df771e8710 con 0x55df772903c0 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df771ec750 con 0x55df77283b10 2024-08-20T01:26:36.183 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.181+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df7728fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd21400640 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 0x55df772927b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd22800640 1 -- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 msgr2=0x55df772927b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd22800640 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 0x55df772927b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd22800640 1 -- 172.21.15.138:0/671567378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 msgr2=0x55df7728fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd22800640 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df7728fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd22800640 1 -- 172.21.15.138:0/671567378 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df771b4de0 con 0x55df77283b10 2024-08-20T01:26:36.184 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd21400640 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 0x55df772927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:26:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.182+0000 7fdd1aa00640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 0x7fdd080b6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd1aa00640 1 -- 172.21.15.138:0/3214613721 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fdd101046e0 con 0x7fdd2406afc0 2024-08-20T01:26:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd22800640 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e30 tx=0x7fdd1c00e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b2df8f0c1ae5abb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd0f400640 1 -- 172.21.15.138:0/671567378 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c003b70 con 0x55df77283b10 2024-08-20T01:26:36.185 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd0f400640 1 -- 172.21.15.138:0/671567378 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1c00ee70 con 0x55df77283b10 2024-08-20T01:26:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 0x7fdd080b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.183+0000 7fdd0f400640 1 -- 172.21.15.138:0/671567378 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0107a0 con 0x55df77283b10 2024-08-20T01:26:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.185+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 msgr2=0x55df77283ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.185+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009e30 tx=0x7fdd1c00e720 comp rx=0 tx=0).stop 2024-08-20T01:26:36.186 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.185+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 0x7fdd080b6300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400a490 tx=0x7fdd1400a420 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.185+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 shutdown_connections 2024-08-20T01:26:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.186+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df772903c0 0x55df772927b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.186+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df7728fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.186+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/671567378 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77283b10 0x55df77283ef0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.187 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.186+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 >> 172.21.15.138:0/671567378 conn(0x55df77281940 msgr2=0x55df77282ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.187+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 shutdown_connections 2024-08-20T01:26:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.187+0000 7fdd2bedb740 1 -- 172.21.15.138:0/671567378 wait complete. 2024-08-20T01:26:36.188 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.187+0000 7fdd2bedb740 1 -- start start 2024-08-20T01:26:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.188+0000 7fdd2bedb740 1 Processor -- start 2024-08-20T01:26:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df7728d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55df77121ab0 con 0x55df772903c0 2024-08-20T01:26:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55df7718f140 con 0x55df77283b10 2024-08-20T01:26:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55df772930a0 con 0x55df77284430 2024-08-20T01:26:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.189+0000 7fdd2bedb740 10 client.0 _ll_register_callbacks cb 0x55df771a25d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.190+0000 7fdd21400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.190+0000 7fdd21400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59336/0 (socket says 172.21.15.138:59336) 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.190+0000 7fdd21400640 1 -- 172.21.15.138:0/3260208804 learned_addr learned my addr 172.21.15.138:0/3260208804 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 -- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 msgr2=0x55df7728fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 -- 172.21.15.138:0/3260208804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 msgr2=0x55df7728d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df7728d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 -- 172.21.15.138:0/3260208804 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df77190170 con 0x55df77284430 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd21400640 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd100c8160 tx=0x7fdd10003e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24345cc2429abe83 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.191+0000 7fdd03e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:26:36.193 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.192+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100ca660 con 0x55df77284430 2024-08-20T01:26:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.193+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fdd100c7180 con 0x55df77284430 2024-08-20T01:26:36.194 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.193+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd1c00ee70 con 0x55df77284430 2024-08-20T01:26:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.194+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd100caa00 con 0x55df77284430 2024-08-20T01:26:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.194+0000 7fdd2bedb740 10 client.27616 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:36.195 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.194+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55df76f4b790 con 0x55df77284430 2024-08-20T01:26:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.195+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd1c00ee70 con 0x55df77284430 2024-08-20T01:26:36.196 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.195+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fdd10104280 con 0x55df77284430 2024-08-20T01:26:36.199 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 10 client.27616 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 10 client.27616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 20 client.27616 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df772b10a0 0x55df772b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 4 client.27616 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:26:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.198+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df771fb160 con 0x55df772b10a0 2024-08-20T01:26:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.200+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df772b10a0 0x55df772b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.203+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df772b10a0 0x55df772b3560 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52e2973e68e60f2f server_cookie=5b94e0eaa822a978 in_seq=0 out_seq=0 2024-08-20T01:26:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.203+0000 7fdd03e00640 10 client.27616 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.204+0000 7fdd03e00640 1 -- 172.21.15.138:0/3260208804 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df771fb160 con 0x55df772b10a0 2024-08-20T01:26:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.204+0000 7fdd03e00640 10 client.27616 handle_command_reply: tid=0 2024-08-20T01:26:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.204+0000 7fdd2bedb740 1 client.27616 shutdown 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.204+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 msgr2=0x55df7728d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.204+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdd100c8160 tx=0x7fdd10003e80 comp rx=0 tx=0).stop 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 shutdown_connections 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df772b10a0 0x55df772b3560 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df7728d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df7728d060 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3260208804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd03e00640 0 client.27616 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:36.206 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 >> 172.21.15.138:0/3260208804 conn(0x55df77281940 msgr2=0x55df772874e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 shutdown_connections 2024-08-20T01:26:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.205+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3260208804 wait complete. 2024-08-20T01:26:36.207 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.206+0000 7fdd2bedb740 20 client.27616 trim_cache size 0 max 16384 2024-08-20T01:26:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.237+0000 7fdd2bedb740 1 Processor -- start 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.237+0000 7fdd2bedb740 1 -- start start 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.237+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df772b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df772b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410d2f0 con 0x55df772903c0 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410ab60 con 0x55df77283b10 2024-08-20T01:26:36.239 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241a04c0 con 0x55df77284430 2024-08-20T01:26:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:34702/0 (socket says 172.21.15.138:34702) 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd21e00640 1 -- 172.21.15.138:0/458381923 learned_addr learned my addr 172.21.15.138:0/458381923 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd21400640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df772b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.238+0000 7fdd22800640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df772b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 -- 172.21.15.138:0/458381923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 msgr2=0x55df772b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df772b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 -- 172.21.15.138:0/458381923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 msgr2=0x55df772b9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df772b9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 -- 172.21.15.138:0/458381923 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df76fc4620 con 0x55df77283b10 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21400640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df772b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd21e00640 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400fbc0 tx=0x7fdd14012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617c18c1ae8bd8ac server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd0ea00640 1 -- 172.21.15.138:0/458381923 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd14016070 con 0x55df77283b10 2024-08-20T01:26:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd0ea00640 1 -- 172.21.15.138:0/458381923 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd14025850 con 0x55df77283b10 2024-08-20T01:26:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.239+0000 7fdd0ea00640 1 -- 172.21.15.138:0/458381923 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1401c860 con 0x55df77283b10 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 -- 172.21.15.138:0/458381923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 msgr2=0x55df7728eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400fbc0 tx=0x7fdd14012000 comp rx=0 tx=0).stop 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 -- 172.21.15.138:0/458381923 shutdown_connections 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df772903c0 0x55df772b9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df77284430 0x55df772b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/458381923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df7728eda0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 -- 172.21.15.138:0/458381923 >> 172.21.15.138:0/458381923 conn(0x55df77281940 msgr2=0x55df7728e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 -- 172.21.15.138:0/458381923 shutdown_connections 2024-08-20T01:26:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.240+0000 7fdd2bedb740 1 -- 172.21.15.138:0/458381923 wait complete. 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 -- start start 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 Processor -- start 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df77295b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df772960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdd24074d00 con 0x55df77284430 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdd2410a490 con 0x55df77283b10 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdd241096f0 con 0x55df772903c0 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.241+0000 7fdd2bedb740 10 client.0 _ll_register_callbacks cb 0x7fdd10009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd21400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df772960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd21e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df77295b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59346/0 (socket says 172.21.15.138:59346) 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd21400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df772960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39642/0 (socket says 172.21.15.138:39642) 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 -- 172.21.15.138:0/3852424021 learned_addr learned my addr 172.21.15.138:0/3852424021 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 -- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 msgr2=0x55df77295b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df77295b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 -- 172.21.15.138:0/3852424021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 msgr2=0x55df772960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df772960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 -- 172.21.15.138:0/3852424021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df77314580 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd22800640 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0189e0 tx=0x7fdd1c018a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29fc4f5f3f8d3cd3 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.242+0000 7fdd03e00640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c00ee70 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd14025850 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fdd1c018bb0 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fdd1c02f050 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 10 client.27622 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55df7731b920 con 0x55df772903c0 2024-08-20T01:26:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd14025850 con 0x55df772903c0 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fdd1c02a030 con 0x55df772903c0 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 10 client.27622 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 10 client.27622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 20 client.27622 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df7731be90 0x55df7731e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 4 client.27622 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.243+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df771f6c00 con 0x55df7731be90 2024-08-20T01:26:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.244+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df7731be90 0x55df7731e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.244+0000 7fdd03e00640 10 client.27622 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:36.250 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.244+0000 7fdd21e00640 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df7731be90 0x55df7731e2b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d78146b5f2910027 server_cookie=56eff8196d5423a5 in_seq=0 out_seq=0 2024-08-20T01:26:36.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 83808 14.63 MB/sec execute 153 sec latency 7130.705 ms 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd03e00640 1 -- 172.21.15.138:0/3852424021 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df771f6c00 con 0x55df7731be90 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd03e00640 10 client.27622 handle_command_reply: tid=0 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd2bedb740 1 client.27622 shutdown 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 msgr2=0x55df77313e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0189e0 tx=0x7fdd1c018a10 comp rx=0 tx=0).stop 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.626+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 shutdown_connections 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55df7731be90 0x55df7731e2b0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.628 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd03e00640 0 client.27622 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55df772903c0 0x55df77313e40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55df77284430 0x55df772960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 --2- 172.21.15.138:0/3852424021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df77283b10 0x55df77295b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 >> 172.21.15.138:0/3852424021 conn(0x55df77281940 msgr2=0x55df7728eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 shutdown_connections 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 1 -- 172.21.15.138:0/3852424021 wait complete. 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.627+0000 7fdd2bedb740 20 client.27622 trim_cache size 0 max 16384 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:26:36.629 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 msgr2=0x7fdd080b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 0x7fdd080b6300 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400a490 tx=0x7fdd1400a420 comp rx=0 tx=0).stop 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 msgr2=0x7fdd2419f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd10004060 comp rx=0 tx=0).stop 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 shutdown_connections 2024-08-20T01:26:36.630 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdd080b3e40 0x7fdd080b6300 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdd24107400 0x7fdd241a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdd2406afc0 0x7fdd2419f8e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 --2- 172.21.15.138:0/3214613721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd2406a530 0x7fdd2419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.628+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 >> 172.21.15.138:0/3214613721 conn(0x7fdd24077270 msgr2=0x7fdd241008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.629+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 shutdown_connections 2024-08-20T01:26:36.631 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:36.629+0000 7fdd23200640 1 -- 172.21.15.138:0/3214613721 wait complete. 2024-08-20T01:26:36.838 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:36 smithi138 ceph-mon[18076]: pgmap v941: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.7 MiB/s wr, 552 op/s 2024-08-20T01:26:37.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:36 smithi167 ceph-mon[21965]: pgmap v941: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.7 MiB/s wr, 552 op/s 2024-08-20T01:26:37.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:36 smithi142 ceph-mon[21799]: pgmap v941: 129 pgs: 129 active+clean; 1.0 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 MiB/s rd, 6.7 MiB/s wr, 552 op/s 2024-08-20T01:26:37.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 84309 14.67 MB/sec execute 154 sec latency 8130.757 ms 2024-08-20T01:26:38.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 85018 14.83 MB/sec execute 155 sec latency 8652.543 ms 2024-08-20T01:26:39.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:38 smithi167 ceph-mon[21965]: pgmap v942: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-08-20T01:26:39.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:38 smithi142 ceph-mon[21799]: pgmap v942: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-08-20T01:26:39.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:38 smithi138 ceph-mon[18076]: pgmap v942: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 MiB/s rd, 7.3 MiB/s wr, 620 op/s 2024-08-20T01:26:39.354 INFO:tasks.workunit.client.0.smithi138.stdout: 10 85759 15.01 MB/sec execute 156 sec latency 84.890 ms 2024-08-20T01:26:40.356 INFO:tasks.workunit.client.0.smithi138.stdout: 10 86356 15.12 MB/sec execute 157 sec latency 201.834 ms 2024-08-20T01:26:41.125 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:40 smithi138 ceph-mon[18076]: pgmap v943: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 5.2 MiB/s wr, 435 op/s 2024-08-20T01:26:41.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:40 smithi167 ceph-mon[21965]: pgmap v943: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 5.2 MiB/s wr, 435 op/s 2024-08-20T01:26:41.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:40 smithi142 ceph-mon[21799]: pgmap v943: 129 pgs: 129 active+clean; 1.0 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 49 MiB/s rd, 5.2 MiB/s wr, 435 op/s 2024-08-20T01:26:41.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 86514 15.08 MB/sec execute 158 sec latency 798.064 ms 2024-08-20T01:26:42.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 86534 14.99 MB/sec execute 159 sec latency 1752.010 ms 2024-08-20T01:26:43.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:42 smithi167 ceph-mon[21965]: pgmap v944: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 7.3 MiB/s wr, 635 op/s 2024-08-20T01:26:43.227 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:42 smithi138 ceph-mon[18076]: pgmap v944: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 7.3 MiB/s wr, 635 op/s 2024-08-20T01:26:43.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:42 smithi142 ceph-mon[21799]: pgmap v944: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 MiB/s rd, 7.3 MiB/s wr, 635 op/s 2024-08-20T01:26:43.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 86751 14.97 MB/sec execute 160 sec latency 1901.443 ms 2024-08-20T01:26:44.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 86751 14.88 MB/sec execute 161 sec latency 2901.483 ms 2024-08-20T01:26:45.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:44 smithi138 ceph-mon[18076]: pgmap v945: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-08-20T01:26:45.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:44 smithi167 ceph-mon[21965]: pgmap v945: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-08-20T01:26:45.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:44 smithi142 ceph-mon[21799]: pgmap v945: 129 pgs: 129 active+clean; 1.0 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 MiB/s rd, 6.4 MiB/s wr, 604 op/s 2024-08-20T01:26:45.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87040 14.86 MB/sec execute 162 sec latency 3612.692 ms 2024-08-20T01:26:46.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:45 smithi138 ceph-mon[18076]: pgmap v946: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-20T01:26:46.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:45 smithi167 ceph-mon[21965]: pgmap v946: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-20T01:26:46.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:45 smithi142 ceph-mon[21799]: pgmap v946: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 6.3 MiB/s wr, 548 op/s 2024-08-20T01:26:46.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87269 14.86 MB/sec execute 163 sec latency 726.398 ms 2024-08-20T01:26:47.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87269 14.77 MB/sec execute 164 sec latency 1726.456 ms 2024-08-20T01:26:48.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:48 smithi138 ceph-mon[18076]: pgmap v947: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 8.9 MiB/s wr, 632 op/s 2024-08-20T01:26:48.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87269 14.68 MB/sec execute 165 sec latency 2726.529 ms 2024-08-20T01:26:48.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:48 smithi167 ceph-mon[21965]: pgmap v947: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 8.9 MiB/s wr, 632 op/s 2024-08-20T01:26:48.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:48 smithi142 ceph-mon[21799]: pgmap v947: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 30 MiB/s rd, 8.9 MiB/s wr, 632 op/s 2024-08-20T01:26:49.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87269 14.59 MB/sec execute 166 sec latency 3726.588 ms 2024-08-20T01:26:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 -- 172.21.15.138:0/3138193843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c075270 msgr2=0x7fa81c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 --2- 172.21.15.138:0/3138193843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c075270 0x7fa81c075650 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a410 comp rx=0 tx=0).stop 2024-08-20T01:26:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 -- 172.21.15.138:0/3138193843 shutdown_connections 2024-08-20T01:26:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 --2- 172.21.15.138:0/3138193843 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c113550 0x7fa81c115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 --2- 172.21.15.138:0/3138193843 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c113010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 --2- 172.21.15.138:0/3138193843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c075270 0x7fa81c075650 secure :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa80c009900 tx=0x7fa80c02a410 comp rx=0 tx=0).stop 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 -- 172.21.15.138:0/3138193843 >> 172.21.15.138:0/3138193843 conn(0x7fa81c0fe280 msgr2=0x7fa81c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 -- 172.21.15.138:0/3138193843 shutdown_connections 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.813+0000 7fa823200640 1 -- 172.21.15.138:0/3138193843 wait complete. 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.818+0000 7fa823200640 1 Processor -- start 2024-08-20T01:26:49.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.818+0000 7fa823200640 1 -- start start 2024-08-20T01:26:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c113550 0x7fa81c19fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 0x7fa81c1a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c118e60 con 0x7fa81c1a4f00 2024-08-20T01:26:49.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c118ce0 con 0x7fa81c113550 2024-08-20T01:26:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c118fe0 con 0x7fa81c075b90 2024-08-20T01:26:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46464/0 (socket says 172.21.15.138:46464) 2024-08-20T01:26:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 -- 172.21.15.138:0/2434074423 learned_addr learned my addr 172.21.15.138:0/2434074423 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa822800640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 0x7fa81c1a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 -- 172.21.15.138:0/2434074423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c113550 msgr2=0x7fa81c19fb70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-08-20T01:26:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c113550 0x7fa81c19fb70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 -- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 msgr2=0x7fa81c1a53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 0x7fa81c1a53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 -- 172.21.15.138:0/2434074423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1a5ae0 con 0x7fa81c075b90 2024-08-20T01:26:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa822800640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 0x7fa81c1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:26:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa821e00640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005ab0 tx=0x7fa80c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa816a00640 1 -- 172.21.15.138:0/2434074423 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c035030 con 0x7fa81c075b90 2024-08-20T01:26:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa816a00640 1 -- 172.21.15.138:0/2434074423 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa80c003d30 con 0x7fa81c075b90 2024-08-20T01:26:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa816a00640 1 -- 172.21.15.138:0/2434074423 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0318a0 con 0x7fa81c075b90 2024-08-20T01:26:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c1ac080 con 0x7fa81c075b90 2024-08-20T01:26:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.819+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c1ac430 con 0x7fa81c075b90 2024-08-20T01:26:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 Processor -- start 2024-08-20T01:26:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 -- start start 2024-08-20T01:26:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b87abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b897b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.828 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b98b7f0210 con 0x55b98b898090 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b98b7df710 con 0x55b98b87a7d0 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa827613740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b98b7e3750 con 0x55b98b87b0f0 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50430/0 (socket says 172.21.15.138:50430) 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 -- 172.21.15.138:0/4174266724 learned_addr learned my addr 172.21.15.138:0/4174266724 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:49.829 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 -- 172.21.15.138:0/4174266724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 msgr2=0x55b98b897b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-08-20T01:26:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b897b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 -- 172.21.15.138:0/4174266724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 msgr2=0x55b98b87abb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:26:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b87abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.825+0000 7fa821400640 1 -- 172.21.15.138:0/4174266724 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98b7abde0 con 0x55b98b898090 2024-08-20T01:26:49.830 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.830+0000 7fa816a00640 1 -- 172.21.15.138:0/2434074423 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fa80c03b580 con 0x7fa81c075b90 2024-08-20T01:26:49.833 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa816a00640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 0x7fa8040b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa816a00640 1 -- 172.21.15.138:0/2434074423 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fa80c104450 con 0x7fa81c075b90 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa821400640 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa810009740 tx=0x7fa810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0a4ba9cbb23981 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa80b000640 1 -- 172.21.15.138:0/4174266724 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810015020 con 0x55b98b898090 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa80b000640 1 -- 172.21.15.138:0/4174266724 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa810009e50 con 0x55b98b898090 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa80b000640 1 -- 172.21.15.138:0/4174266724 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa810010380 con 0x55b98b898090 2024-08-20T01:26:49.834 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa822800640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 0x7fa8040b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 -- 172.21.15.138:0/4174266724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 msgr2=0x55b98b89a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa810009740 tx=0x7fa810009c00 comp rx=0 tx=0).stop 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa822800640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 0x7fa8040b6280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa81c06bd80 tx=0x7fa818009500 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 -- 172.21.15.138:0/4174266724 shutdown_connections 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b89a480 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b897b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 --2- 172.21.15.138:0/4174266724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b87abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 -- 172.21.15.138:0/4174266724 >> 172.21.15.138:0/4174266724 conn(0x55b98b878600 msgr2=0x55b98b8797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 -- 172.21.15.138:0/4174266724 shutdown_connections 2024-08-20T01:26:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.831+0000 7fa827613740 1 -- 172.21.15.138:0/4174266724 wait complete. 2024-08-20T01:26:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 -- start start 2024-08-20T01:26:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 Processor -- start 2024-08-20T01:26:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55b98b718ab0 con 0x55b98b898090 2024-08-20T01:26:49.839 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55b98b786140 con 0x55b98b87a7d0 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55b98b89ad70 con 0x55b98b87b0f0 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 10 client.0 _ll_register_callbacks cb 0x55b98b7995d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50446/0 (socket says 172.21.15.138:50446) 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 -- 172.21.15.138:0/3922036766 learned_addr learned my addr 172.21.15.138:0/3922036766 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:49.840 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 -- 172.21.15.138:0/3922036766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 msgr2=0x55b98b8955f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8955f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 -- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 msgr2=0x55b98b8950b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8950b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 -- 172.21.15.138:0/3922036766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98b787170 con 0x55b98b898090 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa822800640 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa81800fb40 tx=0x7fa818004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950906b0b30ae695 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:49.841 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa7ff400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:26:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81801e620 con 0x55b98b898090 2024-08-20T01:26:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b98b7b6a30 con 0x55b98b898090 2024-08-20T01:26:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.837+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fa81801e7c0 con 0x55b98b898090 2024-08-20T01:26:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.842+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa81801e9c0 con 0x55b98b898090 2024-08-20T01:26:49.842 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.842+0000 7fa827613740 10 client.17610 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.842+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b98b542790 con 0x55b98b898090 2024-08-20T01:26:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.842+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b98b7b6a30 con 0x55b98b898090 2024-08-20T01:26:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa818030c00 con 0x55b98b898090 2024-08-20T01:26:49.843 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 10 client.17610 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 10 client.17610 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 20 client.17610 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b8a8510 0x55b98b8aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 4 client.17610 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:26:49.844 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.843+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b98b7f2160 con 0x55b98b8a8510 2024-08-20T01:26:49.867 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.866+0000 7fa821e00640 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b8a8510 0x55b98b8aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.914 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.914+0000 7fa7ff400640 10 client.17610 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:49.915 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.914+0000 7fa821e00640 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b8a8510 0x55b98b8aa990 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45d491d7f6494f0 server_cookie=997503da3cce559d in_seq=0 out_seq=0 2024-08-20T01:26:49.938 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa7ff400640 1 -- 172.21.15.138:0/3922036766 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b98b7f2160 con 0x55b98b8a8510 2024-08-20T01:26:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa7ff400640 10 client.17610 handle_command_reply: tid=0 2024-08-20T01:26:49.939 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 client.17610 shutdown 2024-08-20T01:26:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 msgr2=0x55b98b895b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa81800fb40 tx=0x7fa818004ab0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.941 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 shutdown_connections 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b8a8510 0x55b98b8aa990 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b895b30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 --2- 172.21.15.138:0/3922036766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 >> 172.21.15.138:0/3922036766 conn(0x55b98b878600 msgr2=0x55b98b87e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa7ff400640 0 client.17610 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 shutdown_connections 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.937+0000 7fa827613740 1 -- 172.21.15.138:0/3922036766 wait complete. 2024-08-20T01:26:49.942 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.942+0000 7fa827613740 20 client.17610 trim_cache size 0 max 16384 2024-08-20T01:26:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.967+0000 7fa827613740 1 Processor -- start 2024-08-20T01:26:49.971 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.967+0000 7fa827613740 1 -- start start 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87b0f0 0x55b98b879340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b898090 0x55b98b879880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1b0650 con 0x55b98b87a7d0 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1adec0 con 0x55b98b87b0f0 2024-08-20T01:26:49.972 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa827613740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c10e650 con 0x55b98b898090 2024-08-20T01:26:49.973 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.972+0000 7fa821e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.973+0000 7fa821e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50456/0 (socket says 172.21.15.138:50456) 2024-08-20T01:26:49.974 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.974+0000 7fa821e00640 1 -- 172.21.15.138:0/4080468704 learned_addr learned my addr 172.21.15.138:0/4080468704 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.974+0000 7fa821e00640 1 -- 172.21.15.138:0/4080468704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b898090 msgr2=0x55b98b879880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.975+0000 7fa821e00640 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b898090 0x55b98b879880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.975+0000 7fa821e00640 1 -- 172.21.15.138:0/4080468704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87b0f0 msgr2=0x55b98b879340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:26:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.975+0000 7fa821e00640 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87b0f0 0x55b98b879340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.975 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.975+0000 7fa821e00640 1 -- 172.21.15.138:0/4080468704 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b98b5bb620 con 0x55b98b87a7d0 2024-08-20T01:26:49.976 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.976+0000 7fa821e00640 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44b521108c82839 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.977+0000 7fa80a600640 1 -- 172.21.15.138:0/4080468704 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0c6450 con 0x55b98b87a7d0 2024-08-20T01:26:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.977+0000 7fa80a600640 1 -- 172.21.15.138:0/4080468704 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b98b7f2160 con 0x55b98b87a7d0 2024-08-20T01:26:49.977 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.977+0000 7fa80a600640 1 -- 172.21.15.138:0/4080468704 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa80c0d3d70 con 0x55b98b87a7d0 2024-08-20T01:26:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.979+0000 7fa827613740 1 -- 172.21.15.138:0/4080468704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 msgr2=0x55b98b892b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.979+0000 7fa827613740 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa80c000c00 tx=0x7fa80c031400 comp rx=0 tx=0).stop 2024-08-20T01:26:49.979 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.979+0000 7fa827613740 1 -- 172.21.15.138:0/4080468704 shutdown_connections 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.979+0000 7fa827613740 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b898090 0x55b98b879880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.979+0000 7fa827613740 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87b0f0 0x55b98b879340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.980+0000 7fa827613740 1 --2- 172.21.15.138:0/4080468704 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b87a7d0 0x55b98b892b20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.980+0000 7fa827613740 1 -- 172.21.15.138:0/4080468704 >> 172.21.15.138:0/4080468704 conn(0x55b98b878600 msgr2=0x55b98b8991a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.980+0000 7fa827613740 1 -- 172.21.15.138:0/4080468704 shutdown_connections 2024-08-20T01:26:49.980 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.980+0000 7fa827613740 1 -- 172.21.15.138:0/4080468704 wait complete. 2024-08-20T01:26:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.981+0000 7fa827613740 1 -- start start 2024-08-20T01:26:49.998 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.982+0000 7fa827613740 1 Processor -- start 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8acff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c10eca0 con 0x55b98b898090 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1ad7f0 con 0x55b98b87a7d0 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1aca50 con 0x55b98b87b0f0 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.983+0000 7fa827613740 10 client.0 _ll_register_callbacks cb 0x7fa818003f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.985+0000 7fa822800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.986+0000 7fa822800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:50462/0 (socket says 172.21.15.138:50462) 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.986+0000 7fa822800640 1 -- 172.21.15.138:0/1105431288 learned_addr learned my addr 172.21.15.138:0/1105431288 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.988+0000 7fa822800640 1 -- 172.21.15.138:0/1105431288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 msgr2=0x55b98b8ad530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.988+0000 7fa822800640 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8ad530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:49.999 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.988+0000 7fa822800640 1 -- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 msgr2=0x55b98b8acff0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.988+0000 7fa822800640 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8acff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.988+0000 7fa822800640 1 -- 172.21.15.138:0/1105431288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b98b9111d0 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.989+0000 7fa822800640 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa81801fda0 tx=0x7fa81801fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59fe69d333ebe0e server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.989+0000 7fa7ff400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.991+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818040060 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.991+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7fa818026980 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.991+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b98b7f2160 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.992+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa818035070 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.992+0000 7fa827613740 10 client.17616 fetch_fsmap learned FSMap version 54 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.992+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) -- 0x55b98b9124e0 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b98b7f2160 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 54) ==== 8366+0+0 (secure 0 0 0) 0x7fa818005040 con 0x55b98b898090 2024-08-20T01:26:50.000 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 10 client.17616 fetch_fsmap finished waiting for FSMap version 54 2024-08-20T01:26:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:26:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 20 client.17616 populate_metadata read hostname 'smithi138' 2024-08-20T01:26:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b912c40 0x55b98b915060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:26:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 4 client.17616 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:26:50.001 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:49.993+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b98b7edc00 con 0x55b98b912c40 2024-08-20T01:26:50.009 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.009+0000 7fa821e00640 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b912c40 0x55b98b915060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:26:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.093+0000 7fa821e00640 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b912c40 0x55b98b915060 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9954896e94efbca server_cookie=ea6f669345b4a219 in_seq=0 out_seq=0 2024-08-20T01:26:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.093+0000 7fa7ff400640 10 client.17616 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:50.331 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.331+0000 7fa7ff400640 1 -- 172.21.15.138:0/1105431288 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b98b7edc00 con 0x55b98b912c40 2024-08-20T01:26:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa7ff400640 10 client.17616 handle_command_reply: tid=0 2024-08-20T01:26:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa827613740 1 client.17616 shutdown 2024-08-20T01:26:50.332 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 msgr2=0x55b98b8ada70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:50.333 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa81801fda0 tx=0x7fa81801fdd0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 shutdown_connections 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.332+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55b98b912c40 0x55b98b915060 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa7ff400640 0 client.17616 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55b98b898090 0x55b98b8ada70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55b98b87b0f0 0x55b98b8ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 --2- 172.21.15.138:0/1105431288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b98b87a7d0 0x55b98b8acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 >> 172.21.15.138:0/1105431288 conn(0x55b98b878600 msgr2=0x55b98b879340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 shutdown_connections 2024-08-20T01:26:50.334 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.333+0000 7fa827613740 1 -- 172.21.15.138:0/1105431288 wait complete. 2024-08-20T01:26:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.334+0000 7fa827613740 20 client.17616 trim_cache size 0 max 16384 2024-08-20T01:26:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:26:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:26:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:26:50.343 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:26:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.337+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 msgr2=0x7fa8040b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.337+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 0x7fa8040b6280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fa81c06bd80 tx=0x7fa818009500 comp rx=0 tx=0).stop 2024-08-20T01:26:50.344 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.337+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 msgr2=0x7fa81c19f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.337+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa80c005ab0 tx=0x7fa80c0045e0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 shutdown_connections 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fa8040b3dc0 0x7fa8040b6280 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa81c1a4f00 0x7fa81c1a53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa81c113550 0x7fa81c19fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 --2- 172.21.15.138:0/2434074423 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fa81c075b90 0x7fa81c19f630 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 >> 172.21.15.138:0/2434074423 conn(0x7fa81c0fe280 msgr2=0x7fa81c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 shutdown_connections 2024-08-20T01:26:50.345 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:26:50.338+0000 7fa823200640 1 -- 172.21.15.138:0/2434074423 wait complete. 2024-08-20T01:26:50.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87283 14.52 MB/sec execute 167 sec latency 4726.718 ms 2024-08-20T01:26:50.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:50 smithi167 ceph-mon[21965]: pgmap v948: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 500 op/s 2024-08-20T01:26:50.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:50 smithi142 ceph-mon[21799]: pgmap v948: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 500 op/s 2024-08-20T01:26:50.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:50 smithi138 ceph-mon[18076]: pgmap v948: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 500 op/s 2024-08-20T01:26:51.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87283 14.44 MB/sec execute 168 sec latency 5726.775 ms 2024-08-20T01:26:52.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87283 14.35 MB/sec execute 169 sec latency 6726.826 ms 2024-08-20T01:26:52.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:52 smithi167 ceph-mon[21965]: pgmap v949: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-08-20T01:26:52.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:52 smithi142 ceph-mon[21799]: pgmap v949: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-08-20T01:26:52.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:52 smithi138 ceph-mon[18076]: pgmap v949: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 623 op/s 2024-08-20T01:26:53.211 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:53 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:26:53.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87286 14.27 MB/sec execute 170 sec latency 7726.876 ms 2024-08-20T01:26:53.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:53 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:26:53.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:53 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:26:54.321 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:54 smithi138 ceph-mon[18076]: pgmap v950: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 479 op/s 2024-08-20T01:26:54.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87286 14.18 MB/sec execute 171 sec latency 8726.925 ms 2024-08-20T01:26:54.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:54 smithi167 ceph-mon[21965]: pgmap v950: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 479 op/s 2024-08-20T01:26:54.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:54 smithi142 ceph-mon[21799]: pgmap v950: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 479 op/s 2024-08-20T01:26:55.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87288 14.10 MB/sec execute 172 sec latency 9726.973 ms 2024-08-20T01:26:56.274 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:56 smithi138 ceph-mon[18076]: pgmap v951: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.4 MiB/s wr, 405 op/s 2024-08-20T01:26:56.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87390 14.07 MB/sec execute 173 sec latency 10727.023 ms 2024-08-20T01:26:56.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:56 smithi167 ceph-mon[21965]: pgmap v951: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.4 MiB/s wr, 405 op/s 2024-08-20T01:26:56.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:56 smithi142 ceph-mon[21799]: pgmap v951: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.4 MiB/s wr, 405 op/s 2024-08-20T01:26:57.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87425 13.99 MB/sec execute 174 sec latency 11727.077 ms 2024-08-20T01:26:58.216 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:26:58 smithi138 ceph-mon[18076]: pgmap v952: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 397 op/s 2024-08-20T01:26:58.358 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87425 13.91 MB/sec execute 175 sec latency 12727.156 ms 2024-08-20T01:26:58.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:26:58 smithi167 ceph-mon[21965]: pgmap v952: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 397 op/s 2024-08-20T01:26:58.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:26:58 smithi142 ceph-mon[21799]: pgmap v952: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 397 op/s 2024-08-20T01:26:59.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 87707 13.88 MB/sec execute 176 sec latency 13357.839 ms 2024-08-20T01:27:00.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 88342 13.98 MB/sec execute 177 sec latency 153.819 ms 2024-08-20T01:27:00.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:00 smithi167 ceph-mon[21965]: pgmap v953: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-08-20T01:27:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:00 smithi142 ceph-mon[21799]: pgmap v953: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-08-20T01:27:00.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:00 smithi138 ceph-mon[18076]: pgmap v953: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.9 MiB/s wr, 278 op/s 2024-08-20T01:27:01.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 88342 13.90 MB/sec execute 178 sec latency 1153.896 ms 2024-08-20T01:27:02.359 INFO:tasks.workunit.client.0.smithi138.stdout: 10 88342 13.82 MB/sec execute 179 sec latency 2154.036 ms 2024-08-20T01:27:02.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:02 smithi167 ceph-mon[21965]: pgmap v954: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-08-20T01:27:02.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:02 smithi142 ceph-mon[21799]: pgmap v954: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-08-20T01:27:02.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:02 smithi138 ceph-mon[18076]: pgmap v954: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.6 MiB/s wr, 292 op/s 2024-08-20T01:27:03.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 88342 13.75 MB/sec execute 180 sec latency 3154.170 ms 2024-08-20T01:27:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 -- 172.21.15.138:0/905952242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290113730 msgr2=0x7f1290115b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 --2- 172.21.15.138:0/905952242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290113730 0x7f1290115b20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1288009ca0 tx=0x7f128802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 -- 172.21.15.138:0/905952242 shutdown_connections 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 --2- 172.21.15.138:0/905952242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290113730 0x7f1290115b20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 --2- 172.21.15.138:0/905952242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290075b90 0x7f12901131f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 --2- 172.21.15.138:0/905952242 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f1290075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 -- 172.21.15.138:0/905952242 >> 172.21.15.138:0/905952242 conn(0x7f12900fe420 msgr2=0x7f1290100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 -- 172.21.15.138:0/905952242 shutdown_connections 2024-08-20T01:27:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.475+0000 7f1298200640 1 -- 172.21.15.138:0/905952242 wait complete. 2024-08-20T01:27:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.476+0000 7f1298200640 1 Processor -- start 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.476+0000 7f1298200640 1 -- start start 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f129019f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 0x7f129019fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290113730 0x7f12901a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f1290118ff0 con 0x7f1290075b90 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f1290118e70 con 0x7f1290113730 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1298200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f1290119170 con 0x7f1290075270 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 0x7f129019fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1295800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f129019f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1295800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f129019f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39678/0 (socket says 172.21.15.138:39678) 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1296200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290113730 0x7f12901a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1295800640 1 -- 172.21.15.138:0/2954170593 learned_addr learned my addr 172.21.15.138:0/2954170593 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 -- 172.21.15.138:0/2954170593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 msgr2=0x7f129019f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f129019f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 -- 172.21.15.138:0/2954170593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290113730 msgr2=0x7f12901a4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290113730 0x7f12901a4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.477+0000 7f1294e00640 1 -- 172.21.15.138:0/2954170593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12901a5570 con 0x7f1290075b90 2024-08-20T01:27:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f1294e00640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 0x7f129019fa10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9f0 tx=0x7f127c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f128e000640 1 -- 172.21.15.138:0/2954170593 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c00cd80 con 0x7f1290075b90 2024-08-20T01:27:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f128e000640 1 -- 172.21.15.138:0/2954170593 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f127c0035f0 con 0x7f1290075b90 2024-08-20T01:27:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f128e000640 1 -- 172.21.15.138:0/2954170593 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c010670 con 0x7f1290075b90 2024-08-20T01:27:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12901a5860 con 0x7f1290075b90 2024-08-20T01:27:03.480 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.478+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901a5c20 con 0x7f1290075b90 2024-08-20T01:27:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.481+0000 7f129b061740 1 Processor -- start 2024-08-20T01:27:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.481+0000 7f129b061740 1 -- start start 2024-08-20T01:27:03.481 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.481+0000 7f128e000640 1 -- 172.21.15.138:0/2954170593 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f127c0037a0 con 0x7f1290075b90 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557611320210 con 0x5576113ab250 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55761130f710 con 0x5576113aa930 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f129b061740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x557611313750 con 0x5576113b71e0 2024-08-20T01:27:03.483 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1294e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1295800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:56298/0 (socket says 172.21.15.138:56298) 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 -- 172.21.15.138:0/3989229092 learned_addr learned my addr 172.21.15.138:0/3989229092 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 -- 172.21.15.138:0/3989229092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 msgr2=0x5576113b95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113b95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 -- 172.21.15.138:0/3989229092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 msgr2=0x5576113b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.482+0000 7f1296200640 1 -- 172.21.15.138:0/3989229092 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576112dbde0 con 0x5576113aa930 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f1296200640 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1288009ca0 tx=0x7f12880093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2840965daf9dfc1d server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f1282a00640 1 -- 172.21.15.138:0/3989229092 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288031cf0 con 0x5576113aa930 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f1282a00640 1 -- 172.21.15.138:0/3989229092 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288036030 con 0x5576113aa930 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f1282a00640 1 -- 172.21.15.138:0/3989229092 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288003e60 con 0x5576113aa930 2024-08-20T01:27:03.484 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f128e000640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 0x7f12780b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.483+0000 7f128e000640 1 -- 172.21.15.138:0/2954170593 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7f127c0e4540 con 0x7f1290075b90 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f1295800640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 0x7f12780b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 -- 172.21.15.138:0/3989229092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 msgr2=0x5576113aad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1288009ca0 tx=0x7f12880093e0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 -- 172.21.15.138:0/3989229092 shutdown_connections 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113b95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.488 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 --2- 172.21.15.138:0/3989229092 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113aad10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 -- 172.21.15.138:0/3989229092 >> 172.21.15.138:0/3989229092 conn(0x5576113a8760 msgr2=0x5576113a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 -- 172.21.15.138:0/3989229092 shutdown_connections 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f129b061740 1 -- 172.21.15.138:0/3989229092 wait complete. 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.484+0000 7f1295800640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 0x7f12780b6280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5576113adab0 tx=0x7f128400b4a0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 -- start start 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 Processor -- start 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 0x5576113b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 0x5576113b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x557611248ab0 con 0x5576113aa930 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5576112b6140 con 0x5576113b71e0 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5576113b9ce0 con 0x5576113ab250 2024-08-20T01:27:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.485+0000 7f129b061740 10 client.0 _ll_register_callbacks cb 0x5576112c95d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57264/0 (socket says 172.21.15.138:57264) 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 -- 172.21.15.138:0/905000886 learned_addr learned my addr 172.21.15.138:0/905000886 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1296200640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 0x5576113b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1294e00640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 0x5576113b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 -- 172.21.15.138:0/905000886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 msgr2=0x5576113b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 0x5576113b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 -- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 msgr2=0x5576113b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 0x5576113b42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 -- 172.21.15.138:0/905000886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576112b7170 con 0x5576113aa930 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1294e00640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 0x5576113b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1296200640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 0x5576113b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1295800640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1284004650 tx=0x7f1284004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a40e8822dbf0e57 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1277400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.486+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128402c060 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1288036030 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7f1284017030 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f1284033050 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 10 client.17628 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x557611072790 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288036030 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f128401c510 con 0x5576113aa930 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 10 client.17628 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 10 client.17628 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 20 client.17628 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5576113c6c20 0x5576113c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 4 client.17628 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:27:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.487+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557611322160 con 0x5576113c6c20 2024-08-20T01:27:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.488+0000 7f1294e00640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5576113c6c20 0x5576113c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.488+0000 7f1294e00640 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5576113c6c20 0x5576113c9030 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37767358a5449eeb server_cookie=8983e20bbccd926c in_seq=0 out_seq=0 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.488+0000 7f1277400640 10 client.17628 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f1277400640 1 -- 172.21.15.138:0/905000886 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557611322160 con 0x5576113c6c20 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f1277400640 10 client.17628 handle_command_reply: tid=0 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 client.17628 shutdown 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 msgr2=0x5576113b6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1284004650 tx=0x7f1284004730 comp rx=0 tx=0).stop 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 shutdown_connections 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5576113c6c20 0x5576113c9030 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113b71e0 0x5576113b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113ab250 0x5576113b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 --2- 172.21.15.138:0/905000886 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113aa930 0x5576113b6930 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 >> 172.21.15.138:0/905000886 conn(0x5576113a8760 msgr2=0x5576113ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f1277400640 0 client.17628 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 shutdown_connections 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.489+0000 7f129b061740 1 -- 172.21.15.138:0/905000886 wait complete. 2024-08-20T01:27:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.490+0000 7f129b061740 20 client.17628 trim_cache size 0 max 16384 2024-08-20T01:27:03.505 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.504+0000 7f129b061740 1 Processor -- start 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 -- start start 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 0x5576113a9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b89f0 con 0x5576113b71e0 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b6260 con 0x5576113ab250 2024-08-20T01:27:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f129b061740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b40d0 con 0x5576113aa930 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1296200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1294e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 0x5576113a9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:39694/0 (socket says 172.21.15.138:39694) 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1296200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57272/0 (socket says 172.21.15.138:57272) 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 -- 172.21.15.138:0/2737599113 learned_addr learned my addr 172.21.15.138:0/2737599113 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 -- 172.21.15.138:0/2737599113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 msgr2=0x5576113a9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 0x5576113a9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 -- 172.21.15.138:0/2737599113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 msgr2=0x5576113ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1295800640 1 -- 172.21.15.138:0/2737599113 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576110eb620 con 0x5576113aa930 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1294e00640 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 0x5576113a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.505+0000 7f1296200640 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.506+0000 7f1295800640 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1284018040 tx=0x7f12840041e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c103beeab5e6797 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.506+0000 7f1282000640 1 -- 172.21.15.138:0/2737599113 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128403c070 con 0x5576113aa930 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.506+0000 7f1282000640 1 -- 172.21.15.138:0/2737599113 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128401c5f0 con 0x5576113aa930 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.506+0000 7f1282000640 1 -- 172.21.15.138:0/2737599113 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284037c90 con 0x5576113aa930 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- 172.21.15.138:0/2737599113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 msgr2=0x5576113a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1284018040 tx=0x7f12840041e0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- 172.21.15.138:0/2737599113 shutdown_connections 2024-08-20T01:27:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113b71e0 0x5576113ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113ab250 0x5576113a9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 --2- 172.21.15.138:0/2737599113 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113aa930 0x5576113a97b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- 172.21.15.138:0/2737599113 >> 172.21.15.138:0/2737599113 conn(0x5576113a8760 msgr2=0x5576113b3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- 172.21.15.138:0/2737599113 shutdown_connections 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- 172.21.15.138:0/2737599113 wait complete. 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 -- start start 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.507+0000 7f129b061740 1 Processor -- start 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113baa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b4720 con 0x5576113ab250 2024-08-20T01:27:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b5b90 con 0x5576113aa930 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f12901b4df0 con 0x5576113b71e0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f129b061740 10 client.0 _ll_register_callbacks cb 0x7f1284004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57274/0 (socket says 172.21.15.138:57274) 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1295800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 -- 172.21.15.138:0/1138091010 learned_addr learned my addr 172.21.15.138:0/1138091010 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1296200640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113baa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 -- 172.21.15.138:0/1138091010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 msgr2=0x5576113baa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113baa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 -- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 msgr2=0x5576113bf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113bf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1294e00640 1 -- 172.21.15.138:0/1138091010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576113bb230 con 0x5576113ab250 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.508+0000 7f1295800640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1294e00640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f127c00cfd0 tx=0x7f127c0a6d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2348ab6762b7e843 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127c0af060 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 9453+0+0 (secure 0 0 0) 0x7f127c009310 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f128401c5f0 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f127c0aad50 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 10 client.17634 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5576114316b0 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128401c5f0 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7f127c0a6fe0 con 0x5576113ab250 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 10 client.17634 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 10 client.17634 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 20 client.17634 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557611431e00 0x557611434220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 4 client.17634 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:27:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.509+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55761131dc00 con 0x557611431e00 2024-08-20T01:27:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.510+0000 7f1295800640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557611431e00 0x557611434220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.510+0000 7f1295800640 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557611431e00 0x557611434220 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e775e1ca3aacec2 server_cookie=4d4df0803a9a5ae4 in_seq=0 out_seq=0 2024-08-20T01:27:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.510+0000 7f1277400640 10 client.17634 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:03.632 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.632+0000 7f1277400640 1 -- 172.21.15.138:0/1138091010 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55761131dc00 con 0x557611431e00 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.632+0000 7f1277400640 10 client.17634 handle_command_reply: tid=0 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.632+0000 7f129b061740 1 client.17634 shutdown 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.632+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 msgr2=0x5576113bf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.632+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f127c00cfd0 tx=0x7f127c0a6d30 comp rx=0 tx=0).stop 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 shutdown_connections 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x557611431e00 0x557611434220 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x5576113b71e0 0x5576113baa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x5576113ab250 0x5576113bf870 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 --2- 172.21.15.138:0/1138091010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5576113aa930 0x5576113bf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f1277400640 0 client.17634 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:03.633 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.633+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 >> 172.21.15.138:0/1138091010 conn(0x5576113a8760 msgr2=0x5576113b53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.634+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 shutdown_connections 2024-08-20T01:27:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.634+0000 7f129b061740 1 -- 172.21.15.138:0/1138091010 wait complete. 2024-08-20T01:27:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.634+0000 7f129b061740 20 client.17634 trim_cache size 0 max 16384 2024-08-20T01:27:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.635+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 msgr2=0x7f12780b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.635+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 0x7f12780b6280 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5576113adab0 tx=0x7f128400b4a0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.635+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 msgr2=0x7f129019fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.635+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 0x7f129019fa10 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f127c00e9f0 tx=0x7f127c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 shutdown_connections 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f12780b3dc0 0x7f12780b6280 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1290113730 0x7f12901a4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1290075b90 0x7f129019fa10 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 --2- 172.21.15.138:0/2954170593 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1290075270 0x7f129019f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 >> 172.21.15.138:0/2954170593 conn(0x7f12900fe420 msgr2=0x7f12900ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 shutdown_connections 2024-08-20T01:27:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:03.636+0000 7f1298200640 1 -- 172.21.15.138:0/2954170593 wait complete. 2024-08-20T01:27:04.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 88806 13.82 MB/sec execute 181 sec latency 3449.803 ms 2024-08-20T01:27:04.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:04 smithi167 ceph-mon[21965]: pgmap v955: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-08-20T01:27:04.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:04 smithi142 ceph-mon[21799]: pgmap v955: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-08-20T01:27:04.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:04 smithi138 ceph-mon[18076]: pgmap v955: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 186 op/s 2024-08-20T01:27:05.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 89110 13.85 MB/sec execute 182 sec latency 416.786 ms 2024-08-20T01:27:06.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 89187 13.80 MB/sec execute 183 sec latency 1076.191 ms 2024-08-20T01:27:06.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:06 smithi167 ceph-mon[21965]: pgmap v956: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-08-20T01:27:06.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:06 smithi167 ceph-mon[21965]: osdmap e114: 12 total, 12 up, 12 in 2024-08-20T01:27:06.492 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:06 smithi142 ceph-mon[21799]: pgmap v956: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-08-20T01:27:06.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:06 smithi142 ceph-mon[21799]: osdmap e114: 12 total, 12 up, 12 in 2024-08-20T01:27:06.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:06 smithi138 ceph-mon[18076]: pgmap v956: 129 pgs: 129 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.5 MiB/s wr, 139 op/s 2024-08-20T01:27:06.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:06 smithi138 ceph-mon[18076]: osdmap e114: 12 total, 12 up, 12 in 2024-08-20T01:27:07.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 89250 13.74 MB/sec execute 184 sec latency 1686.293 ms 2024-08-20T01:27:08.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 89395 13.71 MB/sec execute 185 sec latency 1258.740 ms 2024-08-20T01:27:08.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:08 smithi167 ceph-mon[21965]: pgmap v958: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-08-20T01:27:08.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:08 smithi142 ceph-mon[21799]: pgmap v958: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-08-20T01:27:08.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:08 smithi138 ceph-mon[18076]: pgmap v958: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.6 MiB/s wr, 258 op/s 2024-08-20T01:27:09.360 INFO:tasks.workunit.client.0.smithi138.stdout: 10 89680 13.70 MB/sec execute 186 sec latency 508.250 ms 2024-08-20T01:27:10.175 INFO:tasks.workunit.client.0.smithi138.stdout:[93794] write failed on handle 12938 (Resource temporarily unavailable) 2024-08-20T01:27:10.176 INFO:tasks.workunit.client.0.smithi138.stdout:Child failed with status 1 2024-08-20T01:27:10.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:27:10.196 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-08-20T01:27:10.196 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-08-20T01:27:10.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:10 smithi167 ceph-mon[21965]: pgmap v959: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-08-20T01:27:10.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:10 smithi142 ceph-mon[21799]: pgmap v959: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-08-20T01:27:10.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:10 smithi138 ceph-mon[18076]: pgmap v959: 129 pgs: 11 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 85 active+clean; 1.1 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.6 MiB/s wr, 268 op/s 2024-08-20T01:27:11.309 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi138 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-20T01:27:13.790 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=595f38d07202411191bcc1cec4ad1ea9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_ceph_ceph-c_ef4275410d8483698016fe484b2a3c664080b4c1/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_67db276914d29505c9707d04f3a3b8fb9d8aee4b/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi138 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-08-20T01:27:13.792 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-08-20T01:27:13.804 INFO:tasks.quiescer:joining Quiescers 2024-08-20T01:27:13.804 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-08-20T01:27:13.804 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:27:13.809 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:12 smithi142 ceph-mon[21799]: pgmap v960: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-08-20T01:27:13.810 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:12 smithi167 ceph-mon[21965]: pgmap v960: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-08-20T01:27:13.810 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:12 smithi138 ceph-mon[18076]: pgmap v960: 129 pgs: 5 active+clean+snaptrim, 13 active+clean+snaptrim_wait, 111 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-08-20T01:27:14.194 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:14.403 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:14 smithi138 ceph-mon[18076]: pgmap v961: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-08-20T01:27:14.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:14 smithi167 ceph-mon[21965]: pgmap v961: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-08-20T01:27:14.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:14 smithi142 ceph-mon[21799]: pgmap v961: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 2.8 MiB/s wr, 366 op/s 2024-08-20T01:27:15.090 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1164151477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075b90 msgr2=0x7f7a20113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1164151477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075b90 0x7f7a20113130 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c009900 tx=0x7f7a0c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1164151477 shutdown_connections 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1164151477 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 0x7f7a20115a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1164151477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075b90 0x7f7a20113130 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1164151477 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075270 0x7f7a20075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1164151477 >> 172.21.15.138:0/1164151477 conn(0x7f7a200fe3c0 msgr2=0x7f7a201007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:15.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1164151477 shutdown_connections 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.813+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1164151477 wait complete. 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.814+0000 7f7a1fe00640 1 Processor -- start 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 -- start start 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 0x7f7a2019f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 0x7f7a201a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20118f40 con 0x7f7a20113670 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20118dc0 con 0x7f7a20075270 2024-08-20T01:27:15.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201190c0 con 0x7f7a20075b90 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 0x7f7a2019f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1ea00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:54226/0 (socket says 172.21.15.138:54226) 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1e000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 0x7f7a2019f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42302/0 (socket says 172.21.15.138:42302) 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1ea00640 1 -- 172.21.15.138:0/1230876558 learned_addr learned my addr 172.21.15.138:0/1230876558 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.815+0000 7f7a1f400640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 0x7f7a201a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 -- 172.21.15.138:0/1230876558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 msgr2=0x7f7a2019f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:15.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 0x7f7a2019f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 -- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 msgr2=0x7f7a201a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 0x7f7a201a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 -- 172.21.15.138:0/1230876558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a201a53e0 con 0x7f7a20075270 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a1ea00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000e9f0 tx=0x7f7a1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000cd80 con 0x7f7a20075270 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a100108e0 con 0x7f7a20075270 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.816+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10018980 con 0x7f7a20075270 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.817+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a201a56d0 con 0x7f7a20075270 2024-08-20T01:27:15.094 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.817+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a201abd90 con 0x7f7a20075270 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.818+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a20075650 con 0x7f7a20075270 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.820+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7a10010430 con 0x7f7a20075270 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.821+0000 7f7a17400640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 0x7f7a040b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.821+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9541+0+0 (secure 0 0 0) 0x7f7a100e4700 con 0x7f7a20075270 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.822+0000 7f7a1e000640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 0x7f7a040b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.822+0000 7f7a1e000640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 0x7f7a040b6320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c005ab0 tx=0x7f7a0c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:14.825+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7a100aaf60 con 0x7f7a20075270 2024-08-20T01:27:15.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.056+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a201ac110 con 0x7f7a20075270 2024-08-20T01:27:15.096 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.057+0000 7f7a17400640 1 -- 172.21.15.138:0/1230876558 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+11938 (secure 0 0 0) 0x7f7a100af4b0 con 0x7f7a20075270 2024-08-20T01:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:15.096 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":55,"btime":"2024-08-20T01:26:52:078913+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":55,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:26:51.187459+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:27:15.096 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 55 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 msgr2=0x7f7a040b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 0x7f7a040b6320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7a0c005ab0 tx=0x7f7a0c005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 msgr2=0x7f7a2019f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000e9f0 tx=0x7f7a1000eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 shutdown_connections 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7a040b3e60 0x7f7a040b6320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a20113670 0x7f7a201a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7a20075b90 0x7f7a2019f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.097 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 --2- 172.21.15.138:0/1230876558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7a20075270 0x7f7a2019f330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:15.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 >> 172.21.15.138:0/1230876558 conn(0x7f7a200fe3c0 msgr2=0x7f7a200fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:15.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 shutdown_connections 2024-08-20T01:27:15.098 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:15.060+0000 7f7a1fe00640 1 -- 172.21.15.138:0/1230876558 wait complete. 2024-08-20T01:27:15.464 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:15 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:27:15.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:15 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1230876558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:15.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:15 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:27:15.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:15 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1230876558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:15.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:15 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-08-20T01:27:15.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:15 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1230876558' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 -- 172.21.15.138:0/3138872183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075270 msgr2=0x7ff1ac075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3138872183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075270 0x7ff1ac075650 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0009900 tx=0x7ff1a002a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 -- 172.21.15.138:0/3138872183 shutdown_connections 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3138872183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1113b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3138872183 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac075c20 0x7ff1ac10ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.770 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3138872183 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075270 0x7ff1ac075650 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.771 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 -- 172.21.15.138:0/3138872183 >> 172.21.15.138:0/3138872183 conn(0x7ff1ac0fe480 msgr2=0x7ff1ac1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.768+0000 7ff1b4800640 1 -- 172.21.15.138:0/3138872183 shutdown_connections 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.769+0000 7ff1b4800640 1 -- 172.21.15.138:0/3138872183 wait complete. 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.769+0000 7ff1b4800640 1 Processor -- start 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 -- start start 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 0x7ff1ac19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac114c10 con 0x7ff1ac075c20 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac114a90 con 0x7ff1ac075270 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b4800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac114d90 con 0x7ff1ac10ef80 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 0x7ff1ac19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:54248/0 (socket says 172.21.15.138:54248) 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b1e00640 1 -- 172.21.15.138:0/3502782845 learned_addr learned my addr 172.21.15.138:0/3502782845 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.770+0000 7ff1b2800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 -- 172.21.15.138:0/3502782845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 msgr2=0x7ff1ac1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 -- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 msgr2=0x7ff1ac19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 0x7ff1ac19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 -- 172.21.15.138:0/3502782845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac1a1250 con 0x7ff1ac075270 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b2800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1400640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 0x7ff1ac19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0005ab0 tx=0x7ff1a0003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1a6a00640 1 -- 172.21.15.138:0/3502782845 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a003f070 con 0x7ff1ac075270 2024-08-20T01:27:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.771+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1ac1a14e0 con 0x7ff1ac075270 2024-08-20T01:27:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.772+0000 7ff1a6a00640 1 -- 172.21.15.138:0/3502782845 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a0031b60 con 0x7ff1ac075270 2024-08-20T01:27:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.772+0000 7ff1a6a00640 1 -- 172.21.15.138:0/3502782845 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0035030 con 0x7ff1ac075270 2024-08-20T01:27:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.772+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1ac1a7c00 con 0x7ff1ac075270 2024-08-20T01:27:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.775+0000 7ff1a6a00640 1 -- 172.21.15.138:0/3502782845 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7ff1a0042070 con 0x7ff1ac075270 2024-08-20T01:27:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.776+0000 7ff1b763c740 1 Processor -- start 2024-08-20T01:27:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.776+0000 7ff1b763c740 1 -- start start 2024-08-20T01:27:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a2eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a2faca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55918a264210 con 0x55918a2ee930 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55918a253710 con 0x55918a2fb1e0 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b763c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55918a257750 con 0x55918a2ef250 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a2eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a2faca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1a6a00640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 0x7ff1940b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:54258/0 (socket says 172.21.15.138:54258) 2024-08-20T01:27:16.780 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 -- 172.21.15.138:0/3297432808 learned_addr learned my addr 172.21.15.138:0/3297432808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 -- 172.21.15.138:0/3297432808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 msgr2=0x55918a2faca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a2faca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1a6a00640 1 -- 172.21.15.138:0/3502782845 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7ff1a0104b90 con 0x7ff1ac075270 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 -- 172.21.15.138:0/3297432808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 msgr2=0x55918a2eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a2eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b2800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 0x7ff1940b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.777+0000 7ff1b1400640 1 -- 172.21.15.138:0/3297432808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918a21fde0 con 0x55918a2fb1e0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.778+0000 7ff1b1400640 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0114f0 tx=0x7ff19c0119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb28ff79f4994c5 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.778+0000 7ff19b000640 1 -- 172.21.15.138:0/3297432808 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c004020 con 0x55918a2fb1e0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.778+0000 7ff19b000640 1 -- 172.21.15.138:0/3297432808 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c0041c0 con 0x55918a2fb1e0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.778+0000 7ff19b000640 1 -- 172.21.15.138:0/3297432808 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c00f860 con 0x55918a2fb1e0 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 -- 172.21.15.138:0/3297432808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 msgr2=0x55918a2fd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0114f0 tx=0x7ff19c0119b0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b2800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 0x7ff1940b6200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80099a0 tx=0x7ff1a8004ab0 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 -- 172.21.15.138:0/3297432808 shutdown_connections 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2fd5d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a2faca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 --2- 172.21.15.138:0/3297432808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a2eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 -- 172.21.15.138:0/3297432808 >> 172.21.15.138:0/3297432808 conn(0x55918a2ec760 msgr2=0x55918a2ed900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 -- 172.21.15.138:0/3297432808 shutdown_connections 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.779+0000 7ff1b763c740 1 -- 172.21.15.138:0/3297432808 wait complete. 2024-08-20T01:27:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 -- start start 2024-08-20T01:27:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 Processor -- start 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 0x55918a2f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x55918a18cab0 con 0x55918a2ef250 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x55918a1fa140 con 0x55918a2fb1e0 2024-08-20T01:27:16.787 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x55918a2fdec0 con 0x55918a2ee930 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b763c740 10 client.0 _ll_register_callbacks cb 0x55918a20d5d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 0x55918a2f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.780+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42356/0 (socket says 172.21.15.138:42356) 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 0x55918a2f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39306/0 (socket says 172.21.15.138:39306) 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 -- 172.21.15.138:0/463057573 learned_addr learned my addr 172.21.15.138:0/463057573 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b2800640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 -- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 msgr2=0x55918a2f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.788 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 -- 172.21.15.138:0/463057573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 msgr2=0x55918a2f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 0x55918a2f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 -- 172.21.15.138:0/463057573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55918a1fb170 con 0x55918a2ee930 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0000c00 tx=0x7ff1a0009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c94be66e0894c8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff18f400640 10 client.0 ms_handle_connect on v2:172.21.15.167:3300/0 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0049030 con 0x55918a2ee930 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff19c0041c0 con 0x55918a2ee930 2024-08-20T01:27:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.781+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mon.1 v2:172.21.15.167:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7ff1a0102160 con 0x55918a2ee930 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1a0109040 con 0x55918a2ee930 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 10 client.27661 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x559189fb6790 con 0x55918a2ee930 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mon.1 v2:172.21.15.167:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff19c0041c0 con 0x55918a2ee930 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mon.1 v2:172.21.15.167:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7ff1a00d3fc0 con 0x55918a2ee930 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 10 client.27661 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 10 client.27661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:27:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 20 client.27661 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a30ad30 0x55918a30d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 4 client.27661 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.782+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55918a266160 con 0x55918a30ad30 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.783+0000 7ff1b1400640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a30ad30 0x55918a30d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.783+0000 7ff18f400640 10 client.27661 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.783+0000 7ff1b1400640 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a30ad30 0x55918a30d1d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d2182b971960bd6 server_cookie=8bff986142de0a2f in_seq=0 out_seq=0 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff18f400640 1 -- 172.21.15.138:0/463057573 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55918a266160 con 0x55918a30ad30 2024-08-20T01:27:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff18f400640 10 client.27661 handle_command_reply: tid=0 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 client.27661 shutdown 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 msgr2=0x55918a2fa960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0000c00 tx=0x7ff1a0009620 comp rx=0 tx=0).stop 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 shutdown_connections 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a30ad30 0x55918a30d1d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff18f400640 0 client.27661 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ef250 0x55918a2f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.792 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 --2- 172.21.15.138:0/463057573 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ee930 0x55918a2fa960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 >> 172.21.15.138:0/463057573 conn(0x55918a2ec760 msgr2=0x55918a2f2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.784+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 shutdown_connections 2024-08-20T01:27:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.785+0000 7ff1b763c740 1 -- 172.21.15.138:0/463057573 wait complete. 2024-08-20T01:27:16.793 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.785+0000 7ff1b763c740 20 client.27661 trim_cache size 0 max 16384 2024-08-20T01:27:16.804 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24743 quiesce db --cancel --all' 2024-08-20T01:27:16.805 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- timeout 105 ceph tell mds.24743 quiesce db --cancel --all 2024-08-20T01:27:16.810 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.807+0000 7ff1b763c740 1 Processor -- start 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 -- start start 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a312ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a313020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b4620 con 0x55918a2ee930 2024-08-20T01:27:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b1e90 con 0x55918a2fb1e0 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b763c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1afd00 con 0x55918a2ef250 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a312ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b1e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a312ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39310/0 (socket says 172.21.15.138:39310) 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b1e00640 1 -- 172.21.15.138:0/2379161031 learned_addr learned my addr 172.21.15.138:0/2379161031 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.808+0000 7ff1b1400640 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a313020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 -- 172.21.15.138:0/2379161031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 msgr2=0x55918a313020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a313020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 -- 172.21.15.138:0/2379161031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 msgr2=0x55918a312ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a312ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 -- 172.21.15.138:0/2379161031 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918a02f620 con 0x55918a2fb1e0 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f2c50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8005fd0 tx=0x7ff1a8014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a286ea0a04d6e8f server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff19a600640 1 -- 172.21.15.138:0/2379161031 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a8012d40 con 0x55918a2fb1e0 2024-08-20T01:27:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff19a600640 1 -- 172.21.15.138:0/2379161031 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a801d900 con 0x55918a2fb1e0 2024-08-20T01:27:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.809+0000 7ff19a600640 1 -- 172.21.15.138:0/2379161031 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a80054c0 con 0x55918a2fb1e0 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- 172.21.15.138:0/2379161031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 msgr2=0x55918a2f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f2c50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8005fd0 tx=0x7ff1a8014040 comp rx=0 tx=0).stop 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- 172.21.15.138:0/2379161031 shutdown_connections 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2fb1e0 0x55918a2f2c50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a313020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2379161031 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2ee930 0x55918a312ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- 172.21.15.138:0/2379161031 >> 172.21.15.138:0/2379161031 conn(0x55918a2ec760 msgr2=0x55918a2f6600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- 172.21.15.138:0/2379161031 shutdown_connections 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- 172.21.15.138:0/2379161031 wait complete. 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.810+0000 7ff1b763c740 1 -- start start 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 Processor -- start 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 0x55918a2fd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a3031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b0350 con 0x55918a2fb1e0 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b17c0 con 0x55918a2ee930 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7ff1ac1b0a20 con 0x55918a2ef250 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b763c740 10 client.0 _ll_register_callbacks cb 0x7ff1a0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b2800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:39324/0 (socket says 172.21.15.138:39324) 2024-08-20T01:27:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b2800640 1 -- 172.21.15.138:0/2348026809 learned_addr learned my addr 172.21.15.138:0/2348026809 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.811+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 0x55918a2fd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b1400640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a3031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 -- 172.21.15.138:0/2348026809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 msgr2=0x55918a3031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a3031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 -- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 msgr2=0x55918a2fd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 0x55918a2fd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 -- 172.21.15.138:0/2348026809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55918a2fe2f0 con 0x55918a2fb1e0 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 0x55918a2fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b2800640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009540 tx=0x7ff1a8013f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd8a0cb46e7b013 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff18f400640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a801dd40 con 0x55918a2fb1e0 2024-08-20T01:27:16.821 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7ff1a8023040 con 0x55918a2fb1e0 2024-08-20T01:27:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.812+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1a801d900 con 0x55918a2fb1e0 2024-08-20T01:27:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7ff1a803d050 con 0x55918a2fb1e0 2024-08-20T01:27:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 10 client.17658 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55918a375f60 con 0x55918a2fb1e0 2024-08-20T01:27:16.822 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1a801d900 con 0x55918a2fb1e0 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7ff1a8019040 con 0x55918a2fb1e0 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 10 client.17658 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 10 client.17658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 20 client.17658 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a3766d0 0x55918a378af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 4 client.17658 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-08-20T01:27:16.823 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.813+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55918a261c00 con 0x55918a3766d0 2024-08-20T01:27:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.814+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a3766d0 0x55918a378af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.814+0000 7ff1b1e00640 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a3766d0 0x55918a378af0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa67154a01d07a0 server_cookie=efabc9ee3000f0c6 in_seq=0 out_seq=0 2024-08-20T01:27:16.824 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:16.814+0000 7ff18f400640 10 client.17658 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:17.016 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:16 smithi138 ceph-mon[18076]: pgmap v962: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 360 op/s 2024-08-20T01:27:17.017 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:16 smithi138 ceph-mon[18076]: osdmap e115: 12 total, 12 up, 12 in 2024-08-20T01:27:17.017 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:27:17.017 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:16 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:27:17.017 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:16 smithi138 ceph-mon[18076]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:27:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.120+0000 7ff18f400640 1 -- 172.21.15.138:0/2348026809 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55918a261c00 con 0x55918a3766d0 2024-08-20T01:27:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.120+0000 7ff18f400640 10 client.17658 handle_command_reply: tid=0 2024-08-20T01:27:17.134 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.133+0000 7ff1b763c740 1 client.17658 shutdown 2024-08-20T01:27:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.135+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 msgr2=0x55918a303960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.135+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009540 tx=0x7ff1a8013f80 comp rx=0 tx=0).stop 2024-08-20T01:27:17.145 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 shutdown_connections 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x55918a3766d0 0x55918a378af0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff18f400640 0 client.17658 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x55918a2fb1e0 0x55918a303960 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x55918a2ef250 0x55918a3031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 --2- 172.21.15.138:0/2348026809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55918a2ee930 0x55918a2fd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.146 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.144+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 >> 172.21.15.138:0/2348026809 conn(0x55918a2ec760 msgr2=0x55918a2f2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.145+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 shutdown_connections 2024-08-20T01:27:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.145+0000 7ff1b763c740 1 -- 172.21.15.138:0/2348026809 wait complete. 2024-08-20T01:27:17.147 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.145+0000 7ff1b763c740 20 client.17658 trim_cache size 0 max 16384 2024-08-20T01:27:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:{ 2024-08-20T01:27:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "message": "", 2024-08-20T01:27:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout: "return_code": 0 2024-08-20T01:27:17.148 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stdout:} 2024-08-20T01:27:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.147+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 msgr2=0x7ff1940b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.147+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 0x7ff1940b6200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80099a0 tx=0x7ff1a8004ab0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.147+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 msgr2=0x7ff1ac19b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.147+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7ff1a0005ab0 tx=0x7ff1a0003c10 comp rx=0 tx=0).stop 2024-08-20T01:27:17.149 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 shutdown_connections 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7ff1940b3d40 0x7ff1940b6200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff1ac10ef80 0x7ff1ac1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1ac075c20 0x7ff1ac19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 --2- 172.21.15.138:0/3502782845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff1ac075270 0x7ff1ac19b1b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 >> 172.21.15.138:0/3502782845 conn(0x7ff1ac0fe480 msgr2=0x7ff1ac0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 shutdown_connections 2024-08-20T01:27:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi138.front.sepia.ceph.com.smithi138.stderr:2024-08-20T01:27:17.148+0000 7ff1b4800640 1 -- 172.21.15.138:0/3502782845 wait complete. 2024-08-20T01:27:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:16 smithi167 ceph-mon[21965]: pgmap v962: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 360 op/s 2024-08-20T01:27:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:16 smithi167 ceph-mon[21965]: osdmap e115: 12 total, 12 up, 12 in 2024-08-20T01:27:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:27:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:16 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:27:17.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:16 smithi167 ceph-mon[21965]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:27:17.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:16 smithi142 ceph-mon[21799]: pgmap v962: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 3.0 MiB/s wr, 360 op/s 2024-08-20T01:27:17.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:16 smithi142 ceph-mon[21799]: osdmap e115: 12 total, 12 up, 12 in 2024-08-20T01:27:17.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-08-20T01:27:17.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:16 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-08-20T01:27:17.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:16 smithi142 ceph-mon[21799]: from='mgr.14220 ' entity='mgr.x' 2024-08-20T01:27:17.399 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 -- 172.21.15.138:0/989899314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 msgr2=0x7fdc701132f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 --2- 172.21.15.138:0/989899314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc701132f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009900 tx=0x7fdc5c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.162 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 -- 172.21.15.138:0/989899314 shutdown_connections 2024-08-20T01:27:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 --2- 172.21.15.138:0/989899314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70113830 0x7fdc70115c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 --2- 172.21.15.138:0/989899314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc701132f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 --2- 172.21.15.138:0/989899314 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70075270 0x7fdc70075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.163 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 -- 172.21.15.138:0/989899314 >> 172.21.15.138:0/989899314 conn(0x7fdc700fe520 msgr2=0x7fdc70100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 -- 172.21.15.138:0/989899314 shutdown_connections 2024-08-20T01:27:18.164 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.160+0000 7fdc76200640 1 -- 172.21.15.138:0/989899314 wait complete. 2024-08-20T01:27:18.165 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.161+0000 7fdc76200640 1 Processor -- start 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.161+0000 7fdc76200640 1 -- start start 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc7019fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 0x7fdc701a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70119110 con 0x7fdc70075b90 2024-08-20T01:27:18.166 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70118f90 con 0x7fdc70075270 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc76200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70119290 con 0x7fdc70113830 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc6f400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36314/0 (socket says 172.21.15.138:36314) 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc6f400640 1 -- 172.21.15.138:0/1254993142 learned_addr learned my addr 172.21.15.138:0/1254993142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 0x7fdc701a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.162+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc7019fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.167 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 -- 172.21.15.138:0/1254993142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 msgr2=0x7fdc701a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 0x7fdc701a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 -- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 msgr2=0x7fdc7019fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc7019fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 -- 172.21.15.138:0/1254993142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc701a5670 con 0x7fdc70075270 2024-08-20T01:27:18.168 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 0x7fdc701a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc6f400640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000eb90 tx=0x7fdc6000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc67e00640 1 -- 172.21.15.138:0/1254993142 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60004d20 con 0x7fdc70075270 2024-08-20T01:27:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc701a5960 con 0x7fdc70075270 2024-08-20T01:27:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc67e00640 1 -- 172.21.15.138:0/1254993142 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc6000ee90 con 0x7fdc70075270 2024-08-20T01:27:18.169 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc67e00640 1 -- 172.21.15.138:0/1254993142 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc60010670 con 0x7fdc70075270 2024-08-20T01:27:18.170 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.163+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701ac020 con 0x7fdc70075270 2024-08-20T01:27:18.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.166+0000 7fdc78e74740 1 Processor -- start 2024-08-20T01:27:18.175 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 -- start start 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x562037441cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5620373a9950 con 0x56203743f8d0 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x562037398e50 con 0x562037433020 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc78e74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x56203739ce90 con 0x562037433940 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc67e00640 1 -- 172.21.15.138:0/1254993142 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fdc60005020 con 0x7fdc70075270 2024-08-20T01:27:18.176 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x562037441cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36318/0 (socket says 172.21.15.138:36318) 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.167+0000 7fdc6fe00640 1 -- 172.21.15.138:0/4146382881 learned_addr learned my addr 172.21.15.138:0/4146382881 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 -- 172.21.15.138:0/4146382881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 msgr2=0x56203743f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6f400640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 -- 172.21.15.138:0/4146382881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 msgr2=0x562037441cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.177 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x562037441cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 -- 172.21.15.138:0/4146382881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562037368e80 con 0x562037433020 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6f400640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x562037441cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011470 tx=0x7fdc68011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99e6aac0061004e8 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc5be00640 1 -- 172.21.15.138:0/4146382881 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68004020 con 0x562037433020 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc5be00640 1 -- 172.21.15.138:0/4146382881 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc680041c0 con 0x562037433020 2024-08-20T01:27:18.178 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.168+0000 7fdc5be00640 1 -- 172.21.15.138:0/4146382881 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6800f8b0 con 0x562037433020 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc67e00640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 0x7fdc540b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc67e00640 1 -- 172.21.15.138:0/1254993142 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fdc600e49a0 con 0x7fdc70075270 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 -- 172.21.15.138:0/4146382881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 msgr2=0x562037433400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdc68011470 tx=0x7fdc68011930 comp rx=0 tx=0).stop 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc6f400640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 0x7fdc540b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 -- 172.21.15.138:0/4146382881 shutdown_connections 2024-08-20T01:27:18.179 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x562037441cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 --2- 172.21.15.138:0/4146382881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037433400 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.169+0000 7fdc78e74740 1 -- 172.21.15.138:0/4146382881 >> 172.21.15.138:0/4146382881 conn(0x562037430e50 msgr2=0x562037431ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.170+0000 7fdc78e74740 1 -- 172.21.15.138:0/4146382881 shutdown_connections 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.170+0000 7fdc78e74740 1 -- 172.21.15.138:0/4146382881 wait complete. 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.170+0000 7fdc6f400640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 0x7fdc540b6280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x562037431d30 tx=0x7fdc6000cf90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.170+0000 7fdc78e74740 1 -- start start 2024-08-20T01:27:18.180 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.170+0000 7fdc78e74740 1 Processor -- start 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x56203743cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x5620371fb810 con 0x56203743f8d0 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x5620373340d0 con 0x562037433020 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x5620374425f0 con 0x562037433940 2024-08-20T01:27:18.181 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc78e74740 10 client.0 _ll_register_callbacks cb 0x562037344e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36326/0 (socket says 172.21.15.138:36326) 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46380/0 (socket says 172.21.15.138:46380) 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6fe00640 1 -- 172.21.15.138:0/924097808 learned_addr learned my addr 172.21.15.138:0/924097808 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.171+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x56203743cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.182 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 -- 172.21.15.138:0/924097808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 msgr2=0x56203743c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 -- 172.21.15.138:0/924097808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 msgr2=0x56203743cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x56203743cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 -- 172.21.15.138:0/924097808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562037365ee0 con 0x562037433020 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800f5e0 tx=0x7fdc68003e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e0f8f36a6c9f1fb server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc680055c0 con 0x562037433020 2024-08-20T01:27:18.183 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mon.2 v2:172.21.15.142:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fdc6801f040 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc680041c0 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdc68036050 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc78e74740 10 client.27716 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.172+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5620370f88b0 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mon.2 v2:172.21.15.142:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc680041c0 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mon.2 v2:172.21.15.142:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fdc68004990 con 0x562037433020 2024-08-20T01:27:18.184 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 10 client.27716 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 10 client.27716 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 20 client.27716 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56203744f2b0 0x5620374516a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 4 client.27716 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620373ab8a0 con 0x56203744f2b0 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.173+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56203744f2b0 0x5620374516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.174+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56203744f2b0 0x5620374516a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf1c7c1bf1c19907 server_cookie=17e578ced5273300 in_seq=0 out_seq=0 2024-08-20T01:27:18.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.174+0000 7fdc58c00640 10 client.27716 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:18.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.174+0000 7fdc58c00640 1 -- 172.21.15.138:0/924097808 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5620373ab8a0 con 0x56203744f2b0 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.174+0000 7fdc58c00640 10 client.27716 handle_command_reply: tid=0 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.174+0000 7fdc78e74740 1 client.27716 shutdown 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 msgr2=0x56203743f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800f5e0 tx=0x7fdc68003e70 comp rx=0 tx=0).stop 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 shutdown_connections 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x56203744f2b0 0x5620374516a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x56203743f8d0 0x56203743cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.187 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433940 0x56203743c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 --2- 172.21.15.138:0/924097808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x56203743f340 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 >> 172.21.15.138:0/924097808 conn(0x562037430e50 msgr2=0x5620374369c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc58c00640 0 client.27716 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 shutdown_connections 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 1 -- 172.21.15.138:0/924097808 wait complete. 2024-08-20T01:27:18.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.175+0000 7fdc78e74740 20 client.27716 trim_cache size 0 max 16384 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.192+0000 7fdc78e74740 1 Processor -- start 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.192+0000 7fdc78e74740 1 -- start start 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56203743f8d0 0x56203744b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.194 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b8b10 con 0x562037433940 2024-08-20T01:27:18.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b6380 con 0x562037433020 2024-08-20T01:27:18.195 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc78e74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b41f0 con 0x56203743f8d0 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34214/0 (socket says 172.21.15.138:34214) 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36336/0 (socket says 172.21.15.138:36336) 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 -- 172.21.15.138:0/3049280801 learned_addr learned my addr 172.21.15.138:0/3049280801 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56203743f8d0 0x56203744b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.197 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 -- 172.21.15.138:0/3049280801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56203743f8d0 msgr2=0x56203744b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56203743f8d0 0x56203744b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 -- 172.21.15.138:0/3049280801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 msgr2=0x562037430620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6ea00640 1 -- 172.21.15.138:0/3049280801 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562037451a10 con 0x562037433940 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.193+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.194+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005e00 tx=0x7fdc5c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abf0e451f881e17 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.194+0000 7fdc5b400640 1 -- 172.21.15.138:0/3049280801 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c035040 con 0x562037433940 2024-08-20T01:27:18.198 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.194+0000 7fdc5b400640 1 -- 172.21.15.138:0/3049280801 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c04b070 con 0x562037433940 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.194+0000 7fdc5b400640 1 -- 172.21.15.138:0/3049280801 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0035f0 con 0x562037433940 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- 172.21.15.138:0/3049280801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 msgr2=0x56203744ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c005e00 tx=0x7fdc5c005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- 172.21.15.138:0/3049280801 shutdown_connections 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x56203743f8d0 0x56203744b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x56203744ae70 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 --2- 172.21.15.138:0/3049280801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562037433020 0x562037430620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- 172.21.15.138:0/3049280801 >> 172.21.15.138:0/3049280801 conn(0x562037430e50 msgr2=0x56203744a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.199 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- 172.21.15.138:0/3049280801 shutdown_connections 2024-08-20T01:27:18.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- 172.21.15.138:0/3049280801 wait complete. 2024-08-20T01:27:18.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 -- start start 2024-08-20T01:27:18.200 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.195+0000 7fdc78e74740 1 Processor -- start 2024-08-20T01:27:18.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433020 0x5620374478f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.202 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56203743f8d0 0x5620374428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b4840 con 0x562037433940 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b5cb0 con 0x56203743f8d0 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdc701b4f10 con 0x562037433020 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc78e74740 10 client.0 _ll_register_callbacks cb 0x7fdc68005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.203 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc6ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34216/0 (socket says 172.21.15.138:34216) 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc6ea00640 1 -- 172.21.15.138:0/1746970961 learned_addr learned my addr 172.21.15.138:0/1746970961 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56203743f8d0 0x5620374428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.196+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433020 0x5620374478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 -- 172.21.15.138:0/1746970961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433020 msgr2=0x5620374478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433020 0x5620374478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 -- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56203743f8d0 msgr2=0x5620374428a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56203743f8d0 0x5620374428a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 -- 172.21.15.138:0/1746970961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562037335170 con 0x562037433940 2024-08-20T01:27:18.204 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc6ea00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02a910 tx=0x7fdc5c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6ad7cd7753fe89 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc58c00640 10 client.0 ms_handle_connect on v2:172.21.15.138:3300/0 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c053070 con 0x562037433940 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mon.0 v2:172.21.15.138:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fdc5c004fa0 con 0x562037433940 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5c04b070 con 0x562037433940 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.197+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c035040 con 0x562037433940 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 10 client.17676 fetch_fsmap learned FSMap version 55 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x5620374ba480 con 0x562037433940 2024-08-20T01:27:18.205 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mon.0 v2:172.21.15.138:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5c04b070 con 0x562037433940 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mon.0 v2:172.21.15.138:3300/0 5 ==== fsmap(e 55) ==== 8370+0+0 (secure 0 0 0) 0x7fdc5c003bc0 con 0x562037433940 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 10 client.17676 fetch_fsmap finished waiting for FSMap version 55 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 10 client.17676 resolve_mds: validated gid 24743 aka daemon mds.b 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 20 client.17676 populate_metadata read hostname 'smithi138' 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5620374bac90 0x5620374bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 4 client.17676 mds_command: new command op to 24743 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 --> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5620373a7340 con 0x5620374bac90 2024-08-20T01:27:18.206 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.198+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5620374bac90 0x5620374bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc6fe00640 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5620374bac90 0x5620374bd0b0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3333b2864184cbf server_cookie=b6565e226725a5b2 in_seq=0 out_seq=0 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc58c00640 10 client.17676 ms_handle_connect on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc58c00640 1 -- 172.21.15.138:0/1746970961 <== mds.0 v2:172.21.15.142:6838/80285616 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x5620373a7340 con 0x5620374bac90 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc58c00640 10 client.17676 handle_command_reply: tid=0 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc78e74740 1 client.17676 shutdown 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 msgr2=0x562037447e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.199+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c02a910 tx=0x7fdc5c005fd0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.207 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 shutdown_connections 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:6838/80285616,v1:172.21.15.142:6839/80285616] conn(0x5620374bac90 0x5620374bd0b0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56203743f8d0 0x5620374428a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x562037433940 0x562037447e30 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 --2- 172.21.15.138:0/1746970961 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x562037433020 0x5620374478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 >> 172.21.15.138:0/1746970961 conn(0x562037430e50 msgr2=0x56203744af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc58c00640 0 client.17676 ms_handle_reset on v2:172.21.15.142:6838/80285616 2024-08-20T01:27:18.208 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 shutdown_connections 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 1 -- 172.21.15.138:0/1746970961 wait complete. 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc78e74740 20 client.17676 trim_cache size 0 max 16384 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 55, 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stdout: "leader": 24743, 2024-08-20T01:27:18.209 INFO:teuthology.orchestra.run.smithi138.stdout: "set_version": 68, 2024-08-20T01:27:18.210 INFO:teuthology.orchestra.run.smithi138.stdout: "sets": {} 2024-08-20T01:27:18.210 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:18.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 msgr2=0x7fdc540b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 0x7fdc540b6280 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x562037431d30 tx=0x7fdc6000cf90 comp rx=0 tx=0).stop 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 msgr2=0x7fdc7019f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.200+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fdc6000eb90 tx=0x7fdc6000c6a0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 shutdown_connections 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdc540b3dc0 0x7fdc540b6280 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdc70113830 0x7fdc701a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc70075b90 0x7fdc7019fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.211 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 --2- 172.21.15.138:0/1254993142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc70075270 0x7fdc7019f5c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 >> 172.21.15.138:0/1254993142 conn(0x7fdc700fe520 msgr2=0x7fdc700ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 shutdown_connections 2024-08-20T01:27:18.212 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:18.201+0000 7fdc76200640 1 -- 172.21.15.138:0/1254993142 wait complete. 2024-08-20T01:27:19.005 INFO:tasks.quiescer:done joining Quiescers 2024-08-20T01:27:19.005 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-08-20T01:27:19.015 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:27:19.057 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-08-20T01:27:19.068 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-08-20T01:27:19.069 INFO:teuthology.task.exec:Executing custom commands... 2024-08-20T01:27:19.069 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:27:19.069 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-08-20T01:27:19.147 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:18 smithi138 ceph-mon[18076]: pgmap v964: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-08-20T01:27:19.158 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-08-20T01:27:19.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:18 smithi167 ceph-mon[21965]: pgmap v964: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-08-20T01:27:19.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-08-20T01:27:19.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:18 smithi142 ceph-mon[21799]: pgmap v964: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-08-20T01:27:19.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.390+0000 7f856fe00640 1 -- 172.21.15.138:0/143766877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 msgr2=0x7f857006ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.390+0000 7f856fe00640 1 --2- 172.21.15.138:0/143766877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f857006ab80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8560009900 tx=0x7f856002a410 comp rx=0 tx=0).stop 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 -- 172.21.15.138:0/143766877 shutdown_connections 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 --2- 172.21.15.138:0/143766877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570105970 0x7f857010f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 --2- 172.21.15.138:0/143766877 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570104fd0 0x7f8570105430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 --2- 172.21.15.138:0/143766877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f857006ab80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 -- 172.21.15.138:0/143766877 >> 172.21.15.138:0/143766877 conn(0x7f8570077270 msgr2=0x7f8570077680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:19.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 -- 172.21.15.138:0/143766877 shutdown_connections 2024-08-20T01:27:19.394 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.391+0000 7f856fe00640 1 -- 172.21.15.138:0/143766877 wait complete. 2024-08-20T01:27:19.395 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 Processor -- start 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 -- start start 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 0x7f8570101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570105970 0x7f8570102460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f8570112850 con 0x7f857006a7a0 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f85701126d0 con 0x7f8570104fd0 2024-08-20T01:27:19.396 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856fe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f85701129d0 con 0x7f8570105970 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856ea00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34228/0 (socket says 172.21.15.138:34228) 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.392+0000 7f856ea00640 1 -- 172.21.15.138:0/205643026 learned_addr learned my addr 172.21.15.138:0/205643026 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856e000640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 0x7f8570101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856f400640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570105970 0x7f8570102460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 -- 172.21.15.138:0/205643026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570105970 msgr2=0x7f8570102460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570105970 0x7f8570102460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 -- 172.21.15.138:0/205643026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 msgr2=0x7f8570101f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.397 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 0x7f8570101f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 -- 172.21.15.138:0/205643026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701a7180 con 0x7f857006a7a0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856e000640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 0x7f8570101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856ea00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8560005ab0 tx=0x7f8560003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856003f070 con 0x7f857006a7a0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8560031b60 con 0x7f857006a7a0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701a7410 con 0x7f857006a7a0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560035030 con 0x7f857006a7a0 2024-08-20T01:27:19.398 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a7950 con 0x7f857006a7a0 2024-08-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.393+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857006ab80 con 0x7f857006a7a0 2024-08-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.397+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8560042070 con 0x7f857006a7a0 2024-08-20T01:27:19.399 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.398+0000 7f8567400640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 0x7f85540b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.398+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f8560104a10 con 0x7f857006a7a0 2024-08-20T01:27:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.398+0000 7f856e000640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 0x7f85540b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.400 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.398+0000 7f856e000640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 0x7f85540b6280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85701034d0 tx=0x7f855c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:19.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.400+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f85600cb2d0 con 0x7f857006a7a0 2024-08-20T01:27:19.612 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.611+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f85700632c0 con 0x7f85540b3dc0 2024-08-20T01:27:19.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.623+0000 7f8567400640 1 -- 172.21.15.138:0/205643026 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f85700632c0 con 0x7f85540b3dc0 2024-08-20T01:27:19.625 INFO:teuthology.orchestra.run.smithi138.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-08-20T00:00:00", "created": "2024-08-20T01:08:33", "first": "2024-08-20T01:09:00", "last": "2024-08-20T01:27:00", "last_pruned": "2024-08-20T01:27:03", "created_count": 19, "pruned_count": 13, "active": true} 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 msgr2=0x7f85540b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 0x7f85540b6280 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85701034d0 tx=0x7f855c005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 msgr2=0x7f85701019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8560005ab0 tx=0x7f8560003c10 comp rx=0 tx=0).stop 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 shutdown_connections 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f85540b3dc0 0x7f85540b6280 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.628 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8570105970 0x7f8570102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8570104fd0 0x7f8570101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 --2- 172.21.15.138:0/205643026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f857006a7a0 0x7f85701019e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 >> 172.21.15.138:0/205643026 conn(0x7f8570077270 msgr2=0x7f8570108380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 shutdown_connections 2024-08-20T01:27:19.629 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.626+0000 7f856fe00640 1 -- 172.21.15.138:0/205643026 wait complete. 2024-08-20T01:27:19.644 DEBUG:teuthology.orchestra.run.smithi138:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-08-20T01:27:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 -- 172.21.15.138:0/2333699550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6941083f0 msgr2=0x7fb694108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.828 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 --2- 172.21.15.138:0/2333699550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6941083f0 0x7fb694108830 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb67c009900 tx=0x7fb67c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 -- 172.21.15.138:0/2333699550 shutdown_connections 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 --2- 172.21.15.138:0/2333699550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb694108d70 0x7fb69410f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 --2- 172.21.15.138:0/2333699550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6941083f0 0x7fb694108830 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 --2- 172.21.15.138:0/2333699550 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941029f0 0x7fb694102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 -- 172.21.15.138:0/2333699550 >> 172.21.15.138:0/2333699550 conn(0x7fb6940fe430 msgr2=0x7fb694100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 -- 172.21.15.138:0/2333699550 shutdown_connections 2024-08-20T01:27:19.829 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.827+0000 7fb69b000640 1 -- 172.21.15.138:0/2333699550 wait complete. 2024-08-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 Processor -- start 2024-08-20T01:27:19.831 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 -- start start 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 0x7fb694075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb694112a80 con 0x7fb694108d70 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb694112900 con 0x7fb6941029f0 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.828+0000 7fb69b000640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb694112c00 con 0x7fb6941083f0 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb68b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb68b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46422/0 (socket says 172.21.15.138:46422) 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:36368/0 (socket says 172.21.15.138:36368) 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb68b400640 1 -- 172.21.15.138:0/2351315928 learned_addr learned my addr 172.21.15.138:0/2351315928 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb699000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 0x7fb694075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 -- 172.21.15.138:0/2351315928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 msgr2=0x7fb694075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 -- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 msgr2=0x7fb694075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 0x7fb694075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:19.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 -- 172.21.15.138:0/2351315928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6941a7310 con 0x7fb6941029f0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb68b400640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb699000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 0x7fb694075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.829+0000 7fb693e00640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9f0 tx=0x7fb68400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.830+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb68400ccc0 con 0x7fb6941029f0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.830+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6840108e0 con 0x7fb6941029f0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.830+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb684018840 con 0x7fb6941029f0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.830+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6941a7600 con 0x7fb6941029f0 2024-08-20T01:27:19.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.830+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6941a7ba0 con 0x7fb6941029f0 2024-08-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.831+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb694102dd0 con 0x7fb6941029f0 2024-08-20T01:27:19.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.833+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb684010430 con 0x7fb6941029f0 2024-08-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.834+0000 7fb691600640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 0x7fb6780b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.835+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fb6840e4600 con 0x7fb6941029f0 2024-08-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.835+0000 7fb68b400640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 0x7fb6780b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:19.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.835+0000 7fb68b400640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 0x7fb6780b6280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb694076cc0 tx=0x7fb67c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:19.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:19.838+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb6840ab510 con 0x7fb6941029f0 2024-08-20T01:27:20.050 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.049+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 --> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fb6940632c0 con 0x7fb6780b3dc0 2024-08-20T01:27:20.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.059+0000 7fb691600640 1 -- 172.21.15.138:0/2351315928 <== mgr.14220 v2:172.21.15.138:6800/2106854212 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fb6940632c0 con 0x7fb6780b3dc0 2024-08-20T01:27:20.061 INFO:teuthology.orchestra.run.smithi138.stdout:/ 1m 6m3h 2024-08-20T01:27:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.061+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 msgr2=0x7fb6780b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.061+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 0x7fb6780b6280 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb694076cc0 tx=0x7fb67c005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 msgr2=0x7fb6940751d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fb68400e9f0 tx=0x7fb68400edf0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 shutdown_connections 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb6780b3dc0 0x7fb6780b6280 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb694108d70 0x7fb694075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb6941083f0 0x7fb694075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 --2- 172.21.15.138:0/2351315928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb6941029f0 0x7fb6940751d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 >> 172.21.15.138:0/2351315928 conn(0x7fb6940fe430 msgr2=0x7fb694100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 shutdown_connections 2024-08-20T01:27:20.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.062+0000 7fb69b000640 1 -- 172.21.15.138:0/2351315928 wait complete. 2024-08-20T01:27:20.078 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-08-20T01:27:20.089 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-08-20T01:27:20.243 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 -- 172.21.15.138:0/754413947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18102800 msgr2=0x7f4a18102be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 --2- 172.21.15.138:0/754413947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18102800 0x7f4a18102be0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c009ca0 tx=0x7f4a0c02a8c0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 -- 172.21.15.138:0/754413947 shutdown_connections 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 --2- 172.21.15.138:0/754413947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18108b80 0x7f4a1810f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 --2- 172.21.15.138:0/754413947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a18108640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 --2- 172.21.15.138:0/754413947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18102800 0x7f4a18102be0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 -- 172.21.15.138:0/754413947 >> 172.21.15.138:0/754413947 conn(0x7f4a180fe220 msgr2=0x7f4a18100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 -- 172.21.15.138:0/754413947 shutdown_connections 2024-08-20T01:27:20.245 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.242+0000 7f4a1e800640 1 -- 172.21.15.138:0/754413947 wait complete. 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 Processor -- start 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 -- start start 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 0x7f4a1819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a1819b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18112890 con 0x7f4a18108b80 2024-08-20T01:27:20.247 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18112710 con 0x7f4a18102800 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.243+0000 7f4a1e800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f4a18112a10 con 0x7f4a18108200 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a1819b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 0x7f4a1819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1de00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34276/0 (socket says 172.21.15.138:34276) 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1de00640 1 -- 172.21.15.138:0/1960410365 learned_addr learned my addr 172.21.15.138:0/1960410365 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 -- 172.21.15.138:0/1960410365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 msgr2=0x7f4a1819af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 0x7f4a1819af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.248 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 -- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 msgr2=0x7f4a181a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 -- 172.21.15.138:0/1960410365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a181a1030 con 0x7f4a18108200 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1d400640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 0x7f4a1819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1de00640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.244+0000 7f4a1ca00640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a1819b4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400e9f0 tx=0x7f4a0400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.245+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400ccc0 con 0x7f4a18108200 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.245+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a04005020 con 0x7f4a18108200 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.245+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a181a1320 con 0x7f4a18108200 2024-08-20T01:27:20.249 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.245+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a04004a00 con 0x7f4a18108200 2024-08-20T01:27:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.245+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a181a79e0 con 0x7f4a18108200 2024-08-20T01:27:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.246+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a18102be0 con 0x7f4a18108200 2024-08-20T01:27:20.250 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.248+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f4a04004ba0 con 0x7f4a18108200 2024-08-20T01:27:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.250+0000 7f4a16000640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 0x7f4a000b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.250+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f4a040e4330 con 0x7f4a18108200 2024-08-20T01:27:20.251 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.250+0000 7f4a1d400640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 0x7f4a000b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.252 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.251+0000 7f4a1d400640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 0x7f4a000b6280 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006010 tx=0x7f4a0c009800 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:20.253 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.252+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f4a040ab330 con 0x7f4a18108200 2024-08-20T01:27:20.438 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.437+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a1802e340 con 0x7f4a18108200 2024-08-20T01:27:20.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.438+0000 7f4a16000640 1 -- 172.21.15.138:0/1960410365 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25421 (secure 0 0 0) 0x7f4a040aac30 con 0x7f4a18108200 2024-08-20T01:27:20.440 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:20.440 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":115,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:27:15.820826+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]},{"pool":4,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 msgr2=0x7f4a000b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 0x7f4a000b6280 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c006010 tx=0x7f4a0c009800 comp rx=0 tx=0).stop 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 msgr2=0x7f4a1819b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a1819b4d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400e9f0 tx=0x7f4a0400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 shutdown_connections 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f4a000b3dc0 0x7f4a000b6280 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4a18108b80 0x7f4a181a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.443 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4a18108200 0x7f4a1819b4d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 --2- 172.21.15.138:0/1960410365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a18102800 0x7f4a1819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 >> 172.21.15.138:0/1960410365 conn(0x7f4a180fe220 msgr2=0x7f4a181047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 shutdown_connections 2024-08-20T01:27:20.444 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.441+0000 7f4a1e800640 1 -- 172.21.15.138:0/1960410365 wait complete. 2024-08-20T01:27:20.458 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-08-20T01:00:30.697585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-08-20T01:05:22.428168+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-08-20T01:05:24.585931+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-08-20T01:05:33.948877+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-08-20T01:27:20.459 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-08-20T01:27:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 -- 172.21.15.138:0/2227689845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc06a2f0 msgr2=0x7f92dc06a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 --2- 172.21.15.138:0/2227689845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc06a2f0 0x7f92dc06a6d0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f92cc009900 tx=0x7f92cc02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 -- 172.21.15.138:0/2227689845 shutdown_connections 2024-08-20T01:27:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 --2- 172.21.15.138:0/2227689845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc1059b0 0x7f92dc10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 --2- 172.21.15.138:0/2227689845 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc105010 0x7f92dc105470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 --2- 172.21.15.138:0/2227689845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc06a2f0 0x7f92dc06a6d0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 -- 172.21.15.138:0/2227689845 >> 172.21.15.138:0/2227689845 conn(0x7f92dc077100 msgr2=0x7f92dc0798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 -- 172.21.15.138:0/2227689845 shutdown_connections 2024-08-20T01:27:20.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.616+0000 7f92e0e00640 1 -- 172.21.15.138:0/2227689845 wait complete. 2024-08-20T01:27:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.617+0000 7f92e0e00640 1 Processor -- start 2024-08-20T01:27:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.617+0000 7f92e0e00640 1 -- start start 2024-08-20T01:27:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.617+0000 7f92e0e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92e0e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 0x7f92dc19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.621 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92e0e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 0x7f92dc1a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92e0e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc112830 con 0x7f92dc1059b0 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92e0e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1126b0 con 0x7f92dc105010 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92e0e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f92dc1129b0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92dbe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 0x7f92dc1a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92d3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 0x7f92dc19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46466/0 (socket says 172.21.15.138:46466) 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 -- 172.21.15.138:0/3849560670 learned_addr learned my addr 172.21.15.138:0/3849560670 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 -- 172.21.15.138:0/3849560670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 msgr2=0x7f92dc19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 0x7f92dc19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 -- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 msgr2=0x7f92dc1a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 0x7f92dc1a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.623 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92db400640 1 -- 172.21.15.138:0/3849560670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92dc1a0fa0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92dbe00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 0x7f92dc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.618+0000 7f92d3e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 0x7f92dc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92db400640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005ab0 tx=0x7f92cc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc03f070 con 0x7f92dc06a2f0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92dc1a1230 con 0x7f92dc06a2f0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92dc1a7950 con 0x7f92dc06a2f0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92cc031b60 con 0x7f92dc06a2f0 2024-08-20T01:27:20.624 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.619+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92cc049060 con 0x7f92dc06a2f0 2024-08-20T01:27:20.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.620+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b80052c0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.625 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.623+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f92cc042070 con 0x7f92dc06a2f0 2024-08-20T01:27:20.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.624+0000 7f92d8c00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 0x7f92c00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.624+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f92cc1048c0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.625+0000 7f92d3e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 0x7f92c00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.626 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.625+0000 7f92d3e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 0x7f92c00b6200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f92dc106f80 tx=0x7f92c4005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:20.627 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.626+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f92cc0cb7d0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.811+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f92b8005880 con 0x7f92dc06a2f0 2024-08-20T01:27:20.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.812+0000 7f92d8c00640 1 -- 172.21.15.138:0/3849560670 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) ==== 93+0+10 (secure 0 0 0) 0x7f92cc0cb0d0 con 0x7f92dc06a2f0 2024-08-20T01:27:20.813 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 1 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 msgr2=0x7f92c00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 0x7f92c00b6200 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f92dc106f80 tx=0x7f92c4005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 msgr2=0x7f92dc19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f92cc005ab0 tx=0x7f92cc003c10 comp rx=0 tx=0).stop 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 shutdown_connections 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f92c00b3d40 0x7f92c00b6200 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f92dc1059b0 0x7f92dc1a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92dc105010 0x7f92dc19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 --2- 172.21.15.138:0/3849560670 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f92dc06a2f0 0x7f92dc19af00 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.814+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 >> 172.21.15.138:0/3849560670 conn(0x7f92dc077100 msgr2=0x7f92dc1081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.815+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 shutdown_connections 2024-08-20T01:27:20.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.815+0000 7f92e0e00640 1 -- 172.21.15.138:0/3849560670 wait complete. 2024-08-20T01:27:20.833 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-08-20T01:27:20.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 -- 172.21.15.138:0/2529684037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a41083c0 msgr2=0x7fc8a4108800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 --2- 172.21.15.138:0/2529684037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a41083c0 0x7fc8a4108800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc890009900 tx=0x7fc89002a410 comp rx=0 tx=0).stop 2024-08-20T01:27:20.993 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 -- 172.21.15.138:0/2529684037 shutdown_connections 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 --2- 172.21.15.138:0/2529684037 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a4108d40 0x7fc8a410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 --2- 172.21.15.138:0/2529684037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a41083c0 0x7fc8a4108800 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 --2- 172.21.15.138:0/2529684037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 -- 172.21.15.138:0/2529684037 >> 172.21.15.138:0/2529684037 conn(0x7fc8a40fe420 msgr2=0x7fc8a4100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 -- 172.21.15.138:0/2529684037 shutdown_connections 2024-08-20T01:27:20.994 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.991+0000 7fc8ac400640 1 -- 172.21.15.138:0/2529684037 wait complete. 2024-08-20T01:27:20.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.992+0000 7fc8ac400640 1 Processor -- start 2024-08-20T01:27:20.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.992+0000 7fc8ac400640 1 -- start start 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 0x7fc8a4199430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4112a10 con 0x7fc8a4108d40 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4112890 con 0x7fc8a41029c0 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8ac400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc8a4112b90 con 0x7fc8a41083c0 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.997 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34324/0 (socket says 172.21.15.138:34324) 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 -- 172.21.15.138:0/3354801943 learned_addr learned my addr 172.21.15.138:0/3354801943 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8a9a00640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8a9000640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 0x7fc8a4199430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 -- 172.21.15.138:0/3354801943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 msgr2=0x7fc8a4199430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 0x7fc8a4199430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 -- 172.21.15.138:0/3354801943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 msgr2=0x7fc8a4198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8aa400640 1 -- 172.21.15.138:0/3354801943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8a419ef90 con 0x7fc8a4108d40 2024-08-20T01:27:20.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8a9000640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 0x7fc8a4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.993+0000 7fc8a9a00640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8aa400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eb10 tx=0x7fc89c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c004b20 con 0x7fc8a4108d40 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc89c004cc0 con 0x7fc8a4108d40 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8a419f280 con 0x7fc8a4108d40 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc89c010670 con 0x7fc8a4108d40 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.994+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8a41a5940 con 0x7fc8a4108d40 2024-08-20T01:27:20.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.995+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8a4102da0 con 0x7fc8a4108d40 2024-08-20T01:27:21.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.998+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc89c005000 con 0x7fc8a4108d40 2024-08-20T01:27:21.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.999+0000 7fc8a2000640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 0x7fc88c0b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.000 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.999+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fc89c0e4c10 con 0x7fc8a4108d40 2024-08-20T01:27:21.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:20.999+0000 7fc8a9a00640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 0x7fc88c0b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.001 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.000+0000 7fc8a9a00640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 0x7fc88c0b6200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc898009fc0 tx=0x7fc898005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:21.002 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.001+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc89c055e20 con 0x7fc8a4108d40 2024-08-20T01:27:21.185 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.184+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc8a402e340 con 0x7fc8a4108d40 2024-08-20T01:27:21.186 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.185+0000 7fc8a2000640 1 -- 172.21.15.138:0/3354801943 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) ==== 104+0+11 (secure 0 0 0) 0x7fc89c0abb20 con 0x7fc8a4108d40 2024-08-20T01:27:21.186 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-08-20T01:27:21.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 msgr2=0x7fc88c0b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.188 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 0x7fc88c0b6200 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc898009fc0 tx=0x7fc898005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 msgr2=0x7fc8a419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eb10 tx=0x7fc89c00c2d0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 shutdown_connections 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc88c0b3d40 0x7fc88c0b6200 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc8a4108d40 0x7fc8a419e7c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc8a41083c0 0x7fc8a4199430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 --2- 172.21.15.138:0/3354801943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8a41029c0 0x7fc8a4198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.189 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 >> 172.21.15.138:0/3354801943 conn(0x7fc8a40fe420 msgr2=0x7fc8a4100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:21.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 shutdown_connections 2024-08-20T01:27:21.190 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.187+0000 7fc8ac400640 1 -- 172.21.15.138:0/3354801943 wait complete. 2024-08-20T01:27:21.212 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:20 smithi138 ceph-mon[18076]: pgmap v965: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-08-20T01:27:21.212 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:20 smithi138 ceph-mon[18076]: from='client.17682 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:21.212 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:20 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1960410365' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:21.212 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:20 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3849560670' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:27:21.215 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-08-20T01:27:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:20 smithi167 ceph-mon[21965]: pgmap v965: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-08-20T01:27:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:20 smithi167 ceph-mon[21965]: from='client.17682 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:20 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1960410365' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:21.216 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:20 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3849560670' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:27:21.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:20 smithi142 ceph-mon[21799]: pgmap v965: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-08-20T01:27:21.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:20 smithi142 ceph-mon[21799]: from='client.17682 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:21.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:20 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1960410365' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:21.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:20 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3849560670' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-08-20T01:27:21.374 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 -- 172.21.15.138:0/1003235469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108410 msgr2=0x7fc470108850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 --2- 172.21.15.138:0/1003235469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108410 0x7fc470108850 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 -- 172.21.15.138:0/1003235469 shutdown_connections 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 --2- 172.21.15.138:0/1003235469 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108d90 0x7fc47010f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 --2- 172.21.15.138:0/1003235469 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108410 0x7fc470108850 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 --2- 172.21.15.138:0/1003235469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 0x7fc470102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 -- 172.21.15.138:0/1003235469 >> 172.21.15.138:0/1003235469 conn(0x7fc4700fe430 msgr2=0x7fc470100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:21.375 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 -- 172.21.15.138:0/1003235469 shutdown_connections 2024-08-20T01:27:21.376 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.373+0000 7fc477e00640 1 -- 172.21.15.138:0/1003235469 wait complete. 2024-08-20T01:27:21.377 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 Processor -- start 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 -- start start 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 0x7fc4700751d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108d90 0x7fc470075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fc470112a30 con 0x7fc470108d90 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fc4701128b0 con 0x7fc470102a10 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.374+0000 7fc477e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fc470112bb0 con 0x7fc470108410 2024-08-20T01:27:21.378 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:46498/0 (socket says 172.21.15.138:46498) 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 -- 172.21.15.138:0/3939109102 learned_addr learned my addr 172.21.15.138:0/3939109102 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc475e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108d90 0x7fc470075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc475400640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 0x7fc4700751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 -- 172.21.15.138:0/3939109102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 msgr2=0x7fc4700751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 0x7fc4700751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 -- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108d90 msgr2=0x7fc470075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.379 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108d90 0x7fc470075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 -- 172.21.15.138:0/3939109102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4701a7350 con 0x7fc470108410 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.375+0000 7fc474a00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.376+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c03f070 con 0x7fc470108410 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.376+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4701a75e0 con 0x7fc470108410 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.376+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45c031b60 con 0x7fc470108410 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.376+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c035030 con 0x7fc470108410 2024-08-20T01:27:21.380 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.376+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4701a7ad0 con 0x7fc470108410 2024-08-20T01:27:21.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.377+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc470102df0 con 0x7fc470108410 2024-08-20T01:27:21.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.379+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fc45c042070 con 0x7fc470108410 2024-08-20T01:27:21.381 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.380+0000 7fc46e000640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 0x7fc4580b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.381+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fc45c105290 con 0x7fc470108410 2024-08-20T01:27:21.382 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.381+0000 7fc475400640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 0x7fc4580b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.381+0000 7fc475400640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 0x7fc4580b6200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc464009bd0 tx=0x7fc464005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:21.383 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.382+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fc45c0cc200 con 0x7fc470108410 2024-08-20T01:27:21.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.567+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc47002e340 con 0x7fc470108410 2024-08-20T01:27:21.568 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.567+0000 7fc46e000640 1 -- 172.21.15.138:0/3939109102 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) ==== 100+0+11 (secure 0 0 0) 0x7fc45c0cbb00 con 0x7fc470108410 2024-08-20T01:27:21.569 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 64 2024-08-20T01:27:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.569+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 msgr2=0x7fc4580b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 0x7fc4580b6200 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc464009bd0 tx=0x7fc464005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 msgr2=0x7fc470075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c003c10 comp rx=0 tx=0).stop 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 shutdown_connections 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fc4580b3d40 0x7fc4580b6200 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470108d90 0x7fc470075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fc470108410 0x7fc470075710 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 --2- 172.21.15.138:0/3939109102 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc470102a10 0x7fc4700751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 >> 172.21.15.138:0/3939109102 conn(0x7fc4700fe430 msgr2=0x7fc470100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:21.572 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 shutdown_connections 2024-08-20T01:27:21.573 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.570+0000 7fc477e00640 1 -- 172.21.15.138:0/3939109102 wait complete. 2024-08-20T01:27:21.588 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-08-20T01:27:21.747 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2465773948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28108210 msgr2=0x7f7f28108650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2465773948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28108210 0x7f7f28108650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009900 tx=0x7f7f1402a400 comp rx=0 tx=0).stop 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2465773948 shutdown_connections 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2465773948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 0x7f7f2810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2465773948 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28108210 0x7f7f28108650 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2465773948 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28102810 0x7f7f28102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2465773948 >> 172.21.15.138:0/2465773948 conn(0x7f7f280fe230 msgr2=0x7f7f28100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2465773948 shutdown_connections 2024-08-20T01:27:21.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.746+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2465773948 wait complete. 2024-08-20T01:27:21.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 Processor -- start 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 -- start start 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 0x7f7f2819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 0x7f7f281a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f7f281128a0 con 0x7f7f28102810 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112720 con 0x7f7f28108b90 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2ec00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f7f28112a20 con 0x7f7f28108210 2024-08-20T01:27:21.751 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2d800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34364/0 (socket says 172.21.15.138:34364) 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.747+0000 7f7f2d800640 1 -- 172.21.15.138:0/2420215292 learned_addr learned my addr 172.21.15.138:0/2420215292 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2ce00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 0x7f7f2819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2e200640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 0x7f7f281a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 -- 172.21.15.138:0/2420215292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 msgr2=0x7f7f2819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 0x7f7f2819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 -- 172.21.15.138:0/2420215292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 msgr2=0x7f7f281a07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 0x7f7f281a07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 -- 172.21.15.138:0/2420215292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f281a0fa0 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2ce00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 0x7f7f2819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2d800640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009d50 tx=0x7f7f1c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c003b70 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f1c003d10 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0108a0 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f281a1290 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.748+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f281a7950 con 0x7f7f28102810 2024-08-20T01:27:21.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.749+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f28102bf0 con 0x7f7f28102810 2024-08-20T01:27:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.751+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7f1c004000 con 0x7f7f28102810 2024-08-20T01:27:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.753+0000 7f7f26000640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 0x7f7f100b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:21.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.753+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f7f1c0e4b80 con 0x7f7f28102810 2024-08-20T01:27:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.753+0000 7f7f2ce00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 0x7f7f100b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:21.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.754+0000 7f7f2ce00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 0x7f7f100b6280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000c00 tx=0x7f7f14005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:21.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.756+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f7f1c0aba90 con 0x7f7f28102810 2024-08-20T01:27:21.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.941+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7f2802e340 con 0x7f7f28102810 2024-08-20T01:27:21.942 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.941+0000 7f7f26000640 1 -- 172.21.15.138:0/2420215292 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) ==== 103+0+11 (secure 0 0 0) 0x7f7f1c0ab390 con 0x7f7f28102810 2024-08-20T01:27:21.943 INFO:teuthology.orchestra.run.smithi138.stdout:pg_num: 32 2024-08-20T01:27:21.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.943+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 msgr2=0x7f7f100b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.943+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 0x7f7f100b6280 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000c00 tx=0x7f7f14005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:21.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.943+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 msgr2=0x7f7f2819af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:21.945 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.943+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009d50 tx=0x7f7f1c00c6a0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 shutdown_connections 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f7f100b3dc0 0x7f7f100b6280 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f28108b90 0x7f7f281a07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f7f28108210 0x7f7f2819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 --2- 172.21.15.138:0/2420215292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7f28102810 0x7f7f2819af00 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 >> 172.21.15.138:0/2420215292 conn(0x7f7f280fe230 msgr2=0x7f7f281047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 shutdown_connections 2024-08-20T01:27:21.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:21.944+0000 7f7f2ec00640 1 -- 172.21.15.138:0/2420215292 wait complete. 2024-08-20T01:27:21.959 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:21 smithi138 ceph-mon[18076]: from='client.27734 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:21.959 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:21 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3354801943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T01:27:21.959 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:21 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3939109102' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T01:27:21.961 DEBUG:teuthology.orchestra.run.smithi138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-08-20T01:27:22.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 -- 172.21.15.138:0/3795937422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10ad90 msgr2=0x7fd70c111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:22.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 --2- 172.21.15.138:0/3795937422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10ad90 0x7fd70c111590 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fd704009ca0 tx=0x7fd70402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.121 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 -- 172.21.15.138:0/3795937422 shutdown_connections 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 --2- 172.21.15.138:0/3795937422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c10ad90 0x7fd70c111590 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 --2- 172.21.15.138:0/3795937422 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 0x7fd70c10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 --2- 172.21.15.138:0/3795937422 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c104a10 0x7fd70c104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 -- 172.21.15.138:0/3795937422 >> 172.21.15.138:0/3795937422 conn(0x7fd70c100430 msgr2=0x7fd70c102850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 -- 172.21.15.138:0/3795937422 shutdown_connections 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.119+0000 7fd711200640 1 -- 172.21.15.138:0/3795937422 wait complete. 2024-08-20T01:27:22.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 Processor -- start 2024-08-20T01:27:22.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 -- start start 2024-08-20T01:27:22.124 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 0x7fd70c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 0x7fd70c1a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c114a60 con 0x7fd70c104a10 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c1148e0 con 0x7fd70c10ad90 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.120+0000 7fd711200640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fd70c114be0 con 0x7fd70c10a410 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:22.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:34374/0 (socket says 172.21.15.138:34374) 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 -- 172.21.15.138:0/1011111035 learned_addr learned my addr 172.21.15.138:0/1011111035 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70aa00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 0x7fd70c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70be00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 0x7fd70c1a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 -- 172.21.15.138:0/1011111035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 msgr2=0x7fd70c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 0x7fd70c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 -- 172.21.15.138:0/1011111035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 msgr2=0x7fd70c1a07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 0x7fd70c1a07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:22.126 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 -- 172.21.15.138:0/1011111035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd70c1a0f80 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70be00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 0x7fd70c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.121+0000 7fd70b400640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e9f0 tx=0x7fd6fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.122+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00cd80 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.122+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6fc0108e0 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.122+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd70c1a1270 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.122+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc018980 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.122+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd70c1a7930 con 0x7fd70c104a10 2024-08-20T01:27:22.127 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.123+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd70c104df0 con 0x7fd70c104a10 2024-08-20T01:27:22.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.125+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fd6fc010430 con 0x7fd70c104a10 2024-08-20T01:27:22.128 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.126+0000 7fd703e00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 0x7fd6f00b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.127+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fd6fc0e4c90 con 0x7fd70c104a10 2024-08-20T01:27:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.127+0000 7fd70aa00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 0x7fd6f00b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.127+0000 7fd70aa00640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 0x7fd6f00b6200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd70c106ea0 tx=0x7fd6f8005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:22.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.128+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fd6fc0abba0 con 0x7fd70c104a10 2024-08-20T01:27:22.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:21 smithi167 ceph-mon[21965]: from='client.27734 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:22.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:21 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3354801943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T01:27:22.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:21 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3939109102' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T01:27:22.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:21 smithi142 ceph-mon[21799]: from='client.27734 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-08-20T01:27:22.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:21 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3354801943' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-08-20T01:27:22.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:21 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3939109102' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-08-20T01:27:22.364 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.362+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd70c02e340 con 0x7fd70c104a10 2024-08-20T01:27:22.373 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.372+0000 7fd703e00640 1 -- 172.21.15.138:0/1011111035 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+821918 (secure 0 0 0) 0x7fd6fc0b2e20 con 0x7fd70c104a10 2024-08-20T01:27:22.385 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:22.385 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:22.385 INFO:teuthology.orchestra.run.smithi138.stdout: "epoch": 25, 2024-08-20T01:27:22.385 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.385 INFO:teuthology.orchestra.run.smithi138.stdout: "active_gid": 14220, 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "active_name": "x", 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "active_addrs": { 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:6800", 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2106854212 2024-08-20T01:27:22.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:6801", 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2106854212 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.387 INFO:teuthology.orchestra.run.smithi138.stdout: "active_addr": "172.21.15.138:6801/2106854212", 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "active_change": "2024-08-20T00:58:34.377927+0000", 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "active_mgr_features": 4540701547738038271, 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "available": true, 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "standbys": [ 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 24169, 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "z", 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_features": 4540701547738038271, 2024-08-20T01:27:22.388 INFO:teuthology.orchestra.run.smithi138.stdout: "available_modules": [ 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "alerts", 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:22.389 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.390 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.391 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.392 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.394 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.395 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.396 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.397 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.398 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_destination": { 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_destination", 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.399 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email address to send alerts to", 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.400 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_from_name": { 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_from_name", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "Ceph", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.401 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email From: name", 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_host": { 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_host", 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.402 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP server", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.403 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_password": { 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_password", 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.404 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Password to authenticate with", 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_port": { 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_port", 2024-08-20T01:27:22.405 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "465", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP port", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.406 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_sender": { 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_sender", 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.407 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP envelope sender", 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_ssl": { 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_ssl", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.409 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T01:27:22.410 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.410 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.410 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_user": { 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_user", 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.411 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "User to authenticate as", 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.413 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "balancer", 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "active": { 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "active", 2024-08-20T01:27:22.415 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T01:27:22.416 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_time": { 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_time", 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.417 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0000", 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.418 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_weekday": { 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_weekday", 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.419 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.420 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_max_iterations": { 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_max_iterations", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "25", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "250", 2024-08-20T01:27:22.421 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_metrics": { 2024-08-20T01:27:22.422 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_metrics", 2024-08-20T01:27:22.423 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.423 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.423 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T01:27:22.424 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.424 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.424 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.424 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T01:27:22.425 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T01:27:22.425 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.425 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.425 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_step": { 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_step", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.5", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0.001", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "0.999", 2024-08-20T01:27:22.426 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "aggressiveness of optimization", 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "end_time": { 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_time", 2024-08-20T01:27:22.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2359", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:22.428 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "end_weekday": { 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_weekday", 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.429 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:22.430 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.431 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.432 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.433 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.434 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.435 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.436 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.437 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.438 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.439 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "min_score": { 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_score", 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.440 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "upmap", 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.442 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-compat", 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "none", 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "read", 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap", 2024-08-20T01:27:22.443 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap-read" 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Balancer mode", 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_ids": { 2024-08-20T01:27:22.444 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_ids", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T01:27:22.445 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.446 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.447 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.448 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.448 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.448 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.449 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.449 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.449 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.450 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.450 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.450 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.451 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.451 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.451 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.451 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_deviation": { 2024-08-20T01:27:22.452 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_deviation", 2024-08-20T01:27:22.452 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.452 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.453 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.453 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:22.453 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:22.453 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.453 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.454 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T01:27:22.454 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T01:27:22.454 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.455 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.455 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_optimizations": { 2024-08-20T01:27:22.455 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_optimizations", 2024-08-20T01:27:22.456 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:22.456 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.456 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.456 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.457 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.457 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.457 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.457 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T01:27:22.458 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.458 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.458 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.458 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.459 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.459 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.459 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm", 2024-08-20T01:27:22.460 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.460 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.460 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.460 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_down_multiplier": { 2024-08-20T01:27:22.461 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_down_multiplier", 2024-08-20T01:27:22.461 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.461 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.462 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:22.462 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.462 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.462 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.463 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T01:27:22.463 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.463 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.464 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_refresh_rate": { 2024-08-20T01:27:22.464 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_refresh_rate", 2024-08-20T01:27:22.464 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.465 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.465 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "20", 2024-08-20T01:27:22.465 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.466 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.466 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.466 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T01:27:22.467 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.467 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.467 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.467 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_starting_port": { 2024-08-20T01:27:22.468 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_starting_port", 2024-08-20T01:27:22.468 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.468 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.468 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.469 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4721", 2024-08-20T01:27:22.469 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.469 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.469 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.469 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T01:27:22.470 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.470 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.470 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.470 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.470 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_ptrace": { 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_ptrace", 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.471 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_interval": { 2024-08-20T01:27:22.472 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_interval", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.473 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_memory_target_ratio": { 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.474 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.7", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.475 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm_log_destination": { 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm_log_destination", 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.476 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "file", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "file,syslog", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "syslog" 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "cgroups_split": { 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cgroups_split", 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.478 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "config_checks_enabled": { 2024-08-20T01:27:22.479 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_checks_enabled", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T01:27:22.480 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "config_dashboard": { 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_dashboard", 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.481 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_alertmanager": { 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_alertmanager", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.483 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_base": { 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_base", 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.484 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Container image name, without the tag", 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_elasticsearch": { 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_elasticsearch", 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "elasticsearch container image", 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_grafana": { 2024-08-20T01:27:22.487 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_grafana", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:22.488 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_haproxy": { 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_haproxy", 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.489 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "HAproxy container image", 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_agent": { 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_agent", 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.491 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger agent container image", 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.492 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_collector": { 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_collector", 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.493 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger collector container image", 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.494 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_query": { 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_query", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.495 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger query container image", 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.496 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_keepalived": { 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_keepalived", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.497 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Keepalived container image", 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_loki": { 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_loki", 2024-08-20T01:27:22.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Loki container image", 2024-08-20T01:27:22.499 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nginx": { 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nginx", 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.500 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nginx container image", 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.501 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_node_exporter": { 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_node_exporter", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.502 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nvmeof": { 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nvmeof", 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.503 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nvme-of container image", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.504 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_prometheus": { 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_prometheus", 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.505 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_promtail": { 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_promtail", 2024-08-20T01:27:22.506 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Promtail container image", 2024-08-20T01:27:22.507 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_samba": { 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_samba", 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.508 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Samba/SMB container image", 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.509 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_snmp_gateway": { 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_snmp_gateway", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.510 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SNMP Gateway container image", 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "container_init": { 2024-08-20T01:27:22.511 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_init", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.512 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "daemon_cache_timeout": { 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "daemon_cache_timeout", 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:22.513 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.514 INFO:teuthology.orchestra.run.smithi138.stdout: "default_cephadm_command_timeout": { 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "900", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.515 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "default_registry": { 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_registry", 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.516 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io", 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.517 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "device_cache_timeout": { 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_cache_timeout", 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.518 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1800", 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache device inventory", 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.519 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "device_enhanced_scan": { 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_enhanced_scan", 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.520 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "facts_cache_timeout": { 2024-08-20T01:27:22.521 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "facts_cache_timeout", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.522 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache host facts data", 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "grafana_dashboards_path": { 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "grafana_dashboards_path", 2024-08-20T01:27:22.523 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T01:27:22.524 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "host_check_interval": { 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "host_check_interval", 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.525 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to perform a host check", 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.526 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "hw_monitoring": { 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hw_monitoring", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.527 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "inventory_list_all": { 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "inventory_list_all", 2024-08-20T01:27:22.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.530 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.531 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "log_refresh_metadata": { 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_refresh_metadata", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.533 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.534 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.536 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.537 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.538 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.539 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.540 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.541 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "*", 2024-08-20T01:27:22.542 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "max_count_per_host": { 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_count_per_host", 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.544 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of daemons per service per host", 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd_draining_count": { 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_osd_draining_count", 2024-08-20T01:27:22.545 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T01:27:22.546 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "migration_current": { 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "migration_current", 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "internal - do not modify", 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.548 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.549 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "root", 2024-08-20T01:27:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm-package", 2024-08-20T01:27:22.550 INFO:teuthology.orchestra.run.smithi138.stdout: "root" 2024-08-20T01:27:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T01:27:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.551 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "oob_default_addr": { 2024-08-20T01:27:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "oob_default_addr", 2024-08-20T01:27:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.552 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "169.254.1.1", 2024-08-20T01:27:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.553 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T01:27:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.554 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "prometheus_alerts_path": { 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus_alerts_path", 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.555 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T01:27:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.556 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T01:27:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.557 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_insecure": { 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_insecure", 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.558 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T01:27:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.559 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_password": { 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_password", 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.560 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.561 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_url": { 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_url", 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.562 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_username": { 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_username", 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.564 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "secure_monitoring_stack": { 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secure_monitoring_stack", 2024-08-20T01:27:22.565 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T01:27:22.566 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "service_discovery_port": { 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "service_discovery_port", 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.567 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8765", 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "cephadm service discovery port", 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.569 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "ssh_config_file": { 2024-08-20T01:27:22.570 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssh_config_file", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T01:27:22.571 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "use_agent": { 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_agent", 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.572 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.573 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "use_repo_digest": { 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_repo_digest", 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.574 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.575 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_failed_host_check": { 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_failed_host_check", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.576 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_daemons": { 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_daemons", 2024-08-20T01:27:22.577 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T01:27:22.578 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_hosts": { 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_hosts", 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.579 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crash", 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.581 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.582 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.583 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.584 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.585 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.586 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.587 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.588 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "retain_interval": { 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retain_interval", 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "31536000", 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.590 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.591 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.592 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_recent_interval": { 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_recent_interval", 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1209600", 2024-08-20T01:27:22.593 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.594 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dashboard", 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T01:27:22.595 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.596 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.599 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_ENABLED": { 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T01:27:22.600 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.601 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.602 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.604 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T01:27:22.605 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.606 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.607 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.608 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.609 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.610 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.611 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T01:27:22.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.613 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.614 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.616 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.617 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.618 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.619 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.620 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.621 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_URL": { 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_URL", 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.622 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.623 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:22.624 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.625 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.626 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.627 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.629 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T01:27:22.630 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.631 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.632 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "[]", 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T01:27:22.633 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "{}", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.634 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.635 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.636 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.637 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T01:27:22.638 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.639 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T01:27:22.640 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.641 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T01:27:22.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.644 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.645 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.646 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T01:27:22.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.648 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.649 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.651 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.652 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.653 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.654 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T01:27:22.655 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.656 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T01:27:22.657 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T01:27:22.658 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.658 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.658 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.658 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "45", 2024-08-20T01:27:22.658 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T01:27:22.659 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:22.661 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T01:27:22.662 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.663 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.664 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.665 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.666 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T01:27:22.667 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.668 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.669 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.670 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:22.671 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "cross_origin_url": { 2024-08-20T01:27:22.672 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cross_origin_url", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.673 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "crt_file": { 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crt_file", 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.674 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "debug": { 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "debug", 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.676 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable/disable debug options", 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.677 INFO:teuthology.orchestra.run.smithi138.stdout: "jwt_token_ttl": { 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "jwt_token_ttl", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "28800", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.678 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.680 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.681 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.682 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.684 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.685 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.686 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.688 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "motd": { 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "motd", 2024-08-20T01:27:22.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.690 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "The message of the day", 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect_resolve_ip_addr": { 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T01:27:22.691 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.692 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:22.693 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.694 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.695 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8080", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.699 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl_server_port": { 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl_server_port", 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8443", 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.701 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:22.702 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "redirect", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.703 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect" 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:22.704 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.705 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "url_prefix": { 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url_prefix", 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.706 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.707 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "devicehealth", 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.708 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_monitoring": { 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_monitoring", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.709 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "monitor device health metrics", 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.711 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.712 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.714 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.715 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.715 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.715 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.716 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.716 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.716 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.716 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.716 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.717 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.719 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.719 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.719 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.720 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.720 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.720 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.720 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.721 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.721 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.721 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.721 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.721 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.722 INFO:teuthology.orchestra.run.smithi138.stdout: "mark_out_threshold": { 2024-08-20T01:27:22.722 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mark_out_threshold", 2024-08-20T01:27:22.722 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.722 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2419200", 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.723 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T01:27:22.724 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.724 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.724 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": { 2024-08-20T01:27:22.725 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_name", 2024-08-20T01:27:22.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.725 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.725 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "device_health_metrics", 2024-08-20T01:27:22.726 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.726 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.726 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.726 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T01:27:22.726 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "retention_period": { 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retention_period", 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15552000", 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain device health metrics", 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_frequency": { 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_frequency", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.729 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "self_heal": { 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "self_heal", 2024-08-20T01:27:22.730 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.732 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.734 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_threshold": { 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_threshold", 2024-08-20T01:27:22.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7257600", 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.736 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "diskprediction_local", 2024-08-20T01:27:22.737 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.738 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.739 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.740 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.741 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.742 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.743 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.744 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.745 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.746 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "predict_interval": { 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predict_interval", 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.747 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "predictor_model": { 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predictor_model", 2024-08-20T01:27:22.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "prophetstor", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.750 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.751 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.752 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "influx", 2024-08-20T01:27:22.753 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": false, 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "influxdb python module not found", 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "batch_size": { 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "batch_size", 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.754 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5000", 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "database": { 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "database", 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "ceph", 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.756 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": { 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hostname", 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server hostname", 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "30", 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "5", 2024-08-20T01:27:22.760 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.762 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.763 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.764 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.765 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.766 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.767 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.768 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "password": { 2024-08-20T01:27:22.769 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "password", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.770 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "password of InfluxDB server user", 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "port": { 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "port", 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.771 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8086", 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server port", 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.772 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.773 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:22.774 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "false", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.775 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "threads": { 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threads", 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:22.776 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "32", 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "username": { 2024-08-20T01:27:22.777 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "username", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.778 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "username of InfluxDB server user", 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "verify_ssl": { 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "verify_ssl", 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.779 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "true", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.780 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "insights", 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.781 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.782 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.783 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.784 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.785 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.786 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.787 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.788 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.789 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.790 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.791 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "iostat", 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.792 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.793 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.794 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.795 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.796 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.797 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.798 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.799 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.800 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.801 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.802 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "k8sevents", 2024-08-20T01:27:22.803 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_event_retention_days": { 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ceph_event_retention_days", 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7", 2024-08-20T01:27:22.804 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.805 INFO:teuthology.orchestra.run.smithi138.stdout: "config_check_secs": { 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_check_secs", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "10", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.806 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.807 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.808 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.809 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.810 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.811 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.812 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.813 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.814 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.815 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.816 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "localpool", 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "failure_domain": { 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "failure_domain", 2024-08-20T01:27:22.818 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "host", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "failure domain for any created local pool", 2024-08-20T01:27:22.819 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.820 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.821 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.822 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.823 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.824 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.825 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.826 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.827 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.828 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": { 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_size", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.829 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default min_size for any created local pool", 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "num_rep": { 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "num_rep", 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.830 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default replica count for any created local pool", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.831 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": { 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_num", 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.832 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "128", 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.833 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "prefix": { 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prefix", 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.834 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name prefix for any created local pool", 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.835 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.836 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "subtree": { 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subtree", 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.837 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "rack", 2024-08-20T01:27:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.838 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mds_autoscaler", 2024-08-20T01:27:22.839 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.840 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.841 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.842 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.843 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.844 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.845 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.846 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.847 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.849 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.850 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirroring", 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.851 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.852 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.854 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.855 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.856 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.857 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.858 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.859 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.860 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.861 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "nfs", 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.862 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.863 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.864 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.865 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.866 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.867 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.868 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.870 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.871 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.871 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.871 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.871 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.871 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.872 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.872 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.872 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.872 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.872 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.873 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.873 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.873 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.873 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.874 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.874 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.874 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.875 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.875 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.875 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.875 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.876 INFO:teuthology.orchestra.run.smithi138.stdout: "fail_fs": { 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "fail_fs", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.877 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.878 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.879 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.881 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.882 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.883 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.884 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.885 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator": { 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.887 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm", 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "rook", 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "test_orchestrator" 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Orchestrator backend", 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.888 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.889 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_perf_query", 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.891 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.892 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.893 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.894 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.895 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.896 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.897 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.899 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.900 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.901 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.902 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_support", 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.903 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.904 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.905 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.906 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.907 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.908 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.909 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.910 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.911 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.912 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.913 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.914 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.914 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.914 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_autoscaler", 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.915 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.916 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.917 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.918 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.919 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.920 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.921 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.922 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.923 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:22.924 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.925 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.926 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.927 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "threshold": { 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threshold", 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1.0", 2024-08-20T01:27:22.928 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "scaling threshold", 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.929 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "progress", 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_pg_recovery_event": { 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_pg_recovery_event", 2024-08-20T01:27:22.930 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T01:27:22.931 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.932 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.933 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.934 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.935 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.938 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.939 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.940 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.941 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "max_completed_events": { 2024-08-20T01:27:22.942 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_completed_events", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "50", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.943 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "number of past completed events to remember", 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:22.944 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.945 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long the module is going to sleep", 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.946 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.947 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus", 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.948 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "cache": { 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cache", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.949 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "exclude_perf_counters": { 2024-08-20T01:27:22.950 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "exclude_perf_counters", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T01:27:22.951 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T01:27:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.952 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.953 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.954 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.955 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.956 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.957 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.958 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.960 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.961 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools": { 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.962 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.963 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "300", 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_interval": { 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_interval", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15.0", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.965 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:22.966 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.967 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "9283", 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.968 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.969 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.970 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "stale_cache_strategy": { 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stale_cache_strategy", 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.971 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "log", 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:22.972 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "default", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "default", 2024-08-20T01:27:22.973 INFO:teuthology.orchestra.run.smithi138.stdout: "error" 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.974 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:22.975 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:22.977 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.978 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_support", 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.979 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.980 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.981 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.982 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:22.983 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.984 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.985 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.986 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.987 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_snap_create": { 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_snap_create", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.988 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "mirror_snapshot_schedule": { 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.989 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.990 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.991 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "trash_purge_schedule": { 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "trash_purge_schedule", 2024-08-20T01:27:22.992 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.993 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:22.994 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "restful", 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_auth": { 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_auth", 2024-08-20T01:27:22.995 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.996 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.997 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:22.998 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:22.999 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.000 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.001 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.002 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.003 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.004 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.005 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.007 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.008 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.010 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.011 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rgw", 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.012 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.013 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.015 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.016 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.017 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.018 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.019 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T01:27:23.020 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T01:27:23.021 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.022 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.023 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rook", 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.024 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.025 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.026 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.028 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.029 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.030 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.031 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.032 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.033 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_class": { 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "storage_class", 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.034 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "local", 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T01:27:23.035 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.036 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.036 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.036 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.037 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.037 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.037 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "selftest", 2024-08-20T01:27:23.037 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.038 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.041 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.042 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.043 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.043 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.043 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.043 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.043 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.044 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.045 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.046 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "roption1": { 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption1", 2024-08-20T01:27:23.047 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.048 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "roption2": { 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption2", 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "xyz", 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.049 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption1": { 2024-08-20T01:27:23.050 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption1", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.051 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption2": { 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption2", 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.052 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.053 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption3": { 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption3", 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.054 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption4": { 2024-08-20T01:27:23.055 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption4", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.056 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption5": { 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption5", 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.057 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.058 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption6": { 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption6", 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.059 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption7": { 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption7", 2024-08-20T01:27:23.060 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "42", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.061 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.062 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "testkey": { 2024-08-20T01:27:23.063 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testkey", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.064 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "testlkey": { 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testlkey", 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.065 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.066 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "testnewline": { 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testnewline", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.067 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.068 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smb", 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "internal_store_backend": { 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "internal_store_backend", 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.069 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.070 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.071 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.072 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.073 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.075 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.076 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.077 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.078 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.079 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "update_orchestration": { 2024-08-20T01:27:23.080 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "update_orchestration", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T01:27:23.081 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snap_schedule", 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.082 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_m_granularity": { 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_m_granularity", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.083 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "dump_on_update": { 2024-08-20T01:27:23.084 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dump_on_update", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "dump database to debug log on update", 2024-08-20T01:27:23.085 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.086 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.087 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.088 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.089 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.090 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.091 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.092 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.093 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.094 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.095 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stats", 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.097 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.098 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.099 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.100 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.101 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.102 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.103 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.104 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.105 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "status", 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.106 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.107 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.108 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.109 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.110 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.111 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.112 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.113 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.114 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.115 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telegraf", 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.116 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "address": { 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "address", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.117 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.118 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.119 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.120 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.121 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.122 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.123 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.124 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.125 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.126 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.127 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.128 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telemetry", 2024-08-20T01:27:23.129 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_basic": { 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_basic", 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.130 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_crash": { 2024-08-20T01:27:23.131 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_crash", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.132 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_device": { 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_device", 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.133 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.134 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_ident": { 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_ident", 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.135 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.136 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_perf": { 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_perf", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.137 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "contact": { 2024-08-20T01:27:23.138 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "contact", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.139 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "description": { 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "description", 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.140 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.141 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "device_url": { 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_url", 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T01:27:23.142 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.143 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.144 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.145 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "24", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "8", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.146 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "last_opt_revision": { 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "last_opt_revision", 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1", 2024-08-20T01:27:23.147 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard": { 2024-08-20T01:27:23.148 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.149 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard_description": { 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard_description", 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.150 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.151 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.152 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.153 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.154 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.155 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.156 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.157 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.158 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "organization": { 2024-08-20T01:27:23.159 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "organization", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.160 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "proxy": { 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "proxy", 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.161 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.162 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.163 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "url": { 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url", 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.164 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.165 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "test_orchestrator", 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.166 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.167 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.168 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.169 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.170 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.171 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.172 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.173 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.174 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.175 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "volumes", 2024-08-20T01:27:23.176 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.177 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.178 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.179 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.180 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.181 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.181 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.181 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.182 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.183 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.184 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.185 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.186 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_clones": { 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_clones", 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4", 2024-08-20T01:27:23.187 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "periodic_async_work": { 2024-08-20T01:27:23.188 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "periodic_async_work", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Periodically check for async work", 2024-08-20T01:27:23.189 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_delay": { 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_delay", 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.190 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_no_wait": { 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.192 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.193 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.194 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix", 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.195 INFO:teuthology.orchestra.run.smithi138.stdout: "discovery_interval": { 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "discovery_interval", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "100", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.196 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "identifier": { 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "identifier", 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.197 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.199 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.200 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.201 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.202 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.203 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.205 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.206 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.207 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.208 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_host": { 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_host", 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.210 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_port": { 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_port", 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.211 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10051", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.212 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_sender": { 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_sender", 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T01:27:23.213 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.214 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: "gid": 24182, 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "y", 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: "mgr_features": 4540701547738038271, 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: "available_modules": [ 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.215 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "alerts", 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.216 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.217 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.218 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.219 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.220 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.221 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.222 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.223 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.224 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_destination": { 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_destination", 2024-08-20T01:27:23.225 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email address to send alerts to", 2024-08-20T01:27:23.226 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_from_name": { 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_from_name", 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.227 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "Ceph", 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email From: name", 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_host": { 2024-08-20T01:27:23.228 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_host", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.229 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP server", 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_password": { 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_password", 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.230 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Password to authenticate with", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.231 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_port": { 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_port", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "465", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.232 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP port", 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_sender": { 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_sender", 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.233 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP envelope sender", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.234 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_ssl": { 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_ssl", 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.235 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_user": { 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_user", 2024-08-20T01:27:23.236 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "User to authenticate as", 2024-08-20T01:27:23.237 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.238 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.239 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "balancer", 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "active": { 2024-08-20T01:27:23.240 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "active", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.241 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_time": { 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_time", 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.242 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0000", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:23.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_weekday": { 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_weekday", 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.244 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_max_iterations": { 2024-08-20T01:27:23.245 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_max_iterations", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "25", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "250", 2024-08-20T01:27:23.246 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_metrics": { 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_metrics", 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.247 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T01:27:23.248 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_step": { 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_step", 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.5", 2024-08-20T01:27:23.249 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0.001", 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "0.999", 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "aggressiveness of optimization", 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.250 INFO:teuthology.orchestra.run.smithi138.stdout: "end_time": { 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_time", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2359", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.251 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "end_weekday": { 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_weekday", 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.252 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.253 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.254 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.256 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.257 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.258 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.259 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.260 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "min_score": { 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_score", 2024-08-20T01:27:23.261 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.262 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "upmap", 2024-08-20T01:27:23.263 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-compat", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "none", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "read", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap", 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap-read" 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.264 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Balancer mode", 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_ids": { 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_ids", 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.265 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.266 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.267 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.268 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.269 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_deviation": { 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_deviation", 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.270 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T01:27:23.271 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_optimizations": { 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_optimizations", 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.272 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.273 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm", 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.274 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_down_multiplier": { 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_down_multiplier", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.275 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_refresh_rate": { 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_refresh_rate", 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.276 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "20", 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.277 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_starting_port": { 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_starting_port", 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4721", 2024-08-20T01:27:23.278 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_ptrace": { 2024-08-20T01:27:23.279 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_ptrace", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T01:27:23.280 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_interval": { 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_interval", 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.281 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.282 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_memory_target_ratio": { 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.7", 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.283 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm_log_destination": { 2024-08-20T01:27:23.284 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm_log_destination", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "file", 2024-08-20T01:27:23.285 INFO:teuthology.orchestra.run.smithi138.stdout: "file,syslog", 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "syslog" 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.286 INFO:teuthology.orchestra.run.smithi138.stdout: "cgroups_split": { 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cgroups_split", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.287 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "config_checks_enabled": { 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_checks_enabled", 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.288 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.289 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "config_dashboard": { 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_dashboard", 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.290 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.291 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_alertmanager": { 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_alertmanager", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.292 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_base": { 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_base", 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.293 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Container image name, without the tag", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.294 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_elasticsearch": { 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_elasticsearch", 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T01:27:23.295 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "elasticsearch container image", 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.296 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_grafana": { 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_grafana", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.297 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_haproxy": { 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_haproxy", 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.298 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "HAproxy container image", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.299 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_agent": { 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_agent", 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.300 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger agent container image", 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.301 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_collector": { 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_collector", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.302 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger collector container image", 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_query": { 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_query", 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.303 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger query container image", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.304 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_keepalived": { 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_keepalived", 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.305 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Keepalived container image", 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.306 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_loki": { 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_loki", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.307 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Loki container image", 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nginx": { 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nginx", 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.308 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nginx container image", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.309 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_node_exporter": { 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_node_exporter", 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.310 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.311 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nvmeof": { 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nvmeof", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.312 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nvme-of container image", 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_prometheus": { 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_prometheus", 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.313 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.314 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_promtail": { 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_promtail", 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.315 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Promtail container image", 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.316 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_samba": { 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_samba", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.317 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Samba/SMB container image", 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_snmp_gateway": { 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_snmp_gateway", 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.318 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SNMP Gateway container image", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.319 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "container_init": { 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_init", 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.320 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.321 INFO:teuthology.orchestra.run.smithi138.stdout: "daemon_cache_timeout": { 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "daemon_cache_timeout", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.322 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "default_cephadm_command_timeout": { 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T01:27:23.323 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "900", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.324 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "default_registry": { 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_registry", 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.325 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io", 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.326 INFO:teuthology.orchestra.run.smithi138.stdout: "device_cache_timeout": { 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_cache_timeout", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1800", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.327 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache device inventory", 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "device_enhanced_scan": { 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_enhanced_scan", 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.328 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.329 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "facts_cache_timeout": { 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "facts_cache_timeout", 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.330 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache host facts data", 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "grafana_dashboards_path": { 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "grafana_dashboards_path", 2024-08-20T01:27:23.331 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T01:27:23.332 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "host_check_interval": { 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "host_check_interval", 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:23.333 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to perform a host check", 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.334 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "hw_monitoring": { 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hw_monitoring", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.335 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "inventory_list_all": { 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "inventory_list_all", 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.336 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.337 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.338 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.339 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "log_refresh_metadata": { 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_refresh_metadata", 2024-08-20T01:27:23.340 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.341 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.342 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.344 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.345 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.346 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.348 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T01:27:23.349 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "*", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T01:27:23.350 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "max_count_per_host": { 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_count_per_host", 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.351 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of daemons per service per host", 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.352 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd_draining_count": { 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_osd_draining_count", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.353 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "migration_current": { 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "migration_current", 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.354 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "internal - do not modify", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.355 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "root", 2024-08-20T01:27:23.356 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm-package", 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "root" 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.357 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "oob_default_addr": { 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "oob_default_addr", 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "169.254.1.1", 2024-08-20T01:27:23.358 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "prometheus_alerts_path": { 2024-08-20T01:27:23.359 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus_alerts_path", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.360 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_insecure": { 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_insecure", 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.361 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.362 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_password": { 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_password", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.363 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_url": { 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_url", 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.364 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.365 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_username": { 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_username", 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.366 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "secure_monitoring_stack": { 2024-08-20T01:27:23.367 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secure_monitoring_stack", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.368 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "service_discovery_port": { 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "service_discovery_port", 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8765", 2024-08-20T01:27:23.369 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "cephadm service discovery port", 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.370 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.371 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "ssh_config_file": { 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssh_config_file", 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.372 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.373 INFO:teuthology.orchestra.run.smithi138.stdout: "use_agent": { 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_agent", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.374 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "use_repo_digest": { 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_repo_digest", 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_failed_host_check": { 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_failed_host_check", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.377 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_daemons": { 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_daemons", 2024-08-20T01:27:23.378 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.379 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_hosts": { 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_hosts", 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.380 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.381 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crash", 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.382 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.383 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.384 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.384 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.384 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.385 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.385 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.386 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.387 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.388 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.389 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.390 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.391 INFO:teuthology.orchestra.run.smithi138.stdout: "retain_interval": { 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retain_interval", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "31536000", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.392 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.393 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.394 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_recent_interval": { 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_recent_interval", 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1209600", 2024-08-20T01:27:23.395 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.396 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dashboard", 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T01:27:23.397 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.398 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.399 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.400 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.401 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_ENABLED": { 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.402 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.404 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T01:27:23.405 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.406 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.407 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.408 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.409 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.410 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.411 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.412 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.413 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.414 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.415 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T01:27:23.416 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.417 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.418 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.419 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.420 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.421 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.422 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_URL": { 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_URL", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.423 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.424 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.426 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.427 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.428 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.429 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T01:27:23.430 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.431 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.432 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "[]", 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.433 INFO:teuthology.orchestra.run.smithi138.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "{}", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.434 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.435 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.436 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.437 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T01:27:23.438 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.439 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.440 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.441 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.442 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.443 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.444 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.445 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.446 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.447 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T01:27:23.448 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.449 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.450 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.451 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.452 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T01:27:23.453 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.454 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.455 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8", 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "45", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.457 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T01:27:23.458 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.459 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.460 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.461 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.462 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T01:27:23.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.464 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.465 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.466 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.467 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.468 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.470 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "cross_origin_url": { 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cross_origin_url", 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.471 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.472 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "crt_file": { 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crt_file", 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.473 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "debug": { 2024-08-20T01:27:23.474 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "debug", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable/disable debug options", 2024-08-20T01:27:23.475 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "jwt_token_ttl": { 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "jwt_token_ttl", 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.476 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "28800", 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.477 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.478 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.479 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.480 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.481 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.482 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.483 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.484 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.485 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.486 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "motd": { 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "motd", 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.487 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "The message of the day", 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect_resolve_ip_addr": { 2024-08-20T01:27:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.489 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.490 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.491 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8080", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.492 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.493 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.494 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.495 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.496 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl_server_port": { 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl_server_port", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8443", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.497 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.498 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "redirect", 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect" 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.499 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "url_prefix": { 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url_prefix", 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.502 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.503 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "devicehealth", 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_monitoring": { 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_monitoring", 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.504 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "monitor device health metrics", 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.505 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.506 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.507 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.508 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.509 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.510 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.512 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "mark_out_threshold": { 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mark_out_threshold", 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2419200", 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.514 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": { 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_name", 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "device_health_metrics", 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.515 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.516 INFO:teuthology.orchestra.run.smithi138.stdout: "retention_period": { 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retention_period", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15552000", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.517 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain device health metrics", 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_frequency": { 2024-08-20T01:27:23.518 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_frequency", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T01:27:23.519 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "self_heal": { 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "self_heal", 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.520 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.521 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.522 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.523 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_threshold": { 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_threshold", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7257600", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.525 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.526 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "diskprediction_local", 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.527 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.528 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.529 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.530 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.531 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.532 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.533 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.534 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "predict_interval": { 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predict_interval", 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:23.535 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "predictor_model": { 2024-08-20T01:27:23.536 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predictor_model", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "prophetstor", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.537 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.538 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.539 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.540 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.541 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "influx", 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": false, 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "influxdb python module not found", 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "batch_size": { 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "batch_size", 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5000", 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.543 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "database": { 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "database", 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "ceph", 2024-08-20T01:27:23.544 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": { 2024-08-20T01:27:23.545 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hostname", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server hostname", 2024-08-20T01:27:23.546 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "30", 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "5", 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.548 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.549 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.550 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.551 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.553 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.554 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.555 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "password": { 2024-08-20T01:27:23.556 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "password", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "password of InfluxDB server user", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "port": { 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "port", 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.558 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8086", 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server port", 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.560 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:23.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "false", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "threads": { 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threads", 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:23.563 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "32", 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "username": { 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "username", 2024-08-20T01:27:23.564 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "username of InfluxDB server user", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.565 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "verify_ssl": { 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "verify_ssl", 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.566 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "true", 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.567 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "insights", 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.569 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.570 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.571 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.573 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.574 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.575 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.576 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.577 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "iostat", 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.578 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.579 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.580 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.581 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.582 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.583 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.586 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.587 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.589 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "k8sevents", 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_event_retention_days": { 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ceph_event_retention_days", 2024-08-20T01:27:23.591 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.592 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "config_check_secs": { 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_check_secs", 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.593 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "10", 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.594 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.595 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.596 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.597 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.598 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.599 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.600 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "localpool", 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.604 INFO:teuthology.orchestra.run.smithi138.stdout: "failure_domain": { 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "failure_domain", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "host", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.605 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "failure domain for any created local pool", 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.606 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.607 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.608 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.609 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.610 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.611 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.613 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": { 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_size", 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.614 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default min_size for any created local pool", 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "num_rep": { 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "num_rep", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.616 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default replica count for any created local pool", 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": { 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_num", 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.617 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "128", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.618 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "prefix": { 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prefix", 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.619 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name prefix for any created local pool", 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.620 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.621 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "subtree": { 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subtree", 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "rack", 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.622 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.623 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mds_autoscaler", 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.625 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.627 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.628 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.629 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.630 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.631 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.632 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.633 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirroring", 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.635 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.636 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.637 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.638 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.639 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.640 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.641 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.642 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.643 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "nfs", 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.644 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.645 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.646 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.647 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.648 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.649 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.650 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.651 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.652 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.653 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "fail_fs": { 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "fail_fs", 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.655 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.657 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.658 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.659 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.661 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.662 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.663 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator": { 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.665 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm", 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "rook", 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "test_orchestrator" 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Orchestrator backend", 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.666 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.667 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.668 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_perf_query", 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.669 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.670 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.673 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.674 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.675 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.676 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.677 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_support", 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.680 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.681 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.682 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.683 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.684 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.685 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.686 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.687 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.688 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_autoscaler", 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.689 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.690 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.691 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.692 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.693 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.694 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.695 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.696 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.697 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.698 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.699 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.700 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "threshold": { 2024-08-20T01:27:23.701 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threshold", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1.0", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "scaling threshold", 2024-08-20T01:27:23.702 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "progress", 2024-08-20T01:27:23.703 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_pg_recovery_event": { 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_pg_recovery_event", 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.704 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:23.705 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.706 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.708 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.709 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.710 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.711 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.712 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.713 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.714 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "max_completed_events": { 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_completed_events", 2024-08-20T01:27:23.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "50", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "number of past completed events to remember", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.716 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.717 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long the module is going to sleep", 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.719 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.720 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus", 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "cache": { 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cache", 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.721 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "exclude_perf_counters": { 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "exclude_perf_counters", 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.723 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.724 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.725 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.726 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.727 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.728 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.729 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.730 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.732 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools": { 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools", 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.733 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.734 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "300", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.735 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_interval": { 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_interval", 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.736 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15.0", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.738 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "9283", 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.740 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.741 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.742 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "stale_cache_strategy": { 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stale_cache_strategy", 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.743 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "log", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.744 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.745 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "default", 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "default", 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "error" 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.746 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.747 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.748 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_support", 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.749 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.750 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.751 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.752 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.753 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.754 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.755 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.756 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.757 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_snap_create": { 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_snap_create", 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:23.758 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.759 INFO:teuthology.orchestra.run.smithi138.stdout: "mirror_snapshot_schedule": { 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.760 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.761 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.762 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "trash_purge_schedule": { 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "trash_purge_schedule", 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.763 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.764 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "restful", 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_auth": { 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_auth", 2024-08-20T01:27:23.765 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.766 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.767 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.768 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.769 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.770 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.771 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.772 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.773 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.774 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.775 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.776 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.777 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:23.778 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.779 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.780 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rgw", 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.782 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.783 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.784 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.785 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.786 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.787 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.788 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.789 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.790 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.791 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.793 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.793 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.794 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.795 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rook", 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.796 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.797 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.799 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.800 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.801 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.802 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.803 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.804 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.805 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.806 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_class": { 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "storage_class", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "local", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.807 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "selftest", 2024-08-20T01:27:23.808 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.809 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.810 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.811 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.812 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.813 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.814 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.815 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.816 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "roption1": { 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption1", 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.817 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.818 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "roption2": { 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption2", 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "xyz", 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.819 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption1": { 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption1", 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.820 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.821 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption2": { 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption2", 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.822 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption3": { 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption3", 2024-08-20T01:27:23.823 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.824 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption4": { 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption4", 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.825 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.826 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption5": { 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption5", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.827 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption6": { 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption6", 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.828 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.829 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption7": { 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption7", 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:23.830 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "42", 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.831 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.832 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "testkey": { 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testkey", 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.833 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.834 INFO:teuthology.orchestra.run.smithi138.stdout: "testlkey": { 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testlkey", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.835 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "testnewline": { 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testnewline", 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.836 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.837 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smb", 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.838 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "internal_store_backend": { 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "internal_store_backend", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.839 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.840 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.841 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.842 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.843 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.844 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.845 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.846 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.847 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.848 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.849 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "update_orchestration": { 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "update_orchestration", 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.850 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.851 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snap_schedule", 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.852 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_m_granularity": { 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_m_granularity", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.853 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "dump_on_update": { 2024-08-20T01:27:23.854 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dump_on_update", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.855 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "dump database to debug log on update", 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.856 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.857 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.859 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.860 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.861 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.862 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.863 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.864 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.865 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stats", 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.866 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.867 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.868 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.869 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.870 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.871 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.872 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.873 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.874 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.875 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.876 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "status", 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.877 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.878 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.879 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.880 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.881 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.882 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.883 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.884 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.885 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.886 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.887 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telegraf", 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.888 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "address": { 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "address", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.889 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.890 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.891 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.892 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.893 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.894 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.895 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.896 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.897 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.898 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.899 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.900 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.901 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.902 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telemetry", 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_basic": { 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_basic", 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.903 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.904 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_crash": { 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_crash", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.905 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_device": { 2024-08-20T01:27:23.906 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_device", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T01:27:23.907 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_ident": { 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_ident", 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.908 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.909 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_perf": { 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_perf", 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.910 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "contact": { 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "contact", 2024-08-20T01:27:23.911 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.912 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "description": { 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "description", 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.913 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.914 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "device_url": { 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_url", 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.915 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:23.916 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.917 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.918 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "24", 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "8", 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.919 INFO:teuthology.orchestra.run.smithi138.stdout: "last_opt_revision": { 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "last_opt_revision", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.920 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard": { 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard", 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.921 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.922 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard_description": { 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard_description", 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.923 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.924 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.925 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.926 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.927 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.928 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.929 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.930 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.931 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.932 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "organization": { 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "organization", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.933 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "proxy": { 2024-08-20T01:27:23.934 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "proxy", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.935 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.936 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.937 INFO:teuthology.orchestra.run.smithi138.stdout: "url": { 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.938 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.939 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "test_orchestrator", 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.940 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.941 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.942 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.943 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.944 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.945 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.946 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.947 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.948 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.949 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.950 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "volumes", 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.951 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.952 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.953 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.954 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.955 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.956 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.957 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.958 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_clones": { 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_clones", 2024-08-20T01:27:23.959 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.960 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "periodic_async_work": { 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "periodic_async_work", 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.961 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Periodically check for async work", 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.962 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_delay": { 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_delay", 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.963 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_no_wait": { 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T01:27:23.964 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T01:27:23.965 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.966 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.967 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix", 2024-08-20T01:27:23.968 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "discovery_interval": { 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "discovery_interval", 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "100", 2024-08-20T01:27:23.969 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.970 INFO:teuthology.orchestra.run.smithi138.stdout: "identifier": { 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "identifier", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.971 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.972 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.973 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.974 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.975 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.976 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.977 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.978 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.979 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.980 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.981 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.982 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.983 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_host": { 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_host", 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.984 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_port": { 2024-08-20T01:27:23.985 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_port", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10051", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.986 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_sender": { 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_sender", 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.987 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.988 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.989 INFO:teuthology.orchestra.run.smithi138.stdout: "modules": [ 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm", 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "dashboard", 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "iostat", 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "nfs", 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "restful", 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "snap_schedule" 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.990 INFO:teuthology.orchestra.run.smithi138.stdout: "available_modules": [ 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "alerts", 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:23.991 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How frequently to reexamine health status", 2024-08-20T01:27:23.992 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.993 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:23.994 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:23.998 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:23.999 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.000 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.001 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.002 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.003 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_destination": { 2024-08-20T01:27:24.004 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_destination", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email address to send alerts to", 2024-08-20T01:27:24.005 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_from_name": { 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_from_name", 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.006 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "Ceph", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Email From: name", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_host": { 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_host", 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.008 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP server", 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_password": { 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_password", 2024-08-20T01:27:24.009 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Password to authenticate with", 2024-08-20T01:27:24.010 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_port": { 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_port", 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.011 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "465", 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP port", 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.012 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_sender": { 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_sender", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.013 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SMTP envelope sender", 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_ssl": { 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_ssl", 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.014 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use SSL to connect to SMTP server", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.015 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "smtp_user": { 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smtp_user", 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.016 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "User to authenticate as", 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.017 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.018 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "balancer", 2024-08-20T01:27:24.019 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "active": { 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "active", 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.020 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically balance PGs across cluster", 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.021 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_time": { 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_time", 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0000", 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.022 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "beginning time of day to automatically balance", 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "begin_weekday": { 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "begin_weekday", 2024-08-20T01:27:24.023 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-08-20T01:27:24.024 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_max_iterations": { 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_max_iterations", 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:24.025 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "25", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "250", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.026 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_metrics": { 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_metrics", 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "pgs,objects,bytes", 2024-08-20T01:27:24.027 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.028 INFO:teuthology.orchestra.run.smithi138.stdout: "crush_compat_step": { 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crush_compat_step", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.5", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0.001", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "0.999", 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.029 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "aggressiveness of optimization", 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "end_time": { 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_time", 2024-08-20T01:27:24.030 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2359", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ending time of day to automatically balance", 2024-08-20T01:27:24.031 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "end_weekday": { 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "end_weekday", 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.032 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "0", 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "6", 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.033 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.034 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.035 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.036 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.037 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.038 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.039 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.040 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.041 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "min_score": { 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_score", 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.042 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.043 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "upmap", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.044 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "crush-compat", 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "none", 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "read", 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap", 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap-read" 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Balancer mode", 2024-08-20T01:27:24.045 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_ids": { 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_ids", 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.046 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.047 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.048 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.049 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.050 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_deviation": { 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_deviation", 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.051 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "deviation below which no optimization is attempted", 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.052 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "upmap_max_optimizations": { 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "upmap_max_optimizations", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.053 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.054 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm", 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_down_multiplier": { 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_down_multiplier", 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.055 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.056 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_refresh_rate": { 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_refresh_rate", 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "20", 2024-08-20T01:27:24.057 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.058 INFO:teuthology.orchestra.run.smithi138.stdout: "agent_starting_port": { 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "agent_starting_port", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4721", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.059 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_ptrace": { 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_ptrace", 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.060 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-08-20T01:27:24.061 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_interval": { 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_interval", 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:24.062 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to autotune daemon memory", 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.063 INFO:teuthology.orchestra.run.smithi138.stdout: "autotune_memory_target_ratio": { 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "autotune_memory_target_ratio", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0.7", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.064 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm_log_destination": { 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cephadm_log_destination", 2024-08-20T01:27:24.065 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "file", 2024-08-20T01:27:24.066 INFO:teuthology.orchestra.run.smithi138.stdout: "file,syslog", 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "syslog" 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.067 INFO:teuthology.orchestra.run.smithi138.stdout: "cgroups_split": { 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cgroups_split", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.068 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "config_checks_enabled": { 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_checks_enabled", 2024-08-20T01:27:24.069 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-08-20T01:27:24.070 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "config_dashboard": { 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_dashboard", 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.071 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.072 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_alertmanager": { 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_alertmanager", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.073 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_base": { 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_base", 2024-08-20T01:27:24.074 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/ceph", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Container image name, without the tag", 2024-08-20T01:27:24.075 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_elasticsearch": { 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_elasticsearch", 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.076 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "elasticsearch container image", 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.077 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_grafana": { 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_grafana", 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-08-20T01:27:24.078 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_haproxy": { 2024-08-20T01:27:24.079 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_haproxy", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "HAproxy container image", 2024-08-20T01:27:24.080 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_agent": { 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_agent", 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.081 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger agent container image", 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.082 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_collector": { 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_collector", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.083 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger collector container image", 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_jaeger_query": { 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_jaeger_query", 2024-08-20T01:27:24.084 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Jaeger query container image", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.085 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_keepalived": { 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_keepalived", 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.086 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Keepalived container image", 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_loki": { 2024-08-20T01:27:24.087 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_loki", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.088 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Loki container image", 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nginx": { 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nginx", 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.089 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nginx container image", 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.090 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_node_exporter": { 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_node_exporter", 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.091 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_nvmeof": { 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_nvmeof", 2024-08-20T01:27:24.092 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Nvme-of container image", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.093 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_prometheus": { 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_prometheus", 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.094 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Prometheus container image", 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.095 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_promtail": { 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_promtail", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.096 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Promtail container image", 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_samba": { 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_samba", 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.097 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Samba/SMB container image", 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.098 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "container_image_snmp_gateway": { 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_image_snmp_gateway", 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-08-20T01:27:24.099 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "SNMP Gateway container image", 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "container_init": { 2024-08-20T01:27:24.100 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "container_init", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.101 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Run podman/docker with `--init`", 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "daemon_cache_timeout": { 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "daemon_cache_timeout", 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.102 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.103 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "default_cephadm_command_timeout": { 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_cephadm_command_timeout", 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "900", 2024-08-20T01:27:24.104 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "default_registry": { 2024-08-20T01:27:24.105 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "default_registry", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "docker.io", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.106 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "device_cache_timeout": { 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_cache_timeout", 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.107 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1800", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache device inventory", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.108 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "device_enhanced_scan": { 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_enhanced_scan", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.109 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use libstoragemgmt during device scans", 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "facts_cache_timeout": { 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "facts_cache_timeout", 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.110 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "seconds to cache host facts data", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.111 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "grafana_dashboards_path": { 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "grafana_dashboards_path", 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.112 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "host_check_interval": { 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "host_check_interval", 2024-08-20T01:27:24.113 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to perform a host check", 2024-08-20T01:27:24.114 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "hw_monitoring": { 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hw_monitoring", 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.115 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.116 INFO:teuthology.orchestra.run.smithi138.stdout: "inventory_list_all": { 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "inventory_list_all", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.117 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.118 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.119 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.120 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "log_refresh_metadata": { 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_refresh_metadata", 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.121 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.122 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.123 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.124 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.125 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.126 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.127 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf": { 2024-08-20T01:27:24.128 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-08-20T01:27:24.129 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.130 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "*", 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.131 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "max_count_per_host": { 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_count_per_host", 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.132 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of daemons per service per host", 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "max_osd_draining_count": { 2024-08-20T01:27:24.133 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_osd_draining_count", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.134 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "migration_current": { 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "migration_current", 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.135 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "internal - do not modify", 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.136 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "mode": { 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mode", 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "root", 2024-08-20T01:27:24.137 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm-package", 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "root" 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "mode for remote execution of cephadm", 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.138 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "oob_default_addr": { 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "oob_default_addr", 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "169.254.1.1", 2024-08-20T01:27:24.139 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Default address for RedFish API (oob management).", 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.140 INFO:teuthology.orchestra.run.smithi138.stdout: "prometheus_alerts_path": { 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus_alerts_path", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.141 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_insecure": { 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_insecure", 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.142 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.143 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_password": { 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_password", 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.144 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.145 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_url": { 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_url", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.146 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "registry_username": { 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "registry_username", 2024-08-20T01:27:24.147 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-08-20T01:27:24.148 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "secure_monitoring_stack": { 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secure_monitoring_stack", 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.149 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.150 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "service_discovery_port": { 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "service_discovery_port", 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8765", 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.151 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "cephadm service discovery port", 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.152 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.153 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "ssh_config_file": { 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssh_config_file", 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.154 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.155 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "use_agent": { 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_agent", 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.156 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "use_repo_digest": { 2024-08-20T01:27:24.157 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "use_repo_digest", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.158 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_failed_host_check": { 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_failed_host_check", 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.159 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if the host check fails", 2024-08-20T01:27:24.160 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_daemons": { 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_daemons", 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.161 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.162 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_on_stray_hosts": { 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_on_stray_hosts", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.163 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.164 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crash", 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.165 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.166 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.167 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.168 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.169 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.170 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.171 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.172 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.173 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "retain_interval": { 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retain_interval", 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "31536000", 2024-08-20T01:27:24.174 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain crashes before pruning them", 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.175 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.176 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_recent_interval": { 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_recent_interval", 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.177 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1209600", 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "time interval in which to warn about recent crashes", 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.178 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dashboard", 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.179 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.180 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_HOST": { 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_HOST", 2024-08-20T01:27:24.181 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.182 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.183 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.184 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_ENABLED": { 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_ENABLED", 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.185 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-08-20T01:27:24.186 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.187 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "ENABLE_BROWSABLE_API": { 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ENABLE_BROWSABLE_API", 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.188 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.189 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.190 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.191 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.192 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.193 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.194 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.195 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_NFS": { 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-08-20T01:27:24.196 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.197 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RBD": { 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.198 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.202 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.203 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "FEATURE_TOGGLE_RGW": { 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.204 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-08-20T01:27:24.205 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.206 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_PASSWORD": { 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_PASSWORD", 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.207 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.208 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.209 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.210 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_URL": { 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_URL", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.211 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_API_USERNAME": { 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_API_USERNAME", 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.212 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.213 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.214 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-08-20T01:27:24.215 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.216 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.217 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.218 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "ISSUE_TRACKER_API_KEY": { 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.219 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "MANAGED_BY_CLUSTERS": { 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-08-20T01:27:24.220 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "[]", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.221 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "MULTICLUSTER_CONFIG": { 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "MULTICLUSTER_CONFIG", 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.222 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "{}", 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.223 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_HOST": { 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_HOST", 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.224 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-08-20T01:27:24.225 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.226 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.227 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.228 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.229 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.230 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.231 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-08-20T01:27:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.233 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.234 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.235 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.236 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-08-20T01:27:24.237 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.238 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_ENABLED": { 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_ENABLED", 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.239 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.240 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.241 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-08-20T01:27:24.242 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.243 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8", 2024-08-20T01:27:24.244 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.245 INFO:teuthology.orchestra.run.smithi138.stdout: "REST_REQUESTS_TIMEOUT": { 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "45", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.246 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ACCESS_KEY": { 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ACCESS_KEY", 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.247 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.248 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.249 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "admin", 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.250 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SECRET_KEY": { 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SECRET_KEY", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.251 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "RGW_API_SSL_VERIFY": { 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "RGW_API_SSL_VERIFY", 2024-08-20T01:27:24.252 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.253 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "UNSAFE_TLS_v1_2": { 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "UNSAFE_TLS_v1_2", 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.254 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.255 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.256 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.257 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.258 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:24.259 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "cross_origin_url": { 2024-08-20T01:27:24.260 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cross_origin_url", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.261 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "crt_file": { 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "crt_file", 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.262 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.263 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "debug": { 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "debug", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.264 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Enable/disable debug options", 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "jwt_token_ttl": { 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "jwt_token_ttl", 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.265 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "28800", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.266 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.267 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.268 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.269 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.270 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.271 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.272 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.273 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.274 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.275 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "motd": { 2024-08-20T01:27:24.276 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "motd", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "The message of the day", 2024-08-20T01:27:24.277 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect_resolve_ip_addr": { 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "redirect_resolve_ip_addr", 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.278 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.279 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.280 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.281 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8080", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.282 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.283 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.284 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.285 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl_server_port": { 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl_server_port", 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.286 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8443", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.287 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.288 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "redirect", 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "redirect" 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.289 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.290 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.291 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "url_prefix": { 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url_prefix", 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.292 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.293 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "devicehealth", 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_monitoring": { 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_monitoring", 2024-08-20T01:27:24.294 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "monitor device health metrics", 2024-08-20T01:27:24.295 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.296 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.297 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.298 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.299 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.300 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.301 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.302 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.303 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "mark_out_threshold": { 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mark_out_threshold", 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.304 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "2419200", 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.305 INFO:teuthology.orchestra.run.smithi138.stdout: "pool_name": { 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pool_name", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "device_health_metrics", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.306 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name of pool in which to store device health metrics", 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "retention_period": { 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "retention_period", 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.307 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15552000", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long to retain device health metrics", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.308 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_frequency": { 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_frequency", 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:24.309 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to scrape device health metrics", 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "self_heal": { 2024-08-20T01:27:24.310 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "self_heal", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-08-20T01:27:24.311 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.312 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how frequently to wake up and check device health", 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.313 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.314 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "warn_threshold": { 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "warn_threshold", 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.315 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7257600", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.316 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "diskprediction_local", 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.317 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.318 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.319 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.320 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.321 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.322 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.323 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.324 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.325 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "predict_interval": { 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predict_interval", 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.326 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "86400", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.327 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "predictor_model": { 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "predictor_model", 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "prophetstor", 2024-08-20T01:27:24.328 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.329 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "600", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.330 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.331 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.332 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "influx", 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": false, 2024-08-20T01:27:24.333 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "influxdb python module not found", 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "batch_size": { 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "batch_size", 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5000", 2024-08-20T01:27:24.334 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.335 INFO:teuthology.orchestra.run.smithi138.stdout: "database": { 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "database", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "ceph", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.336 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": { 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "hostname", 2024-08-20T01:27:24.337 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.338 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server hostname", 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.339 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "30", 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "5", 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.340 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.341 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.342 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.343 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.344 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.345 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.346 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.347 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.348 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "password": { 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "password", 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.349 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "password of InfluxDB server user", 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.350 INFO:teuthology.orchestra.run.smithi138.stdout: "port": { 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "port", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "8086", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.351 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "InfluxDB server port", 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.352 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.353 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "ssl": { 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ssl", 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "false", 2024-08-20T01:27:24.354 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.355 INFO:teuthology.orchestra.run.smithi138.stdout: "threads": { 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threads", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "32", 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.356 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "username": { 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "username", 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.357 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "username of InfluxDB server user", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.358 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "verify_ssl": { 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "verify_ssl", 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "true", 2024-08-20T01:27:24.359 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.360 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "insights", 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.361 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.362 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.363 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.364 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.365 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.366 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.367 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.368 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.370 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.371 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "iostat", 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.372 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.373 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.374 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.375 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.376 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.377 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.378 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.379 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.380 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.381 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.382 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "k8sevents", 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_event_retention_days": { 2024-08-20T01:27:24.383 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "ceph_event_retention_days", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "7", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.384 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Days to hold ceph event information within local cache", 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "config_check_secs": { 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "config_check_secs", 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.385 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "10", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.386 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.387 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.388 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.389 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.390 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.391 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.392 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.393 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.394 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.395 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.396 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "localpool", 2024-08-20T01:27:24.397 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "failure_domain": { 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "failure_domain", 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "host", 2024-08-20T01:27:24.398 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "failure domain for any created local pool", 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.399 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.400 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.401 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.402 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.403 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.406 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.406 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.406 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.406 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.407 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.408 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.409 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.410 INFO:teuthology.orchestra.run.smithi138.stdout: "min_size": { 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "min_size", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.411 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default min_size for any created local pool", 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "num_rep": { 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "num_rep", 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.412 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default replica count for any created local pool", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.413 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_num": { 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_num", 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "128", 2024-08-20T01:27:24.414 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "default pg_num for any created local pool", 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.415 INFO:teuthology.orchestra.run.smithi138.stdout: "prefix": { 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prefix", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.416 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "name prefix for any created local pool", 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.417 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.418 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "subtree": { 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "subtree", 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.419 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "rack", 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "CRUSH level for which to create a local pool", 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.420 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mds_autoscaler", 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.421 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.422 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.423 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.424 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.425 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.426 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.427 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.428 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.429 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.430 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.431 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirroring", 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.432 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.433 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.434 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.435 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.436 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.437 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.438 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.439 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.440 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.441 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.442 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "nfs", 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.443 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.444 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.445 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.446 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.447 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.448 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.449 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.450 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.451 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.452 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.453 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.454 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "fail_fs": { 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "fail_fs", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.455 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.456 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.457 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.458 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.459 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.460 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.461 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.462 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.463 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.464 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator": { 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "orchestrator", 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.465 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "cephadm", 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "rook", 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "test_orchestrator" 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Orchestrator backend", 2024-08-20T01:27:24.466 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.467 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_perf_query", 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.469 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.470 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.471 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.472 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.473 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.474 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.475 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.476 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.477 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.478 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.479 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "osd_support", 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.480 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.481 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.482 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.483 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.484 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.485 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.486 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.487 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.488 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.489 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.490 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "pg_autoscaler", 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.491 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.492 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.493 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.494 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.495 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.496 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.497 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.498 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:24.499 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.500 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.501 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.502 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "threshold": { 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "threshold", 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "3.0", 2024-08-20T01:27:24.503 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1.0", 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "scaling threshold", 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.504 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "progress", 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_pg_recovery_event": { 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_pg_recovery_event", 2024-08-20T01:27:24.505 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow the module to show pg recovery progress", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.506 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.507 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.508 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.509 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.510 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.511 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.512 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.513 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.514 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.515 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.516 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "max_completed_events": { 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_completed_events", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "50", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.517 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "number of past completed events to remember", 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "sleep_interval": { 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sleep_interval", 2024-08-20T01:27:24.518 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "how long the module is going to sleep", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.519 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.520 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.521 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "prometheus", 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "cache": { 2024-08-20T01:27:24.522 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "cache", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.523 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "exclude_perf_counters": { 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "exclude_perf_counters", 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.524 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.525 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.526 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.527 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.528 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.529 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.530 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.531 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.532 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools": { 2024-08-20T01:27:24.533 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.534 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_stats_pools_refresh_interval": { 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.535 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "300", 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.536 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "scrape_interval": { 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "scrape_interval", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15.0", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.537 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:24.538 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "::", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.539 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.540 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "9283", 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.541 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "stale_cache_strategy": { 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stale_cache_strategy", 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.543 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "log", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.544 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_behaviour": { 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_behaviour", 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "default", 2024-08-20T01:27:24.545 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "default", 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "error" 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.546 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "standby_error_status_code": { 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "standby_error_status_code", 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.547 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "500", 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "400", 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "599", 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.548 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_support", 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.549 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.550 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.551 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.552 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.553 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.554 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.555 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.556 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.557 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_snap_create": { 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_snap_create", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.558 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "mirror_snapshot_schedule": { 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "mirror_snapshot_schedule", 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.559 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.560 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.561 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "trash_purge_schedule": { 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "trash_purge_schedule", 2024-08-20T01:27:24.562 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.563 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "restful", 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.564 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "enable_auth": { 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enable_auth", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.565 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "key_file": { 2024-08-20T01:27:24.566 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "key_file", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.567 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.568 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.569 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.570 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.571 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.572 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.573 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.575 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "server_addr": { 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_addr", 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.576 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.577 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "server_port": { 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "server_port", 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.578 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.579 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.580 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rgw", 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.581 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.582 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.583 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.584 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.585 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.586 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.587 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.588 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.589 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "secondary_zone_period_retry_limit": { 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "secondary_zone_period_retry_limit", 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "5", 2024-08-20T01:27:24.590 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.591 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.592 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.593 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rook", 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.594 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.595 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.596 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.597 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.598 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.599 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.600 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.601 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.602 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "storage_class": { 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "storage_class", 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "local", 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.604 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.605 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "selftest", 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.606 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.609 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.609 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.610 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.611 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.612 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.613 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.614 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.615 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.616 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "roption1": { 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption1", 2024-08-20T01:27:24.617 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.618 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "roption2": { 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "roption2", 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.619 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "xyz", 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.620 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption1": { 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption1", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.621 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption2": { 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption2", 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.622 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.623 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption3": { 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption3", 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "float", 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.624 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.625 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption4": { 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption4", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.626 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption5": { 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption5", 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.627 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.628 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption6": { 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption6", 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.629 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.630 INFO:teuthology.orchestra.run.smithi138.stdout: "rwoption7": { 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rwoption7", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "1", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "42", 2024-08-20T01:27:24.631 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.632 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.633 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "testkey": { 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testkey", 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.634 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.635 INFO:teuthology.orchestra.run.smithi138.stdout: "testlkey": { 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testlkey", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.636 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "testnewline": { 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "testnewline", 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.637 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.638 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "smb", 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.639 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "internal_store_backend": { 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "internal_store_backend", 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.640 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.641 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.642 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.643 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.644 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.645 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.646 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.647 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.648 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.649 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.650 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.651 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "update_orchestration": { 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "update_orchestration", 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.652 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.653 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snap_schedule", 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "allow_m_granularity": { 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "allow_m_granularity", 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.654 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "allow minute scheduled snapshots", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.655 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "dump_on_update": { 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "dump_on_update", 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.656 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "dump database to debug log on update", 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.657 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.658 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.659 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.660 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.661 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.662 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.663 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.664 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.665 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.666 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.667 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "stats", 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.668 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.669 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.670 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.671 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.672 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.673 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.674 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.675 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.676 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.677 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "status", 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.678 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.679 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.680 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.681 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.682 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.683 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.684 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.685 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.686 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.687 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.688 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telegraf", 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "address": { 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "address", 2024-08-20T01:27:24.689 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.690 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.691 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "15", 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.693 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.694 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.695 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.696 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.697 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.698 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.699 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.700 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.701 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.702 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "telemetry", 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_basic": { 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_basic", 2024-08-20T01:27:24.703 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share basic cluster information (size, version)", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.704 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_crash": { 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_crash", 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.705 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.706 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_device": { 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_device", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.707 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_ident": { 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_ident", 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.708 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "channel_perf": { 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "channel_perf", 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.710 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Share various performance metrics of a cluster", 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "contact": { 2024-08-20T01:27:24.711 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "contact", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.712 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "description": { 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "description", 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.713 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.714 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "device_url": { 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "device_url", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.715 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "enabled": { 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "enabled", 2024-08-20T01:27:24.716 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.717 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.718 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "24", 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "8", 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.719 INFO:teuthology.orchestra.run.smithi138.stdout: "last_opt_revision": { 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "last_opt_revision", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "1", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.720 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard": { 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard", 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.721 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.722 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "leaderboard_description": { 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "leaderboard_description", 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.723 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.724 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.725 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.726 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.727 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.728 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.729 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.730 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.731 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "organization": { 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "organization", 2024-08-20T01:27:24.732 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.733 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "proxy": { 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "proxy", 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.734 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.735 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.736 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "url": { 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "url", 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.737 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.738 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "test_orchestrator", 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.739 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.740 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.741 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.742 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.743 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.744 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.745 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.746 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.747 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.748 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.749 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "volumes", 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.750 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.751 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.752 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.753 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.754 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.755 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.756 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.757 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.758 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "max_concurrent_clones": { 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "max_concurrent_clones", 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.759 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "4", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Number of asynchronous cloner threads", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.760 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "periodic_async_work": { 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "periodic_async_work", 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.761 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Periodically check for async work", 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.762 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_delay": { 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_delay", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-08-20T01:27:24.763 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "snapshot_clone_no_wait": { 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snapshot_clone_no_wait", 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.764 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "True", 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.765 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.766 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.767 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix", 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "can_run": true, 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "error_string": "", 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "module_options": { 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "discovery_interval": { 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "discovery_interval", 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "uint", 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.768 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "100", 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.769 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "identifier": { 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "identifier", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.770 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "interval": { 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "interval", 2024-08-20T01:27:24.771 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "secs", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "60", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.772 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "log_level": { 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_level", 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.773 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.774 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster": { 2024-08-20T01:27:24.775 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.776 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_cluster_level": { 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_cluster_level", 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.777 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "info", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [ 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "critical", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "debug", 2024-08-20T01:27:24.778 INFO:teuthology.orchestra.run.smithi138.stdout: "error", 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "info", 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "warning" 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.779 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "log_to_file": { 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "log_to_file", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "bool", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "False", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.780 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "sqlite3_killpoint": { 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "sqlite3_killpoint", 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.781 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "dev", 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 1, 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "0", 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.782 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_host": { 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_host", 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.783 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "", 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_port": { 2024-08-20T01:27:24.784 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_port", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "int", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "10051", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.785 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "zabbix_sender": { 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "zabbix_sender", 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "str", 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "level": "advanced", 2024-08-20T01:27:24.786 INFO:teuthology.orchestra.run.smithi138.stdout: "flags": 0, 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "min": "", 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "max": "", 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "enum_allowed": [], 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "desc": "", 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "long_desc": "", 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "tags": [], 2024-08-20T01:27:24.787 INFO:teuthology.orchestra.run.smithi138.stdout: "see_also": [] 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: "services": { 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: "dashboard": "https://172.21.15.138:8443/" 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: "always_on_modules": { 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: "octopus": [ 2024-08-20T01:27:24.788 INFO:teuthology.orchestra.run.smithi138.stdout: "balancer", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "crash", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "devicehealth", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscaler", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "progress", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_support", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "status", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "telemetry", 2024-08-20T01:27:24.789 INFO:teuthology.orchestra.run.smithi138.stdout: "volumes" 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "pacific": [ 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "balancer", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "crash", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "devicehealth", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscaler", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "progress", 2024-08-20T01:27:24.790 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_support", 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "status", 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "telemetry", 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "volumes" 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "quincy": [ 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "balancer", 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "crash", 2024-08-20T01:27:24.791 INFO:teuthology.orchestra.run.smithi138.stdout: "devicehealth", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscaler", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "progress", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_support", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "status", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "telemetry", 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: "volumes" 2024-08-20T01:27:24.792 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "reef": [ 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "balancer", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "crash", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "devicehealth", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscaler", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "progress", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_support", 2024-08-20T01:27:24.793 INFO:teuthology.orchestra.run.smithi138.stdout: "status", 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "telemetry", 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "volumes" 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: ], 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "squid": [ 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "balancer", 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "crash", 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "devicehealth", 2024-08-20T01:27:24.794 INFO:teuthology.orchestra.run.smithi138.stdout: "orchestrator", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "pg_autoscaler", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "progress", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "rbd_support", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "status", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "telemetry", 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "volumes" 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.795 INFO:teuthology.orchestra.run.smithi138.stdout: "last_failure_osd_epoch": 5, 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "active_clients": [ 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "devicehealth", 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 237245147 2024-08-20T01:27:24.796 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "libcephsqlite", 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.797 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2008609515 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "rbd_support", 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.798 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1795942723 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snap_schedule", 2024-08-20T01:27:24.799 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 2010663194 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.800 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "snap_schedule", 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 716343923 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:24.801 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: "name": "volumes", 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: "addrvec": [ 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: { 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v2", 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "172.21.15.138:0", 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 1934124547 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.802 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.803 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:24.803 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-08-20T01:27:24.803 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 msgr2=0x7fd6f00b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 0x7fd6f00b6200 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd70c106ea0 tx=0x7fd6f8005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:24.804 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 msgr2=0x7fd70c19aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00e9f0 tx=0x7fd6fc00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 shutdown_connections 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fd6f00b3d40 0x7fd6f00b6200 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd70c10ad90 0x7fd70c1a07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd70c10a410 0x7fd70c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 --2- 172.21.15.138:0/1011111035 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd70c104a10 0x7fd70c19aee0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 >> 172.21.15.138:0/1011111035 conn(0x7fd70c100430 msgr2=0x7fd70c102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 shutdown_connections 2024-08-20T01:27:24.805 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:22.381+0000 7fd711200640 1 -- 172.21.15.138:0/1011111035 wait complete. 2024-08-20T01:27:24.813 DEBUG:tasks.check_counter:Getting stats from a 2024-08-20T01:27:24.814 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.a perf dump 2024-08-20T01:27:24.814 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:22 smithi138 ceph-mon[18076]: pgmap v966: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-08-20T01:27:24.814 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:22 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2420215292' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-20T01:27:24.814 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:22 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1011111035' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T01:27:24.815 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:22 smithi142 ceph-mon[21799]: pgmap v966: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-08-20T01:27:24.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:22 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2420215292' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-20T01:27:24.816 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:22 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1011111035' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T01:27:24.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:22 smithi167 ceph-mon[21965]: pgmap v966: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.0 MiB/s wr, 173 op/s 2024-08-20T01:27:24.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:22 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2420215292' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} : dispatch 2024-08-20T01:27:24.816 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:22 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1011111035' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-08-20T01:27:25.199 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:25.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:24 smithi167 ceph-mon[21965]: pgmap v967: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:27:25.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:24 smithi142 ceph-mon[21799]: pgmap v967: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:27:25.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:24 smithi138 ceph-mon[18076]: pgmap v967: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 552 KiB/s rd, 1.7 MiB/s wr, 125 op/s 2024-08-20T01:27:25.848 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:25.848 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:25.848 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 360, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 354, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1011312, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 219136, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.076955094, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.041361758, 2024-08-20T01:27:25.849 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.043179964, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 353, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.024916135, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000070583 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:25.850 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1011312, 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 219136 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:25.851 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 2, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 271, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 320, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 207696, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.034848683, 2024-08-20T01:27:25.852 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.028936176, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.002751749, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 269, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.025627480, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000095269 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:25.853 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 207696 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:25.854 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 4, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 4, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 2944, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 448, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:25.855 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.003621532, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000060476, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.003441670, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.006075480, 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002025160 2024-08-20T01:27:25.856 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T01:27:25.857 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:25.858 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:25.859 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:25.860 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:25.861 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:25.862 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 150766, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 31, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:25.863 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-08-20T01:27:25.864 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:25.865 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 362, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 960428, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 362, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:25.866 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 362, 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 960428, 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:25.867 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:25.868 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:25.868 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:26.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:25 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:26.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:25 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:26.242 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:25 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:26.701 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-08-20T01:27:26.702 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-08-20T01:27:26.702 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-08-20T01:27:26.702 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-08-20T01:27:26.702 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-08-20T01:27:26.702 DEBUG:tasks.check_counter:Getting stats from d 2024-08-20T01:27:26.702 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.d perf dump 2024-08-20T01:27:27.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:26 smithi138 ceph-mon[18076]: pgmap v968: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:27:27.073 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:27.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:26 smithi167 ceph-mon[21965]: pgmap v968: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:27:27.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:26 smithi142 ceph-mon[21799]: pgmap v968: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 370, 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 362, 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1015888, 2024-08-20T01:27:27.676 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 223424, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.081441644, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.040280354, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.047021127, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:27.677 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 362, 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.042104104, 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000116309 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:27.678 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1015888, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 223424 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 275, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 160, 2024-08-20T01:27:27.679 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 128224, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.032555840, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.027649384, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.001574894, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 273, 2024-08-20T01:27:27.680 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.022695659, 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000083134 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.681 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 128224 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 0, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 2, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 0, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 0, 2024-08-20T01:27:27.682 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.000624668, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000000000, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.000465798, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:27.683 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.684 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:27.685 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:27.686 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:27.687 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:27.688 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:27.689 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 148967, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 29, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:27.690 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-08-20T01:27:27.691 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:27.692 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 367, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 961181, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 367, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:27.693 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 367, 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 961181, 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:27.694 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:28.546 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-08-20T01:27:28.547 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-08-20T01:27:28.547 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-08-20T01:27:28.547 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-08-20T01:27:28.547 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-08-20T01:27:28.547 DEBUG:tasks.check_counter:Getting stats from g 2024-08-20T01:27:28.547 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.g perf dump 2024-08-20T01:27:28.920 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:29.122 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:28 smithi138 ceph-mon[18076]: pgmap v969: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-08-20T01:27:29.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:28 smithi167 ceph-mon[21965]: pgmap v969: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-08-20T01:27:29.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:28 smithi142 ceph-mon[21799]: pgmap v969: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.3 MiB/s wr, 109 op/s 2024-08-20T01:27:29.559 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:29.559 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:29.559 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 102129, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 90467, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 34714711, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 117915368, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 10, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 5, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 11.368412593, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 5.365434671, 2024-08-20T01:27:29.560 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 5.349134970, 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 1.502522968, 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 90465, 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 8.846413494, 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000097788 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:29.561 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 144087, 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.026485407, 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000183 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:29.562 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 143073, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 228264, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 47296490, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 254005219, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 17, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 7, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 37.732194644, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 28.216447175, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 11.447331246, 2024-08-20T01:27:29.563 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.669937522, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 228262, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 33.143264568, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000145198 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 292064, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.116343980, 2024-08-20T01:27:29.564 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000398 2024-08-20T01:27:29.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.565 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 4272, 2024-08-20T01:27:29.565 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 560128 2024-08-20T01:27:29.565 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.566 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:29.566 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 37669, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 44851, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 11529717, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 90358022, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 7, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 6, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 5.392098656, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 2.680570007, 2024-08-20T01:27:29.567 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 1.974019882, 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.751349108, 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 44851, 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 6.747535672, 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000150443 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:29.568 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 47741, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.015294938, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000320 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1211056, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 232144 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 1, 2024-08-20T01:27:29.569 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-MDSRank": { 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 37945, 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 52.718988330, 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001389352 2024-08-20T01:27:29.570 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "queue_len": 0, 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "complete_latency": { 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 26051, 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.881594780, 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000187386 2024-08-20T01:27:29.571 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "mds": { 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "request": 94825, 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "reply": 91678, 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "reply_latency": { 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 91678, 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 807.418918640, 2024-08-20T01:27:29.572 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.008807117 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_reply": 0, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "forward": 3147, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_complete": 1, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_commit": 38, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_split": 3, 2024-08-20T01:27:29.573 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_merge": 0, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes": 2211, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_top": 588, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_bottom": 1548, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pin_tail": 75, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_pinned": 1371, 2024-08-20T01:27:29.574 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_expired": 385, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "inodes_with_caps": 342, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "caps": 342, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "subtrees": 23, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse": 154329, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_hit": 82096, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_forward": 3139, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_discover": 113, 2024-08-20T01:27:29.575 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_dir_fetch": 0, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "traverse_lock": 956, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "load_cent": 41858, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "q": 0, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "exported": 7, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "exported_inodes": 301, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "imported": 24, 2024-08-20T01:27:29.576 INFO:teuthology.orchestra.run.smithi138.stdout: "imported_inodes": 1414, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rfiles": 4, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rbytes": 234, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "root_rsnaps": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:29.577 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_inode_file_caps": 23, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_revoke": 6487, 2024-08-20T01:27:29.578 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_grant": 63283, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flushsnap_ack": 29, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps": 25248, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_caps_dirty": 9116, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_cap_release": 1941, 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: "process_request_cap_release": 72526 2024-08-20T01:27:29.579 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_cache": { 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update": 26, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_update_receipt": 74, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_try_discover": 46, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_send_discover": 166, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "dir_handle_discover": 81, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays": 71, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:29.580 INFO:teuthology.orchestra.run.smithi138.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_created": 15732, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_enqueued": 15661, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "strays_migrated": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:29.581 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_started": 0, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "recovery_completed": 0, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_path": 17, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_quiesce_inode": 1586, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_exportdir": 7, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_flush": 0, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragmentdir": 3, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:29.582 INFO:teuthology.orchestra.run.smithi138.stdout: "ireq_inodestats": 0 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_log": { 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "evlrg": 0, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "evadd": 79968, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "evex": 0, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "evtrm": 0, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "ev": 79968, 2024-08-20T01:27:29.583 INFO:teuthology.orchestra.run.smithi138.stdout: "evexg": 0, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "evexd": 0, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segadd": 111, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segex": 0, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segtrm": 0, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "seg": 111, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segmjr": 8, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segexg": 0, 2024-08-20T01:27:29.584 INFO:teuthology.orchestra.run.smithi138.stdout: "segexd": 0, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "expos": 4194304, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "wrpos": 259020428, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "rdpos": 0, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "jlat": { 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 20505, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 123.489182177, 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.006022393 2024-08-20T01:27:29.585 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "replayed": 0 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_mem": { 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "ino": 1840, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "ino+": 17739, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "ino-": 15899, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "dir": 151, 2024-08-20T01:27:29.586 INFO:teuthology.orchestra.run.smithi138.stdout: "dir+": 199, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "dir-": 48, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "dn": 2210, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "dn+": 29727, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "dn-": 27517, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "cap": 342, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "cap+": 17217, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "cap-": 16875, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "rss": 74416, 2024-08-20T01:27:29.587 INFO:teuthology.orchestra.run.smithi138.stdout: "heap": 285069312 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_server": { 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_client_request": 145368, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "dispatch_server_request": 345, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_request": 94825, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_client_session": 1123, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "handle_peer_request": 9692, 2024-08-20T01:27:29.588 INFO:teuthology.orchestra.run.smithi138.stdout: "req_create_latency": { 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 15615, 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 310.637201354, 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.019893512 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getattr_latency": { 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 965, 2024-08-20T01:27:29.589 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 16.617903497, 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.017220625 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.590 INFO:teuthology.orchestra.run.smithi138.stdout: "req_link_latency": { 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookup_latency": { 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 22216, 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 48.773891616, 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002195439 2024-08-20T01:27:29.591 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupino_latency": { 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2, 2024-08-20T01:27:29.592 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000249526, 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000124763 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupname_latency": { 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.593 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.594 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.595 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.595 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.595 INFO:teuthology.orchestra.run.smithi138.stdout: "req_lssnap_latency": { 2024-08-20T01:27:29.595 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mkdir_latency": { 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.596 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mknod_latency": { 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "req_mksnap_latency": { 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.597 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "req_open_latency": { 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 9, 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.004574505, 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000508278 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "req_readdir_latency": { 2024-08-20T01:27:29.598 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 34417, 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 135.410818298, 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.003934416 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rename_latency": { 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 2421, 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 169.185633760, 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.069882541 2024-08-20T01:27:29.599 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.600 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmdir_latency": { 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:29.601 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.602 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setattr_latency": { 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 29, 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 4.701766358, 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.162129874 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.603 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 272, 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.450816172, 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.001657412 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.604 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setlayout_latency": { 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "req_setxattr_latency": { 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.605 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: "req_symlink_latency": { 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.606 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "req_unlink_latency": { 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 15732, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 121.636063554, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.007731760 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:29.607 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_sessions": { 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "session_count": 2, 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "session_add": 3, 2024-08-20T01:27:29.608 INFO:teuthology.orchestra.run.smithi138.stdout: "session_remove": 1, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_open": 2, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "sessions_stale": 0, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "total_load": 12688, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "average_load": 6344, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "avg_session_uptime": 1121, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-08-20T01:27:29.609 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:29.610 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:29.611 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:29.612 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:29.613 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 750253, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 1495, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 176, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 2, 2024-08-20T01:27:29.614 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 193, 2024-08-20T01:27:29.615 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 6479567, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 67485, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-08-20T01:27:29.616 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "objecter": { 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "op_active": 0, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "op_laggy": 0, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send": 54363, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "op_send_bytes": 258540097, 2024-08-20T01:27:29.617 INFO:teuthology.orchestra.run.smithi138.stdout: "op_resend": 30, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "op_reply": 54363, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "op_latency": { 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 54363, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 444.090411934, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.008168982 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "op_inflight": 0, 2024-08-20T01:27:29.618 INFO:teuthology.orchestra.run.smithi138.stdout: "oplen_avg": { 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 54363, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 55339 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "op": 54363, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "op_r": 695, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "op_w": 53668, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "op_rmw": 0, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "op_pg": 0, 2024-08-20T01:27:29.619 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_stat": 0, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_create": 10, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_read": 694, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_write": 21199, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writefull": 410, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_writesame": 0, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_append": 0, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_zero": 0, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_truncate": 3, 2024-08-20T01:27:29.620 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_delete": 31394, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_mapext": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_getxattr": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_setxattr": 20, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:29.621 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_call": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_watch": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_notify": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "osdop_other": 0, 2024-08-20T01:27:29.622 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_active": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_send": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_resend": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "linger_ping": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_active": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_send": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolop_resend": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_active": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_send": 0, 2024-08-20T01:27:29.623 INFO:teuthology.orchestra.run.smithi138.stdout: "poolstat_resend": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_active": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_send": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "statfs_resend": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "command_active": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "command_send": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "command_resend": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "map_epoch": 115, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "map_full": 0, 2024-08-20T01:27:29.624 INFO:teuthology.orchestra.run.smithi138.stdout: "map_inc": 29, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_sessions": 12, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_open": 12, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_session_close": 0, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_laggy": 0, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_wr": 1203, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_rd": 2, 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_del": 404 2024-08-20T01:27:29.625 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "oft": { 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_objs": 1, 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_updates": 0, 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "omap_total_removes": 0 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "purge_queue": { 2024-08-20T01:27:29.626 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_ops_high_water": 16, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing": 0, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executing_high_water": 8, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed_ops": 31322, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_executed": 15661, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:29.627 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 282867, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 68055818, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 282867, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:29.628 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 282867, 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 68055818, 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.629 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:29.630 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 54363, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 260250050, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 54363, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 260250050, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.631 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 1024, 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:29.632 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 0, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 0, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 54363, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 54363, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 54363, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 54363, 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:29.633 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:29.634 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 15661, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 1709969, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 15661, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:29.635 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 694, 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 1709969, 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.636 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-write_buf_throttle-0x55b09bdb06e0": { 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 8935, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 3758096384, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 79968, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 254826124, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:29.637 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 79968, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 20505, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 254817189, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:29.638 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout: "trackedop": { 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout: "slow_ops_count": 1603 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:29.639 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:30.356 INFO:tasks.check_counter:Daemon mds.g mds.exported=7 2024-08-20T01:27:30.356 INFO:tasks.check_counter:Daemon mds.g mds.imported=24 2024-08-20T01:27:30.356 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-08-20T01:27:30.356 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-08-20T01:27:30.356 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T01:27:30.356 DEBUG:tasks.check_counter:Getting stats from j 2024-08-20T01:27:30.357 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.j perf dump 2024-08-20T01:27:30.729 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:31.116 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:30 smithi138 ceph-mon[18076]: pgmap v970: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-20T01:27:31.117 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:30 smithi138 ceph-mon[18076]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:31.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:30 smithi167 ceph-mon[21965]: pgmap v970: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-20T01:27:31.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:30 smithi167 ceph-mon[21965]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:31.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:30 smithi142 ceph-mon[21799]: pgmap v970: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 95 op/s 2024-08-20T01:27:31.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:30 smithi142 ceph-mon[21799]: fsmap cephfs:3 {0=b=up:active,1=g=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-08-20T01:27:31.422 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-08-20T01:27:31.422 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:31.422 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 368, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 362, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 1013488, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 224256, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.079364786, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.043775258, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.043183056, 2024-08-20T01:27:31.423 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 361, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.026190855, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000072550 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:31.424 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 1013488, 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 224256 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 1, 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 276, 2024-08-20T01:27:31.425 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 160, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 128400, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 3, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.034469904, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.029151116, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.002253561, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:31.426 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 274, 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.025872786, 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000094426 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:31.427 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 128400 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_messages": 4, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages": 4, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_bytes": 2944, 2024-08-20T01:27:31.428 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_bytes": 448, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_total_time": 0.003795093, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_send_time": 0.000064140, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_recv_time": 0.003597706, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 3, 2024-08-20T01:27:31.429 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.006384630, 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.002128210 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.430 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "cct": { 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "total_workers": 0, 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "unhealthy_workers": 0 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "mempool": { 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:31.431 INFO:teuthology.orchestra.run.smithi138.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:31.432 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:31.433 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:31.434 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_items": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:31.435 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_bytes": 149998, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_anon_items": 30, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_bytes": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_items": 0, 2024-08-20T01:27:31.436 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_items": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:31.437 INFO:teuthology.orchestra.run.smithi138.stdout: "pgmap_items": 0, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_co_items": 2, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_1_items": 0, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "unittest_2_items": 0 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-08-20T01:27:31.438 INFO:teuthology.orchestra.run.smithi138.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "val": 0, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "max": 104857600, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "get_started": 0, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "get": 369, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "get_sum": 961441, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "get_or_fail_success": 369, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "take": 0, 2024-08-20T01:27:31.439 INFO:teuthology.orchestra.run.smithi138.stdout: "take_sum": 0, 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "put": 369, 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "put_sum": 961441, 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "wait": { 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgcount": 0, 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "sum": 0.000000000, 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: "avgtime": 0.000000000 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-08-20T01:27:31.440 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-08-20T01:27:32.255 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-08-20T01:27:32.256 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-08-20T01:27:32.256 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-08-20T01:27:32.256 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-08-20T01:27:32.256 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-08-20T01:27:32.256 DEBUG:tasks.check_counter:Getting stats from b 2024-08-20T01:27:32.256 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.b perf dump 2024-08-20T01:27:32.602 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:27:32.993 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:32 smithi142 ceph-mon[21799]: pgmap v971: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T01:27:33.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:32 smithi138 ceph-mon[18076]: pgmap v971: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T01:27:33.209 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 51687, 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 43415, 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 15623603, 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 104208655, 2024-08-20T01:27:33.210 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 14, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 4.485233495, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 1.991875255, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 1.560875162, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.054718985, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 43416, 2024-08-20T01:27:33.211 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 11.518355534, 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000265302 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 49209, 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.008229316, 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000167 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.212 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 598624, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 195984 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 95209, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 73797, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 34437120, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 82492539, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 239, 2024-08-20T01:27:33.213 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 6.317928088, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.137991640, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 3.277802721, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.528965551, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 73800, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 10.921697623, 2024-08-20T01:27:33.214 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000147990 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 126483, 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.021691070, 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000171 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1231024, 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 241152 2024-08-20T01:27:33.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 142427, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 205486, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 47322321, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 222884471, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 23, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 8, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 11.647083336, 2024-08-20T01:27:33.216 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 6.570280152, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 5.985271666, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.660754552, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 205478, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 31.717510010, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000154359 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:33.217 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 251144, 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.051545841, 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000205 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 48272, 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 693088 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-08-20T01:27:33.218 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 36360, 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 46.653132790, 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001283089 2024-08-20T01:27:33.219 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22013, 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.122897581, 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000141866 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 85151, 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 82538, 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 82538, 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 611.398378156, 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007407477 2024-08-20T01:27:33.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 2613, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 10, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 1658, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 6, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 4, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1442, 2024-08-20T01:27:33.222 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 721, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 583, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 138, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 411, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 849, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 276, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 276, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 20, 2024-08-20T01:27:33.223 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 120261, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 72546, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 2594, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 73, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 401, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 89074, 2024-08-20T01:27:33.224 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 22, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 15, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 653, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 15, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 834, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 658, 2024-08-20T01:27:33.225 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 211612228, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:33.226 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 5, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 6353, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 55605, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 51, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:33.227 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 31573, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 11368, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 2828, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 60175 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 27, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 71, 2024-08-20T01:27:33.228 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 43, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 128, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 160, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 88, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 11842, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 11754, 2024-08-20T01:27:33.229 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 17, 2024-08-20T01:27:33.230 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 1585, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 30, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 10, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.231 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 78643, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 78394, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 78394, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 249, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 1013, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 151, 2024-08-20T01:27:33.232 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 150, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 150, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 1, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 1, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 2, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 178852672, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 179917664, 2024-08-20T01:27:33.233 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 18900, 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 115.911949858, 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006132907 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.234 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1193, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 13679, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 12486, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 159, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 246, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 87, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1483, 2024-08-20T01:27:33.235 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 30032, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 28549, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 248, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 14599, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 14351, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 81732, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 313376768 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.236 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 111146, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 1481, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 85151, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 1477, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 4662, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11935, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 203.896718955, 2024-08-20T01:27:33.237 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.017083931 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 480, 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 31.684689881, 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.066009770 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:33.238 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17976, 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 81.257856201, 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004520352 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.240 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-08-20T01:27:33.241 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 19, 2024-08-20T01:27:33.243 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005748900, 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000302573 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 7, 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.011708875, 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001672696 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-08-20T01:27:33.244 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 19, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 21.063696174, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 1.108615588 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-08-20T01:27:33.245 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000162952, 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000162952 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 37736, 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 143.654139656, 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003806819 2024-08-20T01:27:33.246 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2159, 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 28.545185472, 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013221484 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.247 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.248 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 13, 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.112087959, 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008622150 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.249 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27, 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 50.401441578, 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 1.866720058 2024-08-20T01:27:33.250 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 316, 2024-08-20T01:27:33.251 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.415301508, 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001314245 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-08-20T01:27:33.252 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8, 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.020621003, 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002577625 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.253 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 11842, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 50.329019042, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004250043 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:33.254 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-08-20T01:27:33.255 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 4, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 2, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 13558, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 6779, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1123, 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:33.256 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 120, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 120, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:33.257 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:33.258 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:33.259 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-08-20T01:27:33.260 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 314358, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 806, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 88, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 1, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-08-20T01:27:33.261 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 193, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:33.262 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 4308477, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 46538, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:33.263 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 60798, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 191410913, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 34, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 60798, 2024-08-20T01:27:33.264 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 60798, 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 2384.895443177, 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.039226544 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 60798, 2024-08-20T01:27:33.265 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 86391 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 60798, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 488, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 60310, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1553, 2024-08-20T01:27:33.266 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 13721, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 478, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 19378, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 631, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 6, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 24127, 2024-08-20T01:27:33.267 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20753, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-08-20T01:27:33.268 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-08-20T01:27:33.269 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-08-20T01:27:33.270 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 115, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 36, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-08-20T01:27:33.271 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 4442, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 20, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 1282 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-08-20T01:27:33.272 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 16, 2024-08-20T01:27:33.273 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 8, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 23508, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 11754, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:33.274 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 289319, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 71248343, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 289319, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 289319, 2024-08-20T01:27:33.275 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 71248343, 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:33.276 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 60798, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 192695227, 2024-08-20T01:27:33.277 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 60798, 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 192695227, 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:33.278 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 60798, 2024-08-20T01:27:33.279 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 60798, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 60798, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 60798, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.280 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 11754, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1284474, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 11754, 2024-08-20T01:27:33.281 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 478, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1284474, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.282 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55a1b545a6e0": { 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 311400, 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:33.283 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 78643, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 175723360, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 78643, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 18900, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 175411960, 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:33.284 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 1607 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:33.285 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-08-20T01:27:33.286 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:32 smithi167 ceph-mon[21965]: pgmap v971: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.3 MiB/s wr, 96 op/s 2024-08-20T01:27:34.256 INFO:tasks.check_counter:Daemon mds.b mds.exported=15 2024-08-20T01:27:34.256 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2024-08-20T01:27:34.257 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-08-20T01:27:34.257 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=19 2024-08-20T01:27:34.257 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=13 2024-08-20T01:27:34.257 DEBUG:tasks.check_counter:Getting stats from e 2024-08-20T01:27:34.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.e perf dump 2024-08-20T01:27:34.519 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:27:34.962 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:34 smithi142 ceph-mon[21799]: pgmap v972: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 845 KiB/s wr, 49 op/s 2024-08-20T01:27:35.104 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-08-20T01:27:35.104 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1488, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1488, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 265530190, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 472423, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.262264744, 2024-08-20T01:27:35.105 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.059424042, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.159668430, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.032328920, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1487, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.051754814, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000034804 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.106 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:35.107 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 974, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1249, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 201592399, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 948686, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.221051568, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.048688104, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.140571449, 2024-08-20T01:27:35.108 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.022322808, 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1247, 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.045881516, 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000036793 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.109 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 639584 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 3137, 2024-08-20T01:27:35.110 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 3097, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 262048746, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 1088788, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.486746209, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.140418886, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.263781062, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.073843230, 2024-08-20T01:27:35.111 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3096, 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.117222418, 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000037862 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.112 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1200320, 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 229552 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.113 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8322, 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.211607076, 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000025427 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.114 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-08-20T01:27:35.115 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-08-20T01:27:35.116 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 23520, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 4251, 2024-08-20T01:27:35.117 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 19269, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 17447, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 22, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-08-20T01:27:35.118 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-08-20T01:27:35.119 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-08-20T01:27:35.120 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:35.121 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-08-20T01:27:35.122 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-08-20T01:27:35.123 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-08-20T01:27:35.124 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-08-20T01:27:35.125 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-08-20T01:27:35.126 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 122441, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-08-20T01:27:35.127 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 146, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 11, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 160301148, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 730704890, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 730693231, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.128 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 122441 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1450, 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 24519, 2024-08-20T01:27:35.129 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 23069, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 112, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 240, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 128, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 23528, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 25269, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 1741, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-08-20T01:27:35.130 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 80308, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 295878656 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-08-20T01:27:35.131 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.132 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.133 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.135 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:35.136 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.138 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-08-20T01:27:35.139 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.141 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:35.142 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.144 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.145 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.146 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.148 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.149 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-08-20T01:27:35.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 2498, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:35.151 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:35.152 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:35.153 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:35.154 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:35.155 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 308246, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 1900, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:35.156 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 193, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-08-20T01:27:35.157 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 20079444, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 148902, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-08-20T01:27:35.158 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 5158, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 5158, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5158, 2024-08-20T01:27:35.159 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 11.088791753, 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002149823 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5158, 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5159 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.160 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 5158, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 5158, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2975, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 2181, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-08-20T01:27:35.161 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:35.162 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-08-20T01:27:35.163 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-08-20T01:27:35.164 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-08-20T01:27:35.165 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 115, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 29, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 11, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 11, 2024-08-20T01:27:35.166 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-08-20T01:27:35.167 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-08-20T01:27:35.168 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 0, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:35.169 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 5595, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 728642648, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 5595, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5595, 2024-08-20T01:27:35.170 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 728642648, 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:35.171 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5158, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 726544229, 2024-08-20T01:27:35.172 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5158, 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 726544229, 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.173 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:35.174 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5158, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 5158, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5158, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 5158, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.175 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:35.176 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5645c7040960": { 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:35.178 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:35.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:35.180 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-08-20T01:27:35.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:34 smithi167 ceph-mon[21965]: pgmap v972: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 845 KiB/s wr, 49 op/s 2024-08-20T01:27:35.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:34 smithi138 ceph-mon[18076]: pgmap v972: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 845 KiB/s wr, 49 op/s 2024-08-20T01:27:36.039 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-08-20T01:27:36.039 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-08-20T01:27:36.039 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-08-20T01:27:36.039 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-08-20T01:27:36.039 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T01:27:36.039 DEBUG:tasks.check_counter:Getting stats from h 2024-08-20T01:27:36.040 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.h perf dump 2024-08-20T01:27:36.293 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:27:36.887 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-08-20T01:27:36.887 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 74923, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 96337, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 21154508, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 170217291, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 7.413426518, 2024-08-20T01:27:36.888 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 4.035921437, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 2.777078257, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.852214276, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 96336, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.517295049, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000036510 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:36.889 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 126332, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.021757451, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000172 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1214560, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 257456 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 245595, 2024-08-20T01:27:36.890 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 364534, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 78709571, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 587385031, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 17, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 19.425933230, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 10.834159431, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 11.467419438, 2024-08-20T01:27:36.891 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.488050991, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 364531, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 16.304576887, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000044727 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 426858, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.081896822, 2024-08-20T01:27:36.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000191 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 641472 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 101221, 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 112910, 2024-08-20T01:27:36.893 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 28381238, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 288817450, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 8.071925358, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.930027367, 2024-08-20T01:27:36.894 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 3.317258621, 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.175426828, 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 112910, 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.382130602, 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000038810 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:36.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 143611, 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.024238598, 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000168 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:36.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 85715, 2024-08-20T01:27:36.897 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 85.789884967, 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001000873 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-08-20T01:27:36.898 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 39942, 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 6.060375132, 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000151729 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 140310, 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 131871, 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-08-20T01:27:36.899 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 131871, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1046.985343680, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007939466 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 8439, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 7, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:36.900 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 119, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 11, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 7, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 2184, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 682, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1364, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 138, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1209, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 2330, 2024-08-20T01:27:36.901 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 341, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 341, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 24, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 192046, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 131464, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 8419, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 46, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 1, 2024-08-20T01:27:36.902 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 893, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 61661, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 31, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 1802, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 14, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 508, 2024-08-20T01:27:36.903 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 4, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 234, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:36.904 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 40, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 18281, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 93906, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:36.905 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 4, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 57181, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 21087, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 4920, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 77554 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-08-20T01:27:36.906 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 10, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 21, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 14, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 54, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 107, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 93, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:36.907 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 21967, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 21874, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-08-20T01:27:36.908 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_path": 17, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_quiesce_inode": 2061, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 33, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 18, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-08-20T01:27:36.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evlrg": 0, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 122445, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 45656, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 36871, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 85574, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 8785, 2024-08-20T01:27:36.910 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 200, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 72, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 54, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 146, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "segmjr": 7, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 18, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 160301148, 2024-08-20T01:27:36.911 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 730785858, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 46463, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 238.424227224, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005131485 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.912 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1446, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 24633, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 23187, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 171, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 299, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 128, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 2184, 2024-08-20T01:27:36.913 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 43455, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 41271, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 341, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 24216, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 23875, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 87168, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 300802048 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.914 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 187491, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 11959, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 140310, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 3948, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 24410, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22649, 2024-08-20T01:27:36.915 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 277.715086137, 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.012261693 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 13117, 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 89.029440439, 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006787332 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:36.916 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27072, 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 99.871684997, 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003689113 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.918 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-08-20T01:27:36.919 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.921 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:36.921 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.922 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 112, 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.625565929, 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.005585410 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.923 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.924 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 12, 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.006715165, 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000559597 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 40015, 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 247.697450050, 2024-08-20T01:27:36.925 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006190114 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5846, 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 19.557325481, 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003345420 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:36.926 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001216079, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001216079 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.928 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 76, 2024-08-20T01:27:36.929 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 60.054866035, 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.790195605 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:36.930 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1004, 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 1.498090959, 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001492122 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.932 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 21967, 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 250.927902409, 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.011422948 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:36.933 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-08-20T01:27:36.934 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 2, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 0, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 17750, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 8875, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1808, 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:36.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 240, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 240, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:36.936 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:36.937 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:36.938 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-08-20T01:27:36.939 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 315106, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 811, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 176, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 2, 2024-08-20T01:27:36.940 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 193, 2024-08-20T01:27:36.941 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 5779763, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 62102, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-08-20T01:27:36.942 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 93465, 2024-08-20T01:27:36.943 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 732632023, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 33, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 93465, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 93465, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 642.798959798, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006877429 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-08-20T01:27:36.944 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 93465, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 94640 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 93465, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 849, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 92616, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-08-20T01:27:36.945 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 1, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 35, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 842, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 47305, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 474, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-08-20T01:27:36.946 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 11, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 44013, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 70, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:36.947 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-08-20T01:27:36.948 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-08-20T01:27:36.949 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 115, 2024-08-20T01:27:36.950 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 29, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 1404, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 14, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 470 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.951 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:36.952 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 16, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 8, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed_ops": 43748, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 21874, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:36.953 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 421735, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 90261730, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 421735, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:36.954 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 421735, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 90261730, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.955 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:36.956 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 93465, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 735019865, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 93465, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 735019865, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.957 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-08-20T01:27:36.958 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 93465, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 93465, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 93465, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 93465, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.959 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 21874, 2024-08-20T01:27:36.960 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 2387954, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 21874, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 842, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2387954, 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:36.961 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x557ad43606e0": { 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 10547, 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-08-20T01:27:36.962 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 122445, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 726591554, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 122445, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 46463, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 726581007, 2024-08-20T01:27:36.963 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: "trackedop": { 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 1715 2024-08-20T01:27:36.964 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:36.965 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-08-20T01:27:36.965 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:36 smithi142 ceph-mon[21799]: pgmap v973: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 585 KiB/s wr, 30 op/s 2024-08-20T01:27:37.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:36 smithi167 ceph-mon[21965]: pgmap v973: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 585 KiB/s wr, 30 op/s 2024-08-20T01:27:37.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:36 smithi138 ceph-mon[18076]: pgmap v973: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 585 KiB/s wr, 30 op/s 2024-08-20T01:27:37.752 INFO:tasks.check_counter:Daemon mds.h mds.exported=31 2024-08-20T01:27:37.752 INFO:tasks.check_counter:Daemon mds.h mds.imported=14 2024-08-20T01:27:37.752 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=0 2024-08-20T01:27:37.753 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-08-20T01:27:37.753 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T01:27:37.753 DEBUG:tasks.check_counter:Getting stats from k 2024-08-20T01:27:37.753 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.k perf dump 2024-08-20T01:27:38.018 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.b/config 2024-08-20T01:27:38.548 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 4, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 4, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 2944, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 448, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:38.549 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.002948834, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.000045920, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.002161435, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.004028394, 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001342798 2024-08-20T01:27:38.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T01:27:38.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 280, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 320, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 209280, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.020498109, 2024-08-20T01:27:38.552 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.016724433, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.001001461, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 278, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.016268316, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000058519 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.553 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 209280 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.554 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 371, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 365, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 1014304, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 226176, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.058034358, 2024-08-20T01:27:38.555 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.029004752, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.037997212, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 364, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.018312519, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000050309 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.556 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 1014304, 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 226176 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-08-20T01:27:38.557 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 0, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:38.558 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:38.559 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:38.560 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:38.561 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:38.562 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 148721, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 30, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-08-20T01:27:38.563 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:38.564 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 2, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:38.565 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 373, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 961858, 2024-08-20T01:27:38.566 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 373, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 373, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 961858, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-08-20T01:27:38.567 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-08-20T01:27:38.568 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-08-20T01:27:38.568 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:38.568 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-08-20T01:27:38.568 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-08-20T01:27:39.211 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:38 smithi142 ceph-mon[21799]: pgmap v974: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 887 KiB/s wr, 43 op/s 2024-08-20T01:27:39.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:38 smithi167 ceph-mon[21965]: pgmap v974: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 887 KiB/s wr, 43 op/s 2024-08-20T01:27:39.239 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-08-20T01:27:39.239 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-08-20T01:27:39.240 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-08-20T01:27:39.240 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-08-20T01:27:39.240 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-08-20T01:27:39.240 DEBUG:tasks.check_counter:Getting stats from c 2024-08-20T01:27:39.240 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.c perf dump 2024-08-20T01:27:39.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:38 smithi138 ceph-mon[18076]: pgmap v974: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 887 KiB/s wr, 43 op/s 2024-08-20T01:27:39.501 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:27:40.058 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-20T01:27:40.058 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:40.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2853, 2024-08-20T01:27:40.058 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2855, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 93079172, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 907261, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.342247926, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.131662881, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.109619355, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.078766304, 2024-08-20T01:27:40.059 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2853, 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.109615883, 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000038421 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.060 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 885, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1158, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 71555021, 2024-08-20T01:27:40.061 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 831822, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 7, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 6, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.146346599, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.048694055, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.066214625, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.022072666, 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:40.062 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1157, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.049854409, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000043089 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.063 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 551920 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1292, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1252, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 93697845, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 502078, 2024-08-20T01:27:40.064 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.195816246, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.063735978, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.114418328, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.021090696, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1251, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.051468940, 2024-08-20T01:27:40.065 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000041142 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.066 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1200320, 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 229552 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-20T01:27:40.067 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 7664, 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.182528714, 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000023816 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:40.068 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-20T01:27:40.069 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-20T01:27:40.070 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 17827, 2024-08-20T01:27:40.071 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 406, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 17421, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 17248, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 19, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-08-20T01:27:40.072 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-20T01:27:40.073 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:40.074 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:40.075 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T01:27:40.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-20T01:27:40.077 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-08-20T01:27:40.078 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-20T01:27:40.079 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-20T01:27:40.080 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 80143, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-20T01:27:40.081 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 111, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 8, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 4194304, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 259998493, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 259993288, 2024-08-20T01:27:40.082 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 80143 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-20T01:27:40.083 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1810, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 17778, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 15968, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 95, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 114, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 19, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 17827, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 18153, 2024-08-20T01:27:40.084 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 326, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 74208, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 275062784 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-08-20T01:27:40.085 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.086 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.087 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-20T01:27:40.088 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.089 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.090 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.091 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.092 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.093 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.094 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-20T01:27:40.095 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.096 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.097 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-20T01:27:40.098 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.099 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.100 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.101 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.102 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-20T01:27:40.103 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.104 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.105 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.106 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-20T01:27:40.106 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 2, 2024-08-20T01:27:40.106 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 3, 2024-08-20T01:27:40.106 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 1, 2024-08-20T01:27:40.106 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 2, 2024-08-20T01:27:40.107 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-20T01:27:40.107 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-08-20T01:27:40.107 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-08-20T01:27:40.107 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2487, 2024-08-20T01:27:40.107 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:40.108 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:40.109 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:40.110 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:40.111 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:40.112 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 315857, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 2086, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:40.113 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 193, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:40.114 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 16789018, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 121466, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-20T01:27:40.115 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 4590, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 21, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 3, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 4590, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4590, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 8.876843045, 2024-08-20T01:27:40.116 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001933952 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4590, 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 4591 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.117 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 4590, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 4590, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 2647, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-08-20T01:27:40.118 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1941, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-08-20T01:27:40.119 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 1, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:40.120 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-20T01:27:40.121 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-20T01:27:40.122 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 115, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-20T01:27:40.123 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 29, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 11, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 12, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 1, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-08-20T01:27:40.124 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.125 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:40.126 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 5026, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 257854604, 2024-08-20T01:27:40.127 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 5026, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 5026, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 257854604, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.128 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:40.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4590, 2024-08-20T01:27:40.130 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 255856552, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4590, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 255856552, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.131 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:40.132 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 4590, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 4590, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 4590, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 4590, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.133 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:40.134 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:40.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x556ee618e960": { 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:40.136 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:40.137 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:40.138 INFO:teuthology.orchestra.run.smithi167.stdout: "trackedop": { 2024-08-20T01:27:40.139 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-20T01:27:40.139 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:40.139 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-20T01:27:40.901 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-08-20T01:27:40.901 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-08-20T01:27:40.901 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-08-20T01:27:40.901 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-08-20T01:27:40.901 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T01:27:40.902 DEBUG:tasks.check_counter:Getting stats from f 2024-08-20T01:27:40.902 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.f perf dump 2024-08-20T01:27:40.924 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:40 smithi167 ceph-mon[21965]: pgmap v975: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 876 KiB/s wr, 37 op/s 2024-08-20T01:27:41.156 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:27:41.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:40 smithi142 ceph-mon[21799]: pgmap v975: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 876 KiB/s wr, 37 op/s 2024-08-20T01:27:41.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:40 smithi138 ceph-mon[18076]: pgmap v975: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 876 KiB/s wr, 37 op/s 2024-08-20T01:27:41.693 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-20T01:27:41.693 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:41.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 4, 2024-08-20T01:27:41.693 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 4, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 2944, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 448, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.003264945, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000039627, 2024-08-20T01:27:41.694 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.002405240, 2024-08-20T01:27:41.695 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 3, 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.004365167, 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.001455055 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.696 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 448 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.697 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 278, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 320, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 208928, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.020180540, 2024-08-20T01:27:41.698 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.016353655, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000901823, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 276, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.017044783, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000061756 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:41.699 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 208928 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 368, 2024-08-20T01:27:41.700 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 362, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1013488, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 224256, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.058474562, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.029089506, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.037600649, 2024-08-20T01:27:41.701 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 361, 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.018003553, 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000049871 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:41.702 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1013488, 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 224256 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-20T01:27:41.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:41.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:41.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:41.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:41.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148721, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 30, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-20T01:27:41.708 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:41.709 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-20T01:27:41.710 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 370, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 961468, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 370, 2024-08-20T01:27:41.711 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 370, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 961468, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:41.712 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:41.713 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:41.713 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:41.713 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-20T01:27:42.375 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-08-20T01:27:42.375 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-08-20T01:27:42.375 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-08-20T01:27:42.375 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-08-20T01:27:42.375 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-08-20T01:27:42.375 DEBUG:tasks.check_counter:Getting stats from i 2024-08-20T01:27:42.376 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.i perf dump 2024-08-20T01:27:42.734 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:27:43.196 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:42 smithi138 ceph-mon[18076]: pgmap v976: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-08-20T01:27:43.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:42 smithi167 ceph-mon[21965]: pgmap v976: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-08-20T01:27:43.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:42 smithi142 ceph-mon[21799]: pgmap v976: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 1.5 MiB/s wr, 58 op/s 2024-08-20T01:27:43.484 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-20T01:27:43.484 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:43.484 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1513, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1515, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 67499470, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 481144, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 8, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 4, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.194093488, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.067757355, 2024-08-20T01:27:43.485 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.071899016, 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.041483750, 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1513, 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.059898729, 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000039589 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.486 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:43.487 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 1310, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 1286, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 55145071, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 465003, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.152707118, 2024-08-20T01:27:43.488 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.056735678, 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.070329534, 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.022554792, 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 1286, 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.057833792, 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000044971 2024-08-20T01:27:43.489 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 345312, 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 132000 2024-08-20T01:27:43.490 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 286, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 441, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 37729170, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 454544, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 6, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 5, 2024-08-20T01:27:43.491 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.058186168, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.020103978, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.027314312, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.007165249, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 439, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.020964824, 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000047755 2024-08-20T01:27:43.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 160, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 363904 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.493 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 1, 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-MDSRank": { 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 4756, 2024-08-20T01:27:43.494 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.216285138, 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000045476 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "finisher-PurgeQueue": { 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "queue_len": 0, 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "complete_latency": { 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.495 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "mds": { 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "request": 0, 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "reply": 0, 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "reply_latency": { 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.496 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_reply": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "forward": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_complete": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_fetch_keys": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_commit": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_split": 0, 2024-08-20T01:27:43.497 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_merge": 0, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes": 12068, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_top": 8262, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_bottom": 3806, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pin_tail": 0, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_pinned": 271, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_expired": 0, 2024-08-20T01:27:43.498 INFO:teuthology.orchestra.run.smithi167.stdout: "inodes_with_caps": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "subtrees": 9, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_hit": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_forward": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_discover": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_dir_fetch": 0, 2024-08-20T01:27:43.499 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_remote_ino": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "traverse_lock": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "load_cent": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "q": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "exported": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "exported_inodes": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "imported": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "imported_inodes": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_dir_fetch": 0, 2024-08-20T01:27:43.500 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_backtrace_fetch": 0, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "openino_peer_discover": 0, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rfiles": 658, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rbytes": 211612228, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "root_rsnaps": 6, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_fetch": 0, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_set_tag": 0, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_backtrace_repaired": 0, 2024-08-20T01:27:43.501 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_inotable_repaired": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_inodes": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dir_base_inodes": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_dirfrag_rstats": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "scrub_file_inodes": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_inode_file_caps": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_revoke": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_grant": 0, 2024-08-20T01:27:43.502 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_trunc": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "ceph_cap_op_flush_ack": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_caps_dirty": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_cap_release": 0, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "process_request_cap_release": 0 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.503 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_cache": { 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_update_receipt": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_try_discover": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_send_discover": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "dir_handle_discover": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_delayed": 0, 2024-08-20T01:27:43.504 INFO:teuthology.orchestra.run.smithi167.stdout: "num_strays_enqueuing": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_created": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_enqueued": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_reintegrated": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "strays_migrated": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_processing": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_enqueued": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "num_recovering_prioritized": 0, 2024-08-20T01:27:43.505 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_started": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "recovery_completed": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_path": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_quiesce_inode": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_enqueue_scrub": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_exportdir": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_flush": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragmentdir": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_fragstats": 0, 2024-08-20T01:27:43.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ireq_inodestats": 0 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_log": { 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "evlrg": 0, 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "evadd": 0, 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "evex": 0, 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "evtrm": 0, 2024-08-20T01:27:43.507 INFO:teuthology.orchestra.run.smithi167.stdout: "ev": 66676, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "evexg": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "evexd": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segadd": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segex": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segtrm": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "seg": 2, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segmjr": 56, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segexg": 0, 2024-08-20T01:27:43.508 INFO:teuthology.orchestra.run.smithi167.stdout: "segexd": 0, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "expos": 178852672, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "wrpos": 180261377, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "rdpos": 180253993, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "jlat": { 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.509 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "replayed": 66676 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_mem": { 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "ino": 1117, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "ino+": 12381, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "ino-": 11264, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "dir": 76, 2024-08-20T01:27:43.510 INFO:teuthology.orchestra.run.smithi167.stdout: "dir+": 114, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dir-": 38, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dn": 12071, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dn+": 12606, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "dn-": 535, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "cap": 0, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "cap+": 0, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "cap-": 0, 2024-08-20T01:27:43.511 INFO:teuthology.orchestra.run.smithi167.stdout: "rss": 63984, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "heap": 270209024 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_server": { 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_client_request": 0, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "dispatch_server_request": 0, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_request": 0, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_client_session": 0, 2024-08-20T01:27:43.512 INFO:teuthology.orchestra.run.smithi167.stdout: "handle_peer_request": 0, 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "req_create_latency": { 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getattr_latency": { 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.513 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getfilelock_latency": { 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.514 INFO:teuthology.orchestra.run.smithi167.stdout: "req_link_latency": { 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookup_latency": { 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.515 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookuphash_latency": { 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupino_latency": { 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.516 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupname_latency": { 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.517 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupparent_latency": { 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lookupsnap_latency": { 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.518 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: "req_lssnap_latency": { 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.519 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.520 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mkdir_latency": { 2024-08-20T01:27:43.520 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.520 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mknod_latency": { 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.521 INFO:teuthology.orchestra.run.smithi167.stdout: "req_mksnap_latency": { 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "req_open_latency": { 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.522 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: "req_readdir_latency": { 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.523 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rename_latency": { 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "req_renamesnap_latency": { 2024-08-20T01:27:43.524 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "req_snapdiff_latency": { 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.525 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmdir_latency": { 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmsnap_latency": { 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.526 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "req_rmxattr_latency": { 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.527 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setattr_latency": { 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setdirlayout_latency": { 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.528 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setfilelock_latency": { 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setlayout_latency": { 2024-08-20T01:27:43.529 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "req_setxattr_latency": { 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.530 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "req_symlink_latency": { 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "req_unlink_latency": { 2024-08-20T01:27:43.531 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_revoke_eviction": 0, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "cap_acquisition_throttle": 0, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "req_getvxattr_latency": { 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.532 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_sessions": { 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "session_count": 2, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "session_add": 2, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "session_remove": 0, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_open": 0, 2024-08-20T01:27:43.533 INFO:teuthology.orchestra.run.smithi167.stdout: "sessions_stale": 0, 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "total_load": 0, 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "average_load": 0, 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "avg_session_uptime": 2491, 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "mdthresh_evicted": 0 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-20T01:27:43.534 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:43.535 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:43.536 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:43.537 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:43.538 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 286946, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 1343, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-20T01:27:43.539 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 9544, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 193, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:43.540 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 11087408, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 81281, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:43.541 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "objecter": { 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_active": 0, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_laggy": 0, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send": 2845, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_send_bytes": 26, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_resend": 4, 2024-08-20T01:27:43.542 INFO:teuthology.orchestra.run.smithi167.stdout: "op_reply": 2845, 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "op_latency": { 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2845, 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 5.959894638, 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.002094866 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "op_inflight": 0, 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "oplen_avg": { 2024-08-20T01:27:43.543 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 2845, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 2846 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "op": 2845, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "op_r": 2845, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "op_w": 0, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "op_rmw": 0, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "op_pg": 0, 2024-08-20T01:27:43.544 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_stat": 1584, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_create": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_read": 1258, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_write": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writefull": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_writesame": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_append": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_zero": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_truncate": 0, 2024-08-20T01:27:43.545 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_delete": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_mapext": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_sparse_read": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_clonerange": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_getxattr": 2, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_setxattr": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_cmpxattr": 0, 2024-08-20T01:27:43.546 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_rmxattr": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_resetxattrs": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_call": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_watch": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_notify": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_src_cmpxattr": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_pgls_filter": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "osdop_other": 0, 2024-08-20T01:27:43.547 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_active": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_send": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_resend": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "linger_ping": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_active": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_send": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "poolop_resend": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_active": 0, 2024-08-20T01:27:43.548 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_send": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "poolstat_resend": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_active": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_send": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "statfs_resend": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "command_active": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "command_send": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "command_resend": 0, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "map_epoch": 115, 2024-08-20T01:27:43.549 INFO:teuthology.orchestra.run.smithi167.stdout: "map_full": 0, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "map_inc": 26, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_sessions": 11, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_open": 11, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_session_close": 0, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_laggy": 0, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_wr": 0, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_rd": 2, 2024-08-20T01:27:43.550 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_del": 0 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "oft": { 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_objs": 0, 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_kv_pairs": 0, 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_updates": 0, 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "omap_total_removes": 0 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "purge_queue": { 2024-08-20T01:27:43.551 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_ops_high_water": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executing_high_water": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed_ops": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_executed": 0, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "pq_item_in_journal": 0 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.552 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 3105, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 160078156, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 3105, 2024-08-20T01:27:43.553 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 3105, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 160078156, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.554 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_bytes": { 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:43.555 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2845, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 159238775, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2845, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 159238775, 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:43.556 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-objecter_ops": { 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 1024, 2024-08-20T01:27:43.557 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 2845, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 2845, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 2845, 2024-08-20T01:27:43.558 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 2845, 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle": { 2024-08-20T01:27:43.559 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:43.560 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.561 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-write_buf_throttle-0x563ffdf7e3c0": { 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 3758096384, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 0, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 0, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:43.562 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:43.563 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout: "trackedop": { 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout: "slow_ops_count": 0 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:43.564 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-20T01:27:44.339 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-08-20T01:27:44.339 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-08-20T01:27:44.339 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-08-20T01:27:44.339 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-08-20T01:27:44.339 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-08-20T01:27:44.339 DEBUG:tasks.check_counter:Getting stats from l 2024-08-20T01:27:44.339 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph daemon mds.l perf dump 2024-08-20T01:27:44.586 INFO:teuthology.orchestra.run.smithi167.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.c/config 2024-08-20T01:27:44.965 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:44 smithi167 ceph-mon[21965]: pgmap v977: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 62 op/s 2024-08-20T01:27:45.132 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-0": { 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 377, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 369, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 1017792, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 227904, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 0, 2024-08-20T01:27:45.133 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.060083916, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.029640087, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.037229623, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 369, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.023349588, 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000063278 2024-08-20T01:27:45.134 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 1017792, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 227904 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.135 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-1": { 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 2, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 282, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 320, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 209632, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 4, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 2, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.020871104, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.016817348, 2024-08-20T01:27:45.136 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.001025648, 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 280, 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.017292632, 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000061759 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:45.137 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 320, 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 209632 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "AsyncMessenger::Worker-2": { 2024-08-20T01:27:45.138 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_messages": 0, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages": 2, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_bytes": 0, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_bytes": 0, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_created_connections": 2, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_active_connections": 1, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_total_time": 0.000292931, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_send_time": 0.000000000, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_recv_time": 0.000209732, 2024-08-20T01:27:45.139 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_messages_queue_lat": { 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_handle_ack_lat": { 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:45.140 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_recv_encrypted_bytes": 0, 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "msgr_send_encrypted_bytes": 0 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "cct": { 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "total_workers": 0, 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: "unhealthy_workers": 0 2024-08-20T01:27:45.141 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "mempool": { 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_bytes": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bloom_filter_items": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_bytes": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_alloc_items": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_bytes": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_data_items": 0, 2024-08-20T01:27:45.142 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_bytes": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_onode_items": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_bytes": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_meta_items": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_bytes": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_other_items": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_bytes": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_cache_buffer_items": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_bytes": 0, 2024-08-20T01:27:45.143 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_extent_items": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_bytes": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_blob_items": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_bytes": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_shared_blob_items": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_bytes": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_inline_bl_items": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_bytes": 0, 2024-08-20T01:27:45.144 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_fsck_items": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_bytes": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_txc_items": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_bytes": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_deferred_items": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_bytes": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluestore_writing_items": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_bytes": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_items": 0, 2024-08-20T01:27:45.145 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_bytes": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_reader_items": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_bytes": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "bluefs_file_writer_items": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_bytes": 148744, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_anon_items": 30, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_bytes": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "buffer_meta_items": 0, 2024-08-20T01:27:45.146 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_bytes": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_items": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_bytes": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_mapbl_items": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_bytes": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osd_pglog_items": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_bytes": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_items": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_bytes": 0, 2024-08-20T01:27:45.147 INFO:teuthology.orchestra.run.smithi167.stdout: "osdmap_mapping_items": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_bytes": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "pgmap_items": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_bytes": 928, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "mds_co_items": 2, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_bytes": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_1_items": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_bytes": 0, 2024-08-20T01:27:45.148 INFO:teuthology.orchestra.run.smithi167.stdout: "unittest_2_items": 0 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "val": 0, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "max": 104857600, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "get_started": 0, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "get": 375, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "get_sum": 962118, 2024-08-20T01:27:45.149 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_fail": 0, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "get_or_fail_success": 375, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "take": 0, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "take_sum": 0, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "put": 375, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "put_sum": 962118, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "wait": { 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "avgcount": 0, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "sum": 0.000000000, 2024-08-20T01:27:45.150 INFO:teuthology.orchestra.run.smithi167.stdout: "avgtime": 0.000000000 2024-08-20T01:27:45.151 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:45.151 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-08-20T01:27:45.151 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-08-20T01:27:45.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:44 smithi142 ceph-mon[21799]: pgmap v977: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 62 op/s 2024-08-20T01:27:45.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:44 smithi138 ceph-mon[18076]: pgmap v977: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.6 MiB/s wr, 62 op/s 2024-08-20T01:27:46.229 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-08-20T01:27:46.229 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-08-20T01:27:46.229 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-08-20T01:27:46.230 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-08-20T01:27:46.230 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-08-20T01:27:46.230 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-08-20T01:27:46.238 INFO:tasks.kclient:Unmounting kernel clients... 2024-08-20T01:27:46.238 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:27:46.238 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T01:27:46.280 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-08-20T01:27:46.281 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/proc/self/mounts of=/dev/stdout 2024-08-20T01:27:46.360 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd blocklist ls 2024-08-20T01:27:46.806 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:47.153 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:46 smithi138 ceph-mon[18076]: pgmap v978: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.7 MiB/s wr, 69 op/s 2024-08-20T01:27:47.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:46 smithi167 ceph-mon[21965]: pgmap v978: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.7 MiB/s wr, 69 op/s 2024-08-20T01:27:47.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:46 smithi142 ceph-mon[21799]: pgmap v978: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.7 MiB/s wr, 69 op/s 2024-08-20T01:27:47.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.433+0000 7f2d4d400640 1 -- 172.21.15.138:0/3015993724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 msgr2=0x7f2d4810e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:47.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.433+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3015993724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4810e9d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2d34009900 tx=0x7f2d3402a410 comp rx=0 tx=0).stop 2024-08-20T01:27:47.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 -- 172.21.15.138:0/3015993724 shutdown_connections 2024-08-20T01:27:47.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3015993724 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4810ef10 0x7f2d48111300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.464 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3015993724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4810e9d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3015993724 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d48075210 0x7f2d480755f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 -- 172.21.15.138:0/3015993724 >> 172.21.15.138:0/3015993724 conn(0x7f2d480fe350 msgr2=0x7f2d48100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 -- 172.21.15.138:0/3015993724 shutdown_connections 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.434+0000 7f2d4d400640 1 -- 172.21.15.138:0/3015993724 wait complete. 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 Processor -- start 2024-08-20T01:27:47.465 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 -- start start 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 0x7f2d4819af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d4810ef10 0x7f2d481a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48114b40 con 0x7f2d48075210 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2d481149c0 con 0x7f2d48075b30 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d4d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2d48114cc0 con 0x7f2d4810ef10 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.435+0000 7f2d47400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:47.466 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 0x7f2d4819af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:48540/0 (socket says 172.21.15.138:48540) 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47400640 1 -- 172.21.15.138:0/3160092978 learned_addr learned my addr 172.21.15.138:0/3160092978 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d4ca00640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d4810ef10 0x7f2d481a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 -- 172.21.15.138:0/3160092978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d4810ef10 msgr2=0x7f2d481a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d4810ef10 0x7f2d481a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 -- 172.21.15.138:0/3160092978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 msgr2=0x7f2d4819b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.467 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 -- 172.21.15.138:0/3160092978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d481a1020 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.436+0000 7f2d47e00640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 0x7f2d4819af70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.437+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3800ccc0 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.437+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d38005020 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.437+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d38004a00 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.437+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d481a1310 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.437+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d481a79d0 con 0x7f2d48075210 2024-08-20T01:27:47.468 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.438+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d480755f0 con 0x7f2d48075210 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.440+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2d38004ba0 con 0x7f2d48075210 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.442+0000 7f2d44c00640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 0x7f2d2c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.442+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f2d380e43f0 con 0x7f2d48075210 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.442+0000 7f2d47400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 0x7f2d2c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.443+0000 7f2d47400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 0x7f2d2c0b62a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d481104f0 tx=0x7f2d34005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:47.469 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.445+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2d380ab300 con 0x7f2d48075210 2024-08-20T01:27:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.649+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d481a7c90 con 0x7f2d48075210 2024-08-20T01:27:47.650 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.650+0000 7f2d44c00640 1 -- 172.21.15.138:0/3160092978 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v115) ==== 81+0+897 (secure 0 0 0) 0x7f2d380aac00 con 0x7f2d48075210 2024-08-20T01:27:47.651 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:27:47.651 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:27:47.651 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2534070123 2024-08-21T00:57:02.889250+0000 2024-08-20T01:27:47.651 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/256603896 2024-08-21T00:58:34.377803+0000 2024-08-20T01:27:47.651 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3607751007 2024-08-21T00:58:34.377803+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3948920804 2024-08-21T00:58:34.377803+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/200546042 2024-08-21T00:57:02.889250+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6800/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:6801/3853414921 2024-08-21T00:57:30.262737+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/64005733 2024-08-21T00:57:30.262737+0000 2024-08-20T01:27:47.652 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/1354849701 2024-08-21T00:57:02.889250+0000 2024-08-20T01:27:47.653 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/2736003984 2024-08-21T00:57:30.262737+0000 2024-08-20T01:27:47.653 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3346890941 2024-08-21T00:58:34.377803+0000 2024-08-20T01:27:47.653 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/4262320942 2024-08-21T00:57:02.889250+0000 2024-08-20T01:27:47.653 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.15.138:0/3120862368 2024-08-21T00:57:30.262737+0000 2024-08-20T01:27:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:listed 15 entries 2024-08-20T01:27:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.652+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 msgr2=0x7f2d2c0b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.652+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 0x7f2d2c0b62a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2d481104f0 tx=0x7f2d34005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.652+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 msgr2=0x7f2d4819af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:47.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.652+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 0x7f2d4819af70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800e9f0 tx=0x7f2d3800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 shutdown_connections 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f2d2c0b3de0 0x7f2d2c0b62a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2d4810ef10 0x7f2d481a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2d48075b30 0x7f2d4819b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 --2- 172.21.15.138:0/3160092978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d48075210 0x7f2d4819af70 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 >> 172.21.15.138:0/3160092978 conn(0x7f2d480fe350 msgr2=0x7f2d48111100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:47.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 shutdown_connections 2024-08-20T01:27:47.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:47.653+0000 7f2d4d400640 1 -- 172.21.15.138:0/3160092978 wait complete. 2024-08-20T01:27:48.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:47 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3160092978' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:27:48.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:47 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3160092978' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:27:48.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:47 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3160092978' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-08-20T01:27:48.614 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-08-20T01:27:48.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:27:48.614 DEBUG:teuthology.orchestra.run.smithi138:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-08-20T01:27:48.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:27:48.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:27:48.859 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-08-20T01:27:48.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:27:48.932 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-08-20T01:27:48.975 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-08-20T01:27:48.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:27:48.976 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-08-20T01:27:49.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-08-20T01:27:49.074 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-08-20T01:27:49.214 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:48 smithi167 ceph-mon[21965]: pgmap v979: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-08-20T01:27:49.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:48 smithi142 ceph-mon[21799]: pgmap v979: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-08-20T01:27:49.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:48 smithi138 ceph-mon[18076]: pgmap v979: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.6 MiB/s wr, 78 op/s 2024-08-20T01:27:49.392 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-08-20T01:27:49.403 INFO:tasks.cephadm:Teardown begin 2024-08-20T01:27:49.403 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-08-20T01:27:49.403 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:27:49.776 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:50.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 -- 172.21.15.138:0/51386061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 msgr2=0x7f5da8104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:50.403 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 --2- 172.21.15.138:0/51386061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da8104ef0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009ca0 tx=0x7f5d9802a8c0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 -- 172.21.15.138:0/51386061 shutdown_connections 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 --2- 172.21.15.138:0/51386061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 0x7f5da81116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 --2- 172.21.15.138:0/51386061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da810a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 --2- 172.21.15.138:0/51386061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da8104ef0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 -- 172.21.15.138:0/51386061 >> 172.21.15.138:0/51386061 conn(0x7f5da8100540 msgr2=0x7f5da8102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 -- 172.21.15.138:0/51386061 shutdown_connections 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.402+0000 7f5dada00640 1 -- 172.21.15.138:0/51386061 wait complete. 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 Processor -- start 2024-08-20T01:27:50.404 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 -- start start 2024-08-20T01:27:50.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:50.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da819b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:50.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 0x7f5da81a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:50.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114bb0 con 0x7f5da8104b10 2024-08-20T01:27:50.406 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114a30 con 0x7f5da810a520 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.403+0000 7f5dada00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f5da8114d30 con 0x7f5da810aea0 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da819b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38934/0 (socket says 172.21.15.138:38934) 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7e00640 1 -- 172.21.15.138:0/3006932418 learned_addr learned my addr 172.21.15.138:0/3006932418 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5dad000640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 0x7f5da81a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 -- 172.21.15.138:0/3006932418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 msgr2=0x7f5da81a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 0x7f5da81a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 -- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 msgr2=0x7f5da819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 -- 172.21.15.138:0/3006932418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da81a0fd0 con 0x7f5da810a520 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.404+0000 7f5da7400640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da819b460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.405+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400ccc0 con 0x7f5da810a520 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.405+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da81a12c0 con 0x7f5da810a520 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.405+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d940108e0 con 0x7f5da810a520 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.405+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94018880 con 0x7f5da810a520 2024-08-20T01:27:50.408 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.405+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da81a7980 con 0x7f5da810a520 2024-08-20T01:27:50.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.406+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da8104ef0 con 0x7f5da810a520 2024-08-20T01:27:50.409 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.409+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f5d94010430 con 0x7f5da810a520 2024-08-20T01:27:50.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.410+0000 7f5da4c00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 0x7f5d8c0b62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:50.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.410+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f5d940e4b80 con 0x7f5da810a520 2024-08-20T01:27:50.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.410+0000 7f5da7e00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 0x7f5d8c0b62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:50.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.411+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f5d940ab440 con 0x7f5da810a520 2024-08-20T01:27:50.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.411+0000 7f5da7e00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 0x7f5d8c0b62a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009a60 tx=0x7f5d98005d80 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:50.633 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.632+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5da819c050 con 0x7f5da810a520 2024-08-20T01:27:50.634 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.633+0000 7f5da4c00640 1 -- 172.21.15.138:0/3006932418 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11938 (secure 0 0 0) 0x7f5d940af990 con 0x7f5da810a520 2024-08-20T01:27:50.634 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:50.634 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":57,"btime":"2024-08-20T01:27:29:858179+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:27:29.262392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:27:50.635 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 57 2024-08-20T01:27:50.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 msgr2=0x7f5d8c0b62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 0x7f5d8c0b62a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5d98009a60 tx=0x7f5d98005d80 comp rx=0 tx=0).stop 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 msgr2=0x7f5da819b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da819b460 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5d9400e9f0 tx=0x7f5d9400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 shutdown_connections 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f5d8c0b3de0 0x7f5d8c0b62a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f5da810aea0 0x7f5da81a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5da810a520 0x7f5da819b460 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.637 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 --2- 172.21.15.138:0/3006932418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5da8104b10 0x7f5da819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:50.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 >> 172.21.15.138:0/3006932418 conn(0x7f5da8100540 msgr2=0x7f5da8103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:50.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 shutdown_connections 2024-08-20T01:27:50.638 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:50.636+0000 7f5dada00640 1 -- 172.21.15.138:0/3006932418 wait complete. 2024-08-20T01:27:51.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:50 smithi167 ceph-mon[21965]: pgmap v980: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-08-20T01:27:51.017 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:50 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3006932418' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:51.229 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:50 smithi138 ceph-mon[18076]: pgmap v980: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-08-20T01:27:51.230 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:50 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3006932418' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:51.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:50 smithi142 ceph-mon[21799]: pgmap v980: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.3 MiB/s wr, 60 op/s 2024-08-20T01:27:51.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:50 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3006932418' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:51.609 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs dump --format=json 2024-08-20T01:27:51.982 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 -- 172.21.15.138:0/3122496337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c104b10 msgr2=0x7f888c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 --2- 172.21.15.138:0/3122496337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c104b10 0x7f888c104ef0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f887c009900 tx=0x7f887c02a3f0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 -- 172.21.15.138:0/3122496337 shutdown_connections 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 --2- 172.21.15.138:0/3122496337 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10aea0 0x7f888c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 --2- 172.21.15.138:0/3122496337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c10a520 0x7f888c10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 --2- 172.21.15.138:0/3122496337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c104b10 0x7f888c104ef0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.832 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 -- 172.21.15.138:0/3122496337 >> 172.21.15.138:0/3122496337 conn(0x7f888c100540 msgr2=0x7f888c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:52.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 -- 172.21.15.138:0/3122496337 shutdown_connections 2024-08-20T01:27:52.833 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.831+0000 7f8891400640 1 -- 172.21.15.138:0/3122496337 wait complete. 2024-08-20T01:27:52.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.832+0000 7f8891400640 1 Processor -- start 2024-08-20T01:27:52.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.832+0000 7f8891400640 1 -- start start 2024-08-20T01:27:52.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c104b10 0x7f888c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:52.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:52.834 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c10aea0 0x7f888c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f888c114bb0 con 0x7f888c10aea0 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f888c114a30 con 0x7f888c104b10 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8891400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f888c114d30 con 0x7f888c10a520 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f888b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f888b400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59940/0 (socket says 172.21.15.138:59940) 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f8890a00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c10aea0 0x7f888c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f888b400640 1 -- 172.21.15.138:0/3708389612 learned_addr learned my addr 172.21.15.138:0/3708389612 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:52.835 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.833+0000 7f888be00640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c104b10 0x7f888c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 -- 172.21.15.138:0/3708389612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c104b10 msgr2=0x7f888c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c104b10 0x7f888c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 -- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c10aea0 msgr2=0x7f888c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c10aea0 0x7f888c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 -- 172.21.15.138:0/3708389612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c1a0fd0 con 0x7f888c10a520 2024-08-20T01:27:52.836 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f888b400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f887800e9f0 tx=0x7f887800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:52.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887800cd80 con 0x7f888c10a520 2024-08-20T01:27:52.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c1a12c0 con 0x7f888c10a520 2024-08-20T01:27:52.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c1a7b50 con 0x7f888c10a520 2024-08-20T01:27:52.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88780108e0 con 0x7f888c10a520 2024-08-20T01:27:52.837 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.834+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878026060 con 0x7f888c10a520 2024-08-20T01:27:52.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.835+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c0052c0 con 0x7f888c10a520 2024-08-20T01:27:52.838 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.838+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f8878010430 con 0x7f888c10a520 2024-08-20T01:27:52.839 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.839+0000 7f8888c00640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 0x7f88700b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:52.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.839+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f88780e43f0 con 0x7f888c10a520 2024-08-20T01:27:52.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.839+0000 7f888be00640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 0x7f88700b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:52.840 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.840+0000 7f888be00640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 0x7f88700b6320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f887c005ab0 tx=0x7f887c005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:52.841 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:52.840+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f88780aacb0 con 0x7f888c10a520 2024-08-20T01:27:53.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.059+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f886c0055b0 con 0x7f888c10a520 2024-08-20T01:27:53.060 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.060+0000 7f8888c00640 1 -- 172.21.15.138:0/3708389612 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) ==== 94+0+11938 (secure 0 0 0) 0x7f88780af200 con 0x7f888c10a520 2024-08-20T01:27:53.061 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:53.061 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":57,"btime":"2024-08-20T01:27:29:858179+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14634,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6835/1131228617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6834","nonce":1131228617},{"type":"v1","addr":"172.21.15.138:6835","nonce":1131228617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14646,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6837/3862920840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6836","nonce":3862920840},{"type":"v1","addr":"172.21.15.138:6837","nonce":3862920840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14703,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6841/3860533151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6840","nonce":3860533151},{"type":"v1","addr":"172.21.15.138:6841","nonce":3860533151}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24625,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6833/2791099826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":2791099826},{"type":"v1","addr":"172.21.15.167:6833","nonce":2791099826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24638,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/3285613354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":3285613354},{"type":"v1","addr":"172.21.15.142:6833","nonce":3285613354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24673,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/902364258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6834","nonce":902364258},{"type":"v1","addr":"172.21.15.167:6835","nonce":902364258}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":57,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-08-20T01:05:27.648119+0000","modified":"2024-08-20T01:27:29.262392+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24743,"mds_1":24706,"mds_2":24704},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6835/2224495590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":2224495590},{"type":"v1","addr":"172.21.15.142:6835","nonce":2224495590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24679":{"gid":24679,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6837/2270224611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6836","nonce":2270224611},{"type":"v1","addr":"172.21.15.167:6837","nonce":2270224611}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24704":{"gid":24704,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":21,"addr":"172.21.15.142:6837/1103910039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":1103910039},{"type":"v1","addr":"172.21.15.142:6837","nonce":1103910039}]},"join_fscid":-1,"export_targets":[0,1],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24706":{"gid":24706,"name":"g","rank":1,"incarnation":19,"state":"up:active","state_seq":19,"addr":"172.21.15.138:6839/1714719372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6838","nonce":1714719372},{"type":"v1","addr":"172.21.15.138:6839","nonce":1714719372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24743":{"gid":24743,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.142:6839/80285616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":80285616},{"type":"v1","addr":"172.21.15.142:6839","nonce":80285616}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26368":{"gid":26368,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.167:6839/2422640849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6838","nonce":2422640849},{"type":"v1","addr":"172.21.15.167:6839","nonce":2422640849}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24743,"qdb_cluster":[24706,24743,24704]},"id":1}]} 2024-08-20T01:27:53.061 INFO:teuthology.orchestra.run.smithi138.stderr:dumped fsmap epoch 57 2024-08-20T01:27:53.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.062+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 msgr2=0x7f88700b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:53.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.062+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 0x7f88700b6320 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f887c005ab0 tx=0x7f887c005a20 comp rx=0 tx=0).stop 2024-08-20T01:27:53.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.062+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 msgr2=0x7f888c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:53.063 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.062+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f887800e9f0 tx=0x7f887800eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:53.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.062+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 shutdown_connections 2024-08-20T01:27:53.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f88700b3e60 0x7f88700b6320 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:53.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f888c10aea0 0x7f888c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:53.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f888c10a520 0x7f888c19b460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:53.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 --2- 172.21.15.138:0/3708389612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f888c104b10 0x7f888c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:53.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 >> 172.21.15.138:0/3708389612 conn(0x7f888c100540 msgr2=0x7f888c103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:53.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 shutdown_connections 2024-08-20T01:27:53.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:53.063+0000 7f8891400640 1 -- 172.21.15.138:0/3708389612 wait complete. 2024-08-20T01:27:53.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:52 smithi167 ceph-mon[21965]: pgmap v981: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-20T01:27:53.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:52 smithi142 ceph-mon[21799]: pgmap v981: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-20T01:27:53.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:52 smithi138 ceph-mon[18076]: pgmap v981: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 62 op/s 2024-08-20T01:27:53.829 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd dump --format=json 2024-08-20T01:27:54.042 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:53 smithi138 ceph-mon[18076]: pgmap v982: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 812 KiB/s wr, 46 op/s 2024-08-20T01:27:54.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:53 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3708389612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:54.201 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:54.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:53 smithi167 ceph-mon[21965]: pgmap v982: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 812 KiB/s wr, 46 op/s 2024-08-20T01:27:54.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:53 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3708389612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:54.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:53 smithi142 ceph-mon[21799]: pgmap v982: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 812 KiB/s wr, 46 op/s 2024-08-20T01:27:54.243 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:53 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3708389612' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-08-20T01:27:54.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 -- 172.21.15.138:0/4016016878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108e80 msgr2=0x7fb77410f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:54.812 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4016016878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108e80 0x7fb77410f680 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009900 tx=0x7fb76c02a410 comp rx=0 tx=0).stop 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 -- 172.21.15.138:0/4016016878 shutdown_connections 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4016016878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108e80 0x7fb77410f680 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4016016878 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774108500 0x7fb774108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4016016878 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774102af0 0x7fb774102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 -- 172.21.15.138:0/4016016878 >> 172.21.15.138:0/4016016878 conn(0x7fb7740fe540 msgr2=0x7fb774100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 -- 172.21.15.138:0/4016016878 shutdown_connections 2024-08-20T01:27:54.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.787+0000 7fb77bc00640 1 -- 172.21.15.138:0/4016016878 wait complete. 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.788+0000 7fb77bc00640 1 Processor -- start 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.788+0000 7fb77bc00640 1 -- start start 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108500 0x7fb77419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108e80 0x7fb7741a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fb774112b50 con 0x7fb774108500 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fb7741129d0 con 0x7fb774108e80 2024-08-20T01:27:54.814 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb77bc00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fb774112cd0 con 0x7fb774102af0 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb773e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108500 0x7fb77419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:59960/0 (socket says 172.21.15.138:59960) 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 -- 172.21.15.138:0/4288608325 learned_addr learned my addr 172.21.15.138:0/4288608325 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779c00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108e80 0x7fb7741a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 -- 172.21.15.138:0/4288608325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108e80 msgr2=0x7fb7741a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108e80 0x7fb7741a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.815 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 -- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108500 msgr2=0x7fb77419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108500 0x7fb77419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.789+0000 7fb779200640 1 -- 172.21.15.138:0/4288608325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7741a1280 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb779200640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ebc0 tx=0x7fb76400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764010630 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7741a1570 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb764010c80 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764018b60 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.790+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741a7e00 con 0x7fb774102af0 2024-08-20T01:27:54.816 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.791+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb774102ed0 con 0x7fb774102af0 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.794+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fb764018d00 con 0x7fb774102af0 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.795+0000 7fb771600640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 0x7fb7580b6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.795+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7fb7640e4e70 con 0x7fb774102af0 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.796+0000 7fb773e00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 0x7fb7580b6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.796+0000 7fb773e00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 0x7fb7580b6200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb774104bd0 tx=0x7fb75c005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:54.817 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.797+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fb7640ab730 con 0x7fb774102af0 2024-08-20T01:27:54.986 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.985+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb7741a7fd0 con 0x7fb774102af0 2024-08-20T01:27:54.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.987+0000 7fb771600640 1 -- 172.21.15.138:0/4288608325 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+25421 (secure 0 0 0) 0x7fb7640afc80 con 0x7fb774102af0 2024-08-20T01:27:54.988 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-08-20T01:27:54.988 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":115,"fsid":"d876cd00-5e8e-11ef-bccf-c7b262605968","created":"2024-08-20T00:56:38.483080+0000","modified":"2024-08-20T01:27:15.820826+0000","last_up_change":"2024-08-20T01:04:29.929473+0000","last_in_change":"2024-08-20T01:04:06.769301+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-08-20T01:00:30.697585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-08-20T01:05:22.428168+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-08-20T01:05:24.585931+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-08-20T01:05:33.948877+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"27269bf1-da9b-45c9-930c-b9db2c82afb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6803","nonce":2240185671}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6805","nonce":2240185671}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6809","nonce":2240185671}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":2240185671},{"type":"v1","addr":"172.21.15.138:6807","nonce":2240185671}]},"public_addr":"172.21.15.138:6803/2240185671","cluster_addr":"172.21.15.138:6805/2240185671","heartbeat_back_addr":"172.21.15.138:6809/2240185671","heartbeat_front_addr":"172.21.15.138:6807/2240185671","state":["exists","up"]},{"osd":1,"uuid":"1a799e88-8e7f-48d0-a9bf-0c7fef85efd0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6801","nonce":1595380391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6803","nonce":1595380391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6807","nonce":1595380391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":1595380391},{"type":"v1","addr":"172.21.15.142:6805","nonce":1595380391}]},"public_addr":"172.21.15.142:6801/1595380391","cluster_addr":"172.21.15.142:6803/1595380391","heartbeat_back_addr":"172.21.15.142:6807/1595380391","heartbeat_front_addr":"172.21.15.142:6805/1595380391","state":["exists","up"]},{"osd":2,"uuid":"56932528-00db-4491-b933-3db45acdb6e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6801","nonce":3518437214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6803","nonce":3518437214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6807","nonce":3518437214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3518437214},{"type":"v1","addr":"172.21.15.167:6805","nonce":3518437214}]},"public_addr":"172.21.15.167:6801/3518437214","cluster_addr":"172.21.15.167:6803/3518437214","heartbeat_back_addr":"172.21.15.167:6807/3518437214","heartbeat_front_addr":"172.21.15.167:6805/3518437214","state":["exists","up"]},{"osd":3,"uuid":"73bed35d-9793-4b09-ba0b-41150f576f8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6811","nonce":876217371}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6813","nonce":876217371}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6817","nonce":876217371}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":876217371},{"type":"v1","addr":"172.21.15.138:6815","nonce":876217371}]},"public_addr":"172.21.15.138:6811/876217371","cluster_addr":"172.21.15.138:6813/876217371","heartbeat_back_addr":"172.21.15.138:6817/876217371","heartbeat_front_addr":"172.21.15.138:6815/876217371","state":["exists","up"]},{"osd":4,"uuid":"495167fb-1b60-4d94-ba5b-ff307d382779","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6809","nonce":2294964237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6811","nonce":2294964237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6815","nonce":2294964237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":2294964237},{"type":"v1","addr":"172.21.15.142:6813","nonce":2294964237}]},"public_addr":"172.21.15.142:6809/2294964237","cluster_addr":"172.21.15.142:6811/2294964237","heartbeat_back_addr":"172.21.15.142:6815/2294964237","heartbeat_front_addr":"172.21.15.142:6813/2294964237","state":["exists","up"]},{"osd":5,"uuid":"520885a9-d0be-4c97-9744-fd7f32d98544","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6809","nonce":808331745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6811","nonce":808331745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6815","nonce":808331745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":808331745},{"type":"v1","addr":"172.21.15.167:6813","nonce":808331745}]},"public_addr":"172.21.15.167:6809/808331745","cluster_addr":"172.21.15.167:6811/808331745","heartbeat_back_addr":"172.21.15.167:6815/808331745","heartbeat_front_addr":"172.21.15.167:6813/808331745","state":["exists","up"]},{"osd":6,"uuid":"b985af05-d307-4762-a760-8ab5fca6da56","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6819","nonce":2049480484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6821","nonce":2049480484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6825","nonce":2049480484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2049480484},{"type":"v1","addr":"172.21.15.138:6823","nonce":2049480484}]},"public_addr":"172.21.15.138:6819/2049480484","cluster_addr":"172.21.15.138:6821/2049480484","heartbeat_back_addr":"172.21.15.138:6825/2049480484","heartbeat_front_addr":"172.21.15.138:6823/2049480484","state":["exists","up"]},{"osd":7,"uuid":"03d4be9b-678d-41bc-813e-1e1d5bb2a29f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6817","nonce":2538307187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6819","nonce":2538307187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6823","nonce":2538307187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2538307187},{"type":"v1","addr":"172.21.15.142:6821","nonce":2538307187}]},"public_addr":"172.21.15.142:6817/2538307187","cluster_addr":"172.21.15.142:6819/2538307187","heartbeat_back_addr":"172.21.15.142:6823/2538307187","heartbeat_front_addr":"172.21.15.142:6821/2538307187","state":["exists","up"]},{"osd":8,"uuid":"42122c07-2099-4411-8845-faa6f97fd38d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6817","nonce":3511497686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6819","nonce":3511497686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6823","nonce":3511497686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3511497686},{"type":"v1","addr":"172.21.15.167:6821","nonce":3511497686}]},"public_addr":"172.21.15.167:6817/3511497686","cluster_addr":"172.21.15.167:6819/3511497686","heartbeat_back_addr":"172.21.15.167:6823/3511497686","heartbeat_front_addr":"172.21.15.167:6821/3511497686","state":["exists","up"]},{"osd":9,"uuid":"a1167294-30ac-465b-a5ff-e50abac4436f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6827","nonce":1914094697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6828","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6829","nonce":1914094697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6832","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6833","nonce":1914094697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6830","nonce":1914094697},{"type":"v1","addr":"172.21.15.138:6831","nonce":1914094697}]},"public_addr":"172.21.15.138:6827/1914094697","cluster_addr":"172.21.15.138:6829/1914094697","heartbeat_back_addr":"172.21.15.138:6833/1914094697","heartbeat_front_addr":"172.21.15.138:6831/1914094697","state":["exists","up"]},{"osd":10,"uuid":"7ee92625-f415-4027-952c-7eb2099d607c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6825","nonce":1588947901}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6827","nonce":1588947901}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6831","nonce":1588947901}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":1588947901},{"type":"v1","addr":"172.21.15.142:6829","nonce":1588947901}]},"public_addr":"172.21.15.142:6825/1588947901","cluster_addr":"172.21.15.142:6827/1588947901","heartbeat_back_addr":"172.21.15.142:6831/1588947901","heartbeat_front_addr":"172.21.15.142:6829/1588947901","state":["exists","up"]},{"osd":11,"uuid":"b15d21b0-5e3d-4f81-b1ad-d0bbfa75012a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6825","nonce":3808767407}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6827","nonce":3808767407}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6831","nonce":3808767407}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6828","nonce":3808767407},{"type":"v1","addr":"172.21.15.167:6829","nonce":3808767407}]},"public_addr":"172.21.15.167:6825/3808767407","cluster_addr":"172.21.15.167:6827/3808767407","heartbeat_back_addr":"172.21.15.167:6831/3808767407","heartbeat_front_addr":"172.21.15.167:6829/3808767407","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T00:59:41.073488+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:05.048260+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:28.845030+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:00:58.305754+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:23.343643+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:01:48.366235+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:17.988386+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:02:43.107917+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:08.871307+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:03:37.262863+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:02.722940+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-08-20T01:04:27.563989+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.b","mappings":[{"from":2,"to":4}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.1a","mappings":[{"from":5,"to":6}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]},{"pgid":"4.19","mappings":[{"from":9,"to":8}]},{"pgid":"4.1b","mappings":[{"from":9,"to":10}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6801/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6800/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2534070123":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/256603896":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3607751007":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/3948920804":"2024-08-21T00:58:34.377803+0000","172.21.15.138:6801/200546042":"2024-08-21T00:57:02.889250+0000","172.21.15.138:6800/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:6801/3853414921":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/64005733":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/1354849701":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/2736003984":"2024-08-21T00:57:30.262737+0000","172.21.15.138:0/3346890941":"2024-08-21T00:58:34.377803+0000","172.21.15.138:0/4262320942":"2024-08-21T00:57:02.889250+0000","172.21.15.138:0/3120862368":"2024-08-21T00:57:30.262737+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]},{"pool":4,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-08-20T01:27:54.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.989+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 msgr2=0x7fb7580b6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:54.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.989+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 0x7fb7580b6200 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb774104bd0 tx=0x7fb75c005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:54.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.989+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 msgr2=0x7fb77419b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:54.990 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.989+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb76400ebc0 tx=0x7fb76400c6a0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 shutdown_connections 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fb7580b3d40 0x7fb7580b6200 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb774108e80 0x7fb7741a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb774108500 0x7fb77419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 --2- 172.21.15.138:0/4288608325 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fb774102af0 0x7fb77419b1d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 >> 172.21.15.138:0/4288608325 conn(0x7fb7740fe540 msgr2=0x7fb7740750b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 shutdown_connections 2024-08-20T01:27:54.991 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:54.990+0000 7fb77bc00640 1 -- 172.21.15.138:0/4288608325 wait complete. 2024-08-20T01:27:55.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:55 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/4288608325' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:55.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:55 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/4288608325' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:55.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:55 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/4288608325' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-08-20T01:27:55.763 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs fail cephfs 2024-08-20T01:27:56.134 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:56.309 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:56 smithi138 ceph-mon[18076]: pgmap v983: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 521 KiB/s wr, 38 op/s 2024-08-20T01:27:56.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:56 smithi167 ceph-mon[21965]: pgmap v983: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 521 KiB/s wr, 38 op/s 2024-08-20T01:27:56.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:56 smithi142 ceph-mon[21799]: pgmap v983: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 521 KiB/s wr, 38 op/s 2024-08-20T01:27:56.748 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 -- 172.21.15.138:0/3008037046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108c90 msgr2=0x7f81ec10f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 --2- 172.21.15.138:0/3008037046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108c90 0x7f81ec10f490 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009ca0 tx=0x7f81e402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 -- 172.21.15.138:0/3008037046 shutdown_connections 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 --2- 172.21.15.138:0/3008037046 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108c90 0x7f81ec10f490 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 --2- 172.21.15.138:0/3008037046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec108310 0x7f81ec108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 --2- 172.21.15.138:0/3008037046 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec102900 0x7f81ec102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.749 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 -- 172.21.15.138:0/3008037046 >> 172.21.15.138:0/3008037046 conn(0x7f81ec0fe390 msgr2=0x7f81ec1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:56.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 -- 172.21.15.138:0/3008037046 shutdown_connections 2024-08-20T01:27:56.750 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.748+0000 7f81ebe00640 1 -- 172.21.15.138:0/3008037046 wait complete. 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 Processor -- start 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 -- start start 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec102900 0x7f81ec19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec108c90 0x7f81ec1a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec112960 con 0x7f81ec108310 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1127e0 con 0x7f81ec102900 2024-08-20T01:27:56.752 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.749+0000 7f81ebe00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec112ae0 con 0x7f81ec108c90 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:38986/0 (socket says 172.21.15.138:38986) 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 -- 172.21.15.138:0/489069717 learned_addr learned my addr 172.21.15.138:0/489069717 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81eaa00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec102900 0x7f81ec19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81eb400640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec108c90 0x7f81ec1a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 -- 172.21.15.138:0/489069717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec108c90 msgr2=0x7f81ec1a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec108c90 0x7f81ec1a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.753 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 -- 172.21.15.138:0/489069717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec102900 msgr2=0x7f81ec19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec102900 0x7f81ec19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.750+0000 7f81e2000640 1 -- 172.21.15.138:0/489069717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec1a0f60 con 0x7f81ec108310 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81e2000640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f81d400e9f0 tx=0x7f81d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d400cd80 con 0x7f81ec108310 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec1a1250 con 0x7f81ec108310 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d40108e0 con 0x7f81ec108310 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4018980 con 0x7f81ec108310 2024-08-20T01:27:56.754 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.751+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec1a7ae0 con 0x7f81ec108310 2024-08-20T01:27:56.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.752+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81ec103bb0 con 0x7f81ec108310 2024-08-20T01:27:56.755 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.754+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f81d4010430 con 0x7f81ec108310 2024-08-20T01:27:56.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.756+0000 7f81e3e00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 0x7f81d00b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:56.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.756+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9453+0+0 (secure 0 0 0) 0x7f81d40e46a0 con 0x7f81ec108310 2024-08-20T01:27:56.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.757+0000 7f81eaa00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 0x7f81d00b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:56.758 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.757+0000 7f81eaa00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 0x7f81d00b6320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00bbe0 tx=0x7f81dc00b390 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:56.760 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.759+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f81d40aaf60 con 0x7f81ec108310 2024-08-20T01:27:56.980 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.980+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f81ec1a7cb0 con 0x7f81ec108310 2024-08-20T01:27:56.983 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.983+0000 7f81e3e00640 1 -- 172.21.15.138:0/489069717 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) ==== 161+0+0 (secure 0 0 0) 0x7f81d40af4b0 con 0x7f81ec108310 2024-08-20T01:27:56.984 INFO:teuthology.orchestra.run.smithi138.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-08-20T01:27:56.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 msgr2=0x7f81d00b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:56.987 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 0x7f81d00b6320 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00bbe0 tx=0x7f81dc00b390 comp rx=0 tx=0).stop 2024-08-20T01:27:56.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 msgr2=0x7f81ec19b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:56.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f81d400e9f0 tx=0x7f81d400eeb0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 shutdown_connections 2024-08-20T01:27:56.988 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f81d00b3e60 0x7f81d00b6320 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81ec108c90 0x7f81ec1a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f81ec108310 0x7f81ec19b3f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 --2- 172.21.15.138:0/489069717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f81ec102900 0x7f81ec19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.987+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 >> 172.21.15.138:0/489069717 conn(0x7f81ec0fe390 msgr2=0x7f81ec0750e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.988+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 shutdown_connections 2024-08-20T01:27:56.989 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:56.988+0000 7f81ebe00640 1 -- 172.21.15.138:0/489069717 wait complete. 2024-08-20T01:27:57.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:56 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a[18072]: 2024-08-20T01:27:56.981+0000 7f64ff800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T01:27:57.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T01:27:57.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T01:27:57.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T01:27:57.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T01:27:57.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: osdmap e116: 12 total, 12 up, 12 in 2024-08-20T01:27:57.294 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mon[18076]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T01:27:57.294 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.295 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.295 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-mds[63150]: -- 172.21.15.138:0/2989455814 <== mon.0 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556aa36b3860 con 0x556aa3691400 2024-08-20T01:27:57.295 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:27:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: starting mds.g at 2024-08-20T01:27:57.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: osdmap e116: 12 total, 12 up, 12 in 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mon[21965]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T01:27:57.466 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.467 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.467 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mds[53268]: -- 172.21.15.167:0/1521160420 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5631764d9860 con 0x5631764b7400 2024-08-20T01:27:57.467 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: starting mds.c at 2024-08-20T01:27:57.467 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.468 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.468 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-mds[53569]: -- 172.21.15.167:0/2750773150 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e2fe83860 con 0x562e2fe61000 2024-08-20T01:27:57.468 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:27:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: starting mds.i at 2024-08-20T01:27:57.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-08-20T01:27:57.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-08-20T01:27:57.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-08-20T01:27:57.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/489069717' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-08-20T01:27:57.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: osdmap e116: 12 total, 12 up, 12 in 2024-08-20T01:27:57.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mon[21799]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-08-20T01:27:57.494 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.494 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.494 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mds[52684]: -- 172.21.15.142:0/2147212260 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ad9af3f860 con 0x55ad9af1d000 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: starting mds.h at 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mds[52983]: -- 172.21.15.142:0/1004546475 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ae9f53bc20 con 0x55ae9f518800 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: starting mds.b at 2024-08-20T01:27:57.495 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: ignoring --setuser ceph since I am not root 2024-08-20T01:27:57.496 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: ignoring --setgroup ceph since I am not root 2024-08-20T01:27:57.496 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mds[52383]: -- 172.21.15.142:0/1423928872 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564324c97520 con 0x564324c6b000 2024-08-20T01:27:57.496 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-mds[52383]: -- 172.21.15.142:0/1423928872 <== mon.1 v2:172.21.15.167:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564324c8dc20 con 0x564324c6b000 2024-08-20T01:27:57.496 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:27:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: starting mds.e at 2024-08-20T01:27:57.755 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-08-20T01:27:58.127 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:27:58.319 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: pgmap v985: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-08-20T01:27:58.319 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6836/2055392329,v1:172.21.15.167:6838/2055392329] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.167:6837/1473082639,v1:172.21.15.167:6839/1473082639] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6834/1715220281,v1:172.21.15.142:6837/1715220281] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6836/2109628311,v1:172.21.15.142:6839/2109628311] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.142:6835/3512089093,v1:172.21.15.142:6838/3512089093] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: mds.? [v2:172.21.15.138:6838/506909375,v1:172.21.15.138:6839/506909375] up:boot 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:27:58.320 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:27:58.321 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:27:58.321 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:27:58.321 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:27:58.321 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:58 smithi138 ceph-mon[18076]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:27:58.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: pgmap v985: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-08-20T01:27:58.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6836/2055392329,v1:172.21.15.167:6838/2055392329] up:boot 2024-08-20T01:27:58.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.167:6837/1473082639,v1:172.21.15.167:6839/1473082639] up:boot 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6834/1715220281,v1:172.21.15.142:6837/1715220281] up:boot 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6836/2109628311,v1:172.21.15.142:6839/2109628311] up:boot 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.142:6835/3512089093,v1:172.21.15.142:6838/3512089093] up:boot 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: mds.? [v2:172.21.15.138:6838/506909375,v1:172.21.15.138:6839/506909375] up:boot 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:27:58.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:27:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:27:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:27:58.467 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:58 smithi167 ceph-mon[21965]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:27:58.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: pgmap v985: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 16 op/s 2024-08-20T01:27:58.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6836/2055392329,v1:172.21.15.167:6838/2055392329] up:boot 2024-08-20T01:27:58.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.167:6837/1473082639,v1:172.21.15.167:6839/1473082639] up:boot 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6834/1715220281,v1:172.21.15.142:6837/1715220281] up:boot 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6836/2109628311,v1:172.21.15.142:6839/2109628311] up:boot 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.142:6835/3512089093,v1:172.21.15.142:6838/3512089093] up:boot 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: mds.? [v2:172.21.15.138:6838/506909375,v1:172.21.15.138:6839/506909375] up:boot 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-08-20T01:27:58.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-08-20T01:27:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-08-20T01:27:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-08-20T01:27:58.495 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:58 smithi142 ceph-mon[21799]: from='mgr.14220 172.21.15.138:0/2516676640' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-08-20T01:27:58.716 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 -- 172.21.15.138:0/1698303566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c10a510 msgr2=0x7f717c10a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 --2- 172.21.15.138:0/1698303566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c10a510 0x7f717c10a950 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7168009900 tx=0x7f716802a400 comp rx=0 tx=0).stop 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 -- 172.21.15.138:0/1698303566 shutdown_connections 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 --2- 172.21.15.138:0/1698303566 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 --2- 172.21.15.138:0/1698303566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c10a510 0x7f717c10a950 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 --2- 172.21.15.138:0/1698303566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c104b00 0x7f717c104ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:58.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 -- 172.21.15.138:0/1698303566 >> 172.21.15.138:0/1698303566 conn(0x7f717c100550 msgr2=0x7f717c102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 -- 172.21.15.138:0/1698303566 shutdown_connections 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.693+0000 7f7182400640 1 -- 172.21.15.138:0/1698303566 wait complete. 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.694+0000 7f7182400640 1 Processor -- start 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.694+0000 7f7182400640 1 -- start start 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 0x7f717c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:58.718 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f717c114b60 con 0x7f717c104b00 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f717c1149e0 con 0x7f717c10a510 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7182400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f717c114ce0 con 0x7f717c10ae90 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181000640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 0x7f717c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42100/0 (socket says 172.21.15.138:42100) 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:54476/0 (socket says 172.21.15.138:54476) 2024-08-20T01:27:58.719 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181a00640 1 -- 172.21.15.138:0/2318253967 learned_addr learned my addr 172.21.15.138:0/2318253967 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 -- 172.21.15.138:0/2318253967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 msgr2=0x7f717c1a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 -- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 msgr2=0x7f717c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 0x7f717c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f717be00640 1 -- 172.21.15.138:0/2318253967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c1a0fd0 con 0x7f717c10a510 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181a00640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.695+0000 7f7181000640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 0x7f717c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-08-20T01:27:58.720 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f717be00640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f716802a9f0 tx=0x7f7168003b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716803f070 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7168031b50 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168035030 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c1a1260 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.696+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c1a7980 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.698+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c104ee0 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.700+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f7168042070 con 0x7f717c10a510 2024-08-20T01:27:58.721 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.702+0000 7f7179600640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 0x7f71640b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:27:58.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.702+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9623+0+0 (secure 0 0 0) 0x7f7168104ac0 con 0x7f717c10a510 2024-08-20T01:27:58.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.702+0000 7f7181000640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 0x7f71640b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:27:58.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.703+0000 7f7181000640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 0x7f71640b6370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7170009b50 tx=0x7f7170005f90 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:27:58.722 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.705+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f71680cb920 con 0x7f717c10a510 2024-08-20T01:27:58.929 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:58.928+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f717c10cca0 con 0x7f717c10a510 2024-08-20T01:27:59.062 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.061+0000 7f7179600640 1 -- 172.21.15.138:0/2318253967 <== mon.2 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7f71680cb220 con 0x7f717c10a510 2024-08-20T01:27:59.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 msgr2=0x7f71640b6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:59.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 0x7f71640b6370 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7170009b50 tx=0x7f7170005f90 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 msgr2=0x7f717c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f716802a9f0 tx=0x7f7168003b20 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 shutdown_connections 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f71640b3eb0 0x7f71640b6370 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f717c10ae90 0x7f717c1a0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f717c10a510 0x7f717c19b460 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 --2- 172.21.15.138:0/2318253967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f717c104b00 0x7f717c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:27:59.065 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 >> 172.21.15.138:0/2318253967 conn(0x7f717c100550 msgr2=0x7f717c107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:27:59.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 shutdown_connections 2024-08-20T01:27:59.066 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:27:59.064+0000 7f7182400640 1 -- 172.21.15.138:0/2318253967 wait complete. 2024-08-20T01:27:59.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:59 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/2318253967' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:27:59 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:59 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/2318253967' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:27:59 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:59 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/2318253967' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:27:59 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-08-20T01:27:59.914 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-08-20T01:28:00.291 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:28:00.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: pgmap v986: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 15 op/s 2024-08-20T01:28:00.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T01:28:00.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T01:28:00.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: Cluster is now healthy 2024-08-20T01:28:00.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:28:00.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:00 smithi138 ceph-mon[18076]: fsmap 12 up:standby 2024-08-20T01:28:00.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: pgmap v986: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 15 op/s 2024-08-20T01:28:00.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T01:28:00.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T01:28:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: Cluster is now healthy 2024-08-20T01:28:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:28:00.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:00 smithi167 ceph-mon[21965]: fsmap 12 up:standby 2024-08-20T01:28:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: pgmap v986: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 217 KiB/s wr, 15 op/s 2024-08-20T01:28:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-08-20T01:28:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-08-20T01:28:00.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: Cluster is now healthy 2024-08-20T01:28:00.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-08-20T01:28:00.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:00 smithi142 ceph-mon[21799]: fsmap 12 up:standby 2024-08-20T01:28:00.940 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 -- 172.21.15.138:0/2674345696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2108104ca0 msgr2=0x7f2108105080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 --2- 172.21.15.138:0/2674345696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2108104ca0 0x7f2108105080 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009860 tx=0x7f20f802a340 comp rx=0 tx=0).stop 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 -- 172.21.15.138:0/2674345696 shutdown_connections 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 --2- 172.21.15.138:0/2674345696 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f2108111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 --2- 172.21.15.138:0/2674345696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f210810a6b0 0x7f210810aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 --2- 172.21.15.138:0/2674345696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2108104ca0 0x7f2108105080 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 -- 172.21.15.138:0/2674345696 >> 172.21.15.138:0/2674345696 conn(0x7f2108100730 msgr2=0x7f2108102b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 -- 172.21.15.138:0/2674345696 shutdown_connections 2024-08-20T01:28:01.037 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.905+0000 7f210d400640 1 -- 172.21.15.138:0/2674345696 wait complete. 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.906+0000 7f210d400640 1 Processor -- start 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.906+0000 7f210d400640 1 -- start start 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108104ca0 0x7f210819b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114cf0 con 0x7f2108104ca0 2024-08-20T01:28:01.038 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114b70 con 0x7f210810a6b0 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f210d400640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f2108114e70 con 0x7f210810b030 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2106800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108104ca0 0x7f210819b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f20ffe00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42124/0 (socket says 172.21.15.138:42124) 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 -- 172.21.15.138:0/1569654142 learned_addr learned my addr 172.21.15.138:0/1569654142 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f20ffe00640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.138:54496/0 (socket says 172.21.15.138:54496) 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 -- 172.21.15.138:0/1569654142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 msgr2=0x7f210819b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:01.039 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 -- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108104ca0 msgr2=0x7f210819b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108104ca0 0x7f210819b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f2107200640 1 -- 172.21.15.138:0/1569654142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081a1230 con 0x7f210810b030 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.907+0000 7f20ffe00640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f2107200640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f210000eb10 tx=0x7f210000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100004b20 con 0x7f210810b030 2024-08-20T01:28:01.040 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081a1520 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2100004cc0 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100010690 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.908+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081a7db0 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.909+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2108105080 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.911+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f2100005000 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.913+0000 7f20ff400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 0x7f20ec0b6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.913+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9623+0+0 (secure 0 0 0) 0x7f21000e4b80 con 0x7f210810b030 2024-08-20T01:28:01.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.913+0000 7f2106800640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 0x7f20ec0b6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:01.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.914+0000 7f2106800640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 0x7f20ec0b6280 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009830 tx=0x7f20f8030340 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:01.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:00.915+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f2100055e20 con 0x7f210810b030 2024-08-20T01:28:01.113 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:01.112+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f210819c2b0 con 0x7f210810b030 2024-08-20T01:28:02.064 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.062+0000 7f20ff400640 1 -- 172.21.15.138:0/1569654142 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) ==== 181+0+0 (secure 0 0 0) 0x7f21000ab9e0 con 0x7f210810b030 2024-08-20T01:28:02.064 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_metadata' removed 2024-08-20T01:28:02.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.069+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 msgr2=0x7f20ec0b6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:02.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.069+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 0x7f20ec0b6280 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009830 tx=0x7f20f8030340 comp rx=0 tx=0).stop 2024-08-20T01:28:02.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.069+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 msgr2=0x7f21081a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:02.071 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.069+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f210000eb10 tx=0x7f210000c2d0 comp rx=0 tx=0).stop 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 shutdown_connections 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f20ec0b3dc0 0x7f20ec0b6280 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f210810b030 0x7f21081a0a60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f210810a6b0 0x7f210819b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 --2- 172.21.15.138:0/1569654142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108104ca0 0x7f210819b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:02.072 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.070+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 >> 172.21.15.138:0/1569654142 conn(0x7f2108100730 msgr2=0x7f2108102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.072+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 shutdown_connections 2024-08-20T01:28:02.073 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:02.072+0000 7f210d400640 1 -- 172.21.15.138:0/1569654142 wait complete. 2024-08-20T01:28:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:02 smithi167 ceph-mon[21965]: pgmap v987: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 123 KiB/s wr, 10 op/s 2024-08-20T01:28:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:02 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/1569654142' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.215 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:02 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:02 smithi142 ceph-mon[21799]: pgmap v987: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 123 KiB/s wr, 10 op/s 2024-08-20T01:28:02.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:02 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/1569654142' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:02 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.504 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:02 smithi138 ceph-mon[18076]: pgmap v987: 129 pgs: 129 active+clean; 1.1 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 123 KiB/s wr, 10 op/s 2024-08-20T01:28:02.504 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:02 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/1569654142' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.504 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:02 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:02.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-08-20T01:28:03.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:03 smithi138 ceph-mon[18076]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:03.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:03 smithi138 ceph-mon[18076]: osdmap e117: 12 total, 12 up, 12 in 2024-08-20T01:28:03.299 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:28:03.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:03 smithi167 ceph-mon[21965]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:03.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:03 smithi167 ceph-mon[21965]: osdmap e117: 12 total, 12 up, 12 in 2024-08-20T01:28:03.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:03 smithi142 ceph-mon[21799]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:03.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:03 smithi142 ceph-mon[21799]: osdmap e117: 12 total, 12 up, 12 in 2024-08-20T01:28:04.117 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:04 smithi138 ceph-mon[18076]: pgmap v989: 97 pgs: 97 active+clean; 315 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:04.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 -- 172.21.15.138:0/3677145738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff4104b00 msgr2=0x7fdff4104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:04.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3677145738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff4104b00 0x7fdff4104ee0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009900 tx=0x7fdfe402a410 comp rx=0 tx=0).stop 2024-08-20T01:28:04.141 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 -- 172.21.15.138:0/3677145738 shutdown_connections 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3677145738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410ae90 0x7fdff4111690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3677145738 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff410a510 0x7fdff410a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3677145738 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff4104b00 0x7fdff4104ee0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 -- 172.21.15.138:0/3677145738 >> 172.21.15.138:0/3677145738 conn(0x7fdff4100550 msgr2=0x7fdff4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 -- 172.21.15.138:0/3677145738 shutdown_connections 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.114+0000 7fdff3e00640 1 -- 172.21.15.138:0/3677145738 wait complete. 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 Processor -- start 2024-08-20T01:28:04.142 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 -- start start 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410a510 0x7fdff419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 0x7fdff41a07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4114ba0 con 0x7fdff410ae90 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4114a20 con 0x7fdff410a510 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.116+0000 7fdff3e00640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7fdff4114d20 con 0x7fdff4104b00 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:04.143 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff2a00640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.138:42152/0 (socket says 172.21.15.138:42152) 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff2a00640 1 -- 172.21.15.138:0/3405804496 learned_addr learned my addr 172.21.15.138:0/3405804496 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 0x7fdff41a07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdfea000640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410a510 0x7fdff419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 -- 172.21.15.138:0/3405804496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 msgr2=0x7fdff419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 -- 172.21.15.138:0/3405804496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410a510 msgr2=0x7fdff419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410a510 0x7fdff419b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:04.144 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 -- 172.21.15.138:0/3405804496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff41a0fc0 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff2a00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.117+0000 7fdff3400640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 0x7fdff41a07f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e940 tx=0x7fdfec00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.118+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec004d20 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.118+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfec005020 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.118+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec0116d0 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.118+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff41a12b0 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.118+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff41a7970 con 0x7fdff410ae90 2024-08-20T01:28:04.145 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.119+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff4104ee0 con 0x7fdff410ae90 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.121+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7fdfec005460 con 0x7fdff410ae90 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.122+0000 7fdfebe00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 0x7fdfd80b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.123+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9098+0+0 (secure 0 0 0) 0x7fdfec0e4650 con 0x7fdff410ae90 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.123+0000 7fdff2a00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 0x7fdfd80b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.123+0000 7fdff2a00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 0x7fdfd80b6320 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40096c0 tx=0x7fdfe4005a20 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:04.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.124+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7fdfec016030 con 0x7fdff410ae90 2024-08-20T01:28:04.312 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:04.311+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdff41a7d00 con 0x7fdff410ae90 2024-08-20T01:28:04.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:04 smithi167 ceph-mon[21965]: pgmap v989: 97 pgs: 97 active+clean; 315 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:04.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:04 smithi142 ceph-mon[21799]: pgmap v989: 97 pgs: 97 active+clean; 315 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:05.075 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.073+0000 7fdfebe00640 1 -- 172.21.15.138:0/3405804496 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) ==== 169+0+0 (secure 0 0 0) 0x7fdfec0ab0f0 con 0x7fdff410ae90 2024-08-20T01:28:05.075 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data' removed 2024-08-20T01:28:05.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.076+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 msgr2=0x7fdfd80b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:05.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.076+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 0x7fdfd80b6320 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40096c0 tx=0x7fdfe4005a20 comp rx=0 tx=0).stop 2024-08-20T01:28:05.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.076+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 msgr2=0x7fdff41a07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:05.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.076+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 0x7fdff41a07f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00e940 tx=0x7fdfec00ee00 comp rx=0 tx=0).stop 2024-08-20T01:28:05.079 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 shutdown_connections 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7fdfd80b3e60 0x7fdfd80b6320 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdff410ae90 0x7fdff41a07f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdff410a510 0x7fdff419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 --2- 172.21.15.138:0/3405804496 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdff4104b00 0x7fdff419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 >> 172.21.15.138:0/3405804496 conn(0x7fdff4100550 msgr2=0x7fdff4101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:05.080 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.077+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 shutdown_connections 2024-08-20T01:28:05.082 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:05.078+0000 7fdff3e00640 1 -- 172.21.15.138:0/3405804496 wait complete. 2024-08-20T01:28:05.446 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:05 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:05.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:05 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:05.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:05 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:06.159 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1 shell --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-08-20T01:28:06.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:06 smithi167 ceph-mon[21965]: pgmap v990: 97 pgs: 97 active+clean; 315 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:06 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:06.466 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:06 smithi167 ceph-mon[21965]: osdmap e118: 12 total, 12 up, 12 in 2024-08-20T01:28:06.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:06 smithi142 ceph-mon[21799]: pgmap v990: 97 pgs: 97 active+clean; 315 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:06.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:06 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:06.494 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:06 smithi142 ceph-mon[21799]: osdmap e118: 12 total, 12 up, 12 in 2024-08-20T01:28:06.539 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/mon.a/config 2024-08-20T01:28:06.542 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:06 smithi138 ceph-mon[18076]: pgmap v990: 97 pgs: 97 active+clean; 315 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:06.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:06 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/3405804496' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:06.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:06 smithi138 ceph-mon[18076]: osdmap e118: 12 total, 12 up, 12 in 2024-08-20T01:28:07.259 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.144+0000 7f6589800640 1 -- 172.21.15.138:0/2751773057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6584104b10 msgr2=0x7f6584104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.144+0000 7f6589800640 1 --2- 172.21.15.138:0/2751773057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6584104b10 0x7f6584104ef0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6574009ca0 tx=0x7f657402a8c0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 -- 172.21.15.138:0/2751773057 shutdown_connections 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 --2- 172.21.15.138:0/2751773057 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f658410aea0 0x7f65841116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 --2- 172.21.15.138:0/2751773057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410a520 0x7f658410a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 --2- 172.21.15.138:0/2751773057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6584104b10 0x7f6584104ef0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 -- 172.21.15.138:0/2751773057 >> 172.21.15.138:0/2751773057 conn(0x7f6584100540 msgr2=0x7f6584102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 -- 172.21.15.138:0/2751773057 shutdown_connections 2024-08-20T01:28:07.260 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.145+0000 7f6589800640 1 -- 172.21.15.138:0/2751773057 wait complete. 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 Processor -- start 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 -- start start 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6584104b10 0x7f658419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410aea0 0x7f65841a0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 -- 0x7f6584114bb0 con 0x7f658410a520 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 -- 0x7f6584114a30 con 0x7f658410aea0 2024-08-20T01:28:07.261 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6589800640 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 -- 0x7f6584114d30 con 0x7f6584104b10 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6583400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6583400640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:57034/0 (socket says 172.21.15.138:57034) 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.146+0000 7f6583400640 1 -- 172.21.15.138:0/65365098 learned_addr learned my addr 172.21.15.138:0/65365098 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6588e00640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410aea0 0x7f65841a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583e00640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6584104b10 0x7f658419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 -- 172.21.15.138:0/65365098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6584104b10 msgr2=0x7f658419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:07.262 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6584104b10 0x7f658419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 -- 172.21.15.138:0/65365098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410aea0 msgr2=0x7f65841a0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410aea0 0x7f65841a0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 -- 172.21.15.138:0/65365098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65841a1010 con 0x7f658410a520 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6583400640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f656c00e9f0 tx=0x7f656c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c00cd80 con 0x7f658410a520 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c0108e0 con 0x7f658410a520 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c018980 con 0x7f658410a520 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.147+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65841a1300 con 0x7f658410a520 2024-08-20T01:28:07.263 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.148+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65841a7b90 con 0x7f658410a520 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.149+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6584104ef0 con 0x7f658410a520 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.152+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) ==== 151191+0+0 (secure 0 0 0) 0x7f656c010430 con 0x7f658410a520 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.153+0000 7f6580c00640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 0x7f65680b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.153+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8539+0+0 (secure 0 0 0) 0x7f656c0e47d0 con 0x7f658410a520 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.153+0000 7f6583e00640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 0x7f65680b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.154+0000 7f6583e00640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 0x7f65680b6320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6574006010 tx=0x7f6574009820 comp rx=0 tx=0).ready entity=mgr.14220 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-08-20T01:28:07.264 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.154+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196112 (secure 0 0 0) 0x7f656c0ab420 con 0x7f658410a520 2024-08-20T01:28:07.345 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:07.344+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f658419c090 con 0x7f658410a520 2024-08-20T01:28:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.085+0000 7f6580c00640 1 -- 172.21.15.138:0/65365098 <== mon.0 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v119) ==== 178+0+0 (secure 0 0 0) 0x7f658419c090 con 0x7f658410a520 2024-08-20T01:28:08.087 INFO:teuthology.orchestra.run.smithi138.stderr:pool 'cephfs_data_ec' removed 2024-08-20T01:28:08.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.089+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 msgr2=0x7f65680b6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:08.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.089+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 0x7f65680b6320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6574006010 tx=0x7f6574009820 comp rx=0 tx=0).stop 2024-08-20T01:28:08.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.089+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 msgr2=0x7f658419b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-08-20T01:28:08.091 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.089+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f656c00e9f0 tx=0x7f656c00eeb0 comp rx=0 tx=0).stop 2024-08-20T01:28:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 shutdown_connections 2024-08-20T01:28:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:6800/2106854212,v1:172.21.15.138:6801/2106854212] conn(0x7f65680b3e60 0x7f65680b6320 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f658410aea0 0x7f65841a0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f658410a520 0x7f658419b4a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:08.092 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 --2- 172.21.15.138:0/65365098 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6584104b10 0x7f658419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-08-20T01:28:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.090+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 >> 172.21.15.138:0/65365098 conn(0x7f6584100540 msgr2=0x7f6584103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-08-20T01:28:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.091+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 shutdown_connections 2024-08-20T01:28:08.093 INFO:teuthology.orchestra.run.smithi138.stderr:2024-08-20T01:28:08.091+0000 7f6589800640 1 -- 172.21.15.138:0/65365098 wait complete. 2024-08-20T01:28:08.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:08 smithi167 ceph-mon[21965]: pgmap v992: 33 pgs: 33 active+clean; 315 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:08.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:08 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:08.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:08 smithi142 ceph-mon[21799]: pgmap v992: 33 pgs: 33 active+clean; 315 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:08.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:08 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:08.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:08 smithi138 ceph-mon[18076]: pgmap v992: 33 pgs: 33 active+clean; 315 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail 2024-08-20T01:28:08.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:08 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} : dispatch 2024-08-20T01:28:09.075 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:28:09.136 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:28:09.177 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:28:09.206 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-08-20T01:28:09.206 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T01:28:09.242 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T01:28:09.258 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-08-20T01:28:09.273 INFO:tasks.cephadm:Stopping all daemons... 2024-08-20T01:28:09.273 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-08-20T01:28:09.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a 2024-08-20T01:28:09.292 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 ceph-mon[18076]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:09.293 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 ceph-mon[18076]: osdmap e119: 12 total, 12 up, 12 in 2024-08-20T01:28:09.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:09 smithi167 ceph-mon[21965]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:09.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:09 smithi167 ceph-mon[21965]: osdmap e119: 12 total, 12 up, 12 in 2024-08-20T01:28:09.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:09 smithi142 ceph-mon[21799]: from='client.? 172.21.15.138:0/65365098' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-08-20T01:28:09.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:09 smithi142 ceph-mon[21799]: osdmap e119: 12 total, 12 up, 12 in 2024-08-20T01:28:09.712 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 systemd[1]: Stopping Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:10.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a[18072]: 2024-08-20T01:28:09.709+0000 7f6506600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:10.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a[18072]: 2024-08-20T01:28:09.709+0000 7f6506600640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-08-20T01:28:10.043 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:09 smithi138 podman[96574]: 2024-08-20 01:28:09.762732262 +0000 UTC m=+0.205571487 container died 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-08-20T01:28:10.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 podman[96574]: 2024-08-20 01:28:10.227741645 +0000 UTC m=+0.670580877 container cleanup 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-08-20T01:28:10.543 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 bash[96574]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a 2024-08-20T01:28:10.770 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service' 2024-08-20T01:28:10.860 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 podman[96585]: 2024-08-20 01:28:10.602226583 +0000 UTC m=+0.825426423 container remove 41ad98b5aaeae356d962a80893e3d47341137a3b94dc1ffd7b2047de81cac21d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-a, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:28:10.860 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service: Deactivated successfully. 2024-08-20T01:28:10.861 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 systemd[1]: Stopped Ceph mon.a for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:28:10.861 INFO:journalctl@ceph.mon.a.smithi138.stdout:Aug 20 01:28:10 smithi138 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.a.service: Consumed 30.044s CPU time. 2024-08-20T01:28:10.911 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:10.911 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-08-20T01:28:10.912 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-08-20T01:28:10.912 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.b 2024-08-20T01:28:11.239 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:10 smithi142 systemd[1]: Stopping Ceph mon.b for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:11.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-b[21795]: 2024-08-20T01:28:11.238+0000 7f27c815a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:11.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-b[21795]: 2024-08-20T01:28:11.238+0000 7f27c815a640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-08-20T01:28:11.493 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:11 smithi142 podman[59057]: 2024-08-20 01:28:11.269933812 +0000 UTC m=+0.154027909 container died df50ac551d24d80ad817891ebf354539ab62708048f5e924bc47f8e9463d8037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T01:28:11.941 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:11 smithi142 podman[59057]: 2024-08-20 01:28:11.648006095 +0000 UTC m=+0.532100194 container cleanup df50ac551d24d80ad817891ebf354539ab62708048f5e924bc47f8e9463d8037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-b, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-20T01:28:11.942 INFO:journalctl@ceph.mon.b.smithi142.stdout:Aug 20 01:28:11 smithi142 bash[59057]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-b 2024-08-20T01:28:12.077 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.b.service' 2024-08-20T01:28:12.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:12.122 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-08-20T01:28:12.123 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-08-20T01:28:12.123 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.c 2024-08-20T01:28:12.403 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:12 smithi167 systemd[1]: Stopping Ceph mon.c for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:13.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c[21961]: 2024-08-20T01:28:13.111+0000 7faad9c34640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:13.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c[21961]: 2024-08-20T01:28:13.111+0000 7faad9c34640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-08-20T01:28:13.465 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:13 smithi167 podman[60395]: 2024-08-20 01:28:13.203639727 +0000 UTC m=+0.224710122 container died f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-20T01:28:13.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:13 smithi167 podman[60395]: 2024-08-20 01:28:13.59632767 +0000 UTC m=+0.617398111 container cleanup f61e619d275c3931e3d00b24f3f9ca68af9f3da277399ed259f46af3c657ad21 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-08-20T01:28:13.875 INFO:journalctl@ceph.mon.c.smithi167.stdout:Aug 20 01:28:13 smithi167 bash[60395]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mon-c 2024-08-20T01:28:14.024 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mon.c.service' 2024-08-20T01:28:14.074 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:14.074 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-08-20T01:28:14.075 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-08-20T01:28:14.075 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x 2024-08-20T01:28:14.543 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:14 smithi138 systemd[1]: Stopping Ceph mgr.x for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:15.043 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:28:14.579+0000 7f57ce800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:15.043 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x[18300]: 2024-08-20T01:28:14.579+0000 7f57ce800640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-08-20T01:28:15.043 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:14 smithi138 podman[96775]: 2024-08-20 01:28:14.662428179 +0000 UTC m=+0.285715285 container died 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-08-20T01:28:15.312 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:15 smithi138 podman[96775]: 2024-08-20 01:28:15.309339186 +0000 UTC m=+0.932626292 container cleanup 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:28:15.312 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:15 smithi138 bash[96775]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x 2024-08-20T01:28:15.852 INFO:journalctl@ceph.mgr.x.smithi138.stdout:Aug 20 01:28:15 smithi138 podman[96787]: 2024-08-20 01:28:15.578116895 +0000 UTC m=+0.900977766 container remove 1b4a09a97cbdac266f81f60eb993ed66b68c8d919a98c7cbe3086ca615744d3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:ef4275410d8483698016fe484b2a3c664080b4c1, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-x, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-08-20T01:28:15.876 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.x.service' 2024-08-20T01:28:15.958 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:15.958 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-08-20T01:28:15.958 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-08-20T01:28:15.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.y 2024-08-20T01:28:16.243 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 01:28:16 smithi142 systemd[1]: Stopping Ceph mgr.y for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:16.636 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 01:28:16 smithi142 podman[59524]: 2024-08-20 01:28:16.327961296 +0000 UTC m=+0.179352653 container died 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-08-20T01:28:16.993 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 01:28:16 smithi142 podman[59524]: 2024-08-20 01:28:16.732542245 +0000 UTC m=+0.583933591 container cleanup 56c9043d2154436687660166ae6fbf06449f164df3442a1d7e39adebd7c4b69c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.36.0) 2024-08-20T01:28:16.993 INFO:journalctl@ceph.mgr.y.smithi142.stdout:Aug 20 01:28:16 smithi142 bash[59524]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-y 2024-08-20T01:28:17.175 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.y.service' 2024-08-20T01:28:17.212 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:17.212 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-08-20T01:28:17.212 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-08-20T01:28:17.213 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z 2024-08-20T01:28:17.551 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:17 smithi167 systemd[1]: Stopping Ceph mgr.z for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:17.871 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:17 smithi167 podman[60628]: 2024-08-20 01:28:17.550796176 +0000 UTC m=+0.145003597 container died e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T01:28:18.213 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:17 smithi167 podman[60628]: 2024-08-20 01:28:17.970674267 +0000 UTC m=+0.564881679 container cleanup e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-08-20T01:28:18.214 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:17 smithi167 bash[60628]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z 2024-08-20T01:28:18.457 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z.service' 2024-08-20T01:28:18.465 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:18 smithi167 podman[60640]: 2024-08-20 01:28:18.21378489 +0000 UTC m=+0.660039965 container remove e8812edead2764daf284dae75c871b3e1d97aca6f8d903fd9d2b80f47ae32a71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-08-20T01:28:18.465 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:18 smithi167 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-08-20T01:28:18.465 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:18 smithi167 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-08-20T01:28:18.465 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:18 smithi167 systemd[1]: Stopped Ceph mgr.z for d876cd00-5e8e-11ef-bccf-c7b262605968. 2024-08-20T01:28:18.465 INFO:journalctl@ceph.mgr.z.smithi167.stdout:Aug 20 01:28:18 smithi167 systemd[1]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mgr.z.service: Consumed 7.577s CPU time. 2024-08-20T01:28:18.491 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:18.492 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-08-20T01:28:18.492 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-08-20T01:28:18.492 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.0 2024-08-20T01:28:18.961 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:18 smithi138 systemd[1]: Stopping Ceph osd.0 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:19.293 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T01:28:18.959+0000 7ff444000640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:19.293 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T01:28:18.959+0000 7ff444000640 -1 osd.0 119 *** Got signal Terminated *** 2024-08-20T01:28:19.293 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0[36663]: 2024-08-20T01:28:18.959+0000 7ff444000640 -1 osd.0 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:28:24.791 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:24 smithi138 podman[96902]: 2024-08-20 01:28:24.483909873 +0000 UTC m=+5.706329878 container died 368db4e0282275e86fee8828396a55f21e4bed75af064926091ccdae13c70e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-08-20T01:28:25.543 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:25 smithi138 podman[96902]: 2024-08-20 01:28:25.06546314 +0000 UTC m=+6.287883149 container cleanup 368db4e0282275e86fee8828396a55f21e4bed75af064926091ccdae13c70e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:28:25.543 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:25 smithi138 bash[96902]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0 2024-08-20T01:28:25.543 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:25 smithi138 podman[96921]: 2024-08-20 01:28:25.292450625 +0000 UTC m=+0.796862989 container remove 368db4e0282275e86fee8828396a55f21e4bed75af064926091ccdae13c70e4b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-20T01:28:26.217 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:25 smithi138 podman[97015]: 2024-08-20 01:28:25.733517458 +0000 UTC m=+0.026989337 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:28:26.217 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:25 smithi138 podman[97015]: 2024-08-20 01:28:25.884826571 +0000 UTC m=+0.178298448 container create 3de1c4b49f1bdceaa318c33d592d023b05c89a6fbc677d26ac8011622bf67afb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-08-20T01:28:26.217 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:26 smithi138 podman[97015]: 2024-08-20 01:28:26.216883787 +0000 UTC m=+0.510355664 container init 3de1c4b49f1bdceaa318c33d592d023b05c89a6fbc677d26ac8011622bf67afb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-08-20T01:28:26.487 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:26 smithi138 podman[97015]: 2024-08-20 01:28:26.223722533 +0000 UTC m=+0.517194402 container start 3de1c4b49f1bdceaa318c33d592d023b05c89a6fbc677d26ac8011622bf67afb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T01:28:26.487 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:26 smithi138 podman[97015]: 2024-08-20 01:28:26.329642714 +0000 UTC m=+0.623114591 container attach 3de1c4b49f1bdceaa318c33d592d023b05c89a6fbc677d26ac8011622bf67afb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:26.792 INFO:journalctl@ceph.osd.0.smithi138.stdout:Aug 20 01:28:26 smithi138 podman[97037]: 2024-08-20 01:28:26.529337545 +0000 UTC m=+0.026026178 container died 3de1c4b49f1bdceaa318c33d592d023b05c89a6fbc677d26ac8011622bf67afb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, ceph=True) 2024-08-20T01:28:27.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.0.service' 2024-08-20T01:28:27.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:27.358 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-08-20T01:28:27.359 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-08-20T01:28:27.359 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.1 2024-08-20T01:28:27.671 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:27 smithi142 systemd[1]: Stopping Ceph osd.1 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:27.993 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:28:27.670+0000 7f2d1f99e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:27.993 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:28:27.670+0000 7f2d1f99e640 -1 osd.1 119 *** Got signal Terminated *** 2024-08-20T01:28:27.993 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1[27577]: 2024-08-20T01:28:27.670+0000 7f2d1f99e640 -1 osd.1 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:28:33.571 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:33 smithi142 podman[59658]: 2024-08-20 01:28:33.262667398 +0000 UTC m=+5.712722585 container died b972917eb59de2f3fc992944f9a10a815c06f3d003b53df93d549d4a85d5f8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:28:33.912 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:33 smithi142 podman[59658]: 2024-08-20 01:28:33.660845416 +0000 UTC m=+6.110900602 container cleanup b972917eb59de2f3fc992944f9a10a815c06f3d003b53df93d549d4a85d5f8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:28:33.912 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:33 smithi142 bash[59658]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1 2024-08-20T01:28:34.219 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:33 smithi142 podman[59673]: 2024-08-20 01:28:33.912092179 +0000 UTC m=+0.651009493 container remove b972917eb59de2f3fc992944f9a10a815c06f3d003b53df93d549d4a85d5f8cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:28:34.482 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.122283814 +0000 UTC m=+0.015970509 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:28:34.482 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.229761586 +0000 UTC m=+0.123471159 container create 81f30d84605dde97a606a776b54c2383d02e9ab32040509684a50a87428b196f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-20T01:28:34.743 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.482293049 +0000 UTC m=+0.375979743 container init 81f30d84605dde97a606a776b54c2383d02e9ab32040509684a50a87428b196f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-08-20T01:28:34.744 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.485664329 +0000 UTC m=+0.379351026 container start 81f30d84605dde97a606a776b54c2383d02e9ab32040509684a50a87428b196f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.36.0) 2024-08-20T01:28:34.744 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.615928971 +0000 UTC m=+0.509615667 container attach 81f30d84605dde97a606a776b54c2383d02e9ab32040509684a50a87428b196f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1-deactivate, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:34.744 INFO:journalctl@ceph.osd.1.smithi142.stdout:Aug 20 01:28:34 smithi142 podman[59762]: 2024-08-20 01:28:34.724434521 +0000 UTC m=+0.618121217 container died 81f30d84605dde97a606a776b54c2383d02e9ab32040509684a50a87428b196f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-1-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2) 2024-08-20T01:28:35.345 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.1.service' 2024-08-20T01:28:35.385 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:35.385 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-08-20T01:28:35.385 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-08-20T01:28:35.385 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.2 2024-08-20T01:28:35.715 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:35 smithi167 systemd[1]: Stopping Ceph osd.2 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:36.215 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:28:35.791+0000 7ff39898c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:36.215 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:28:35.791+0000 7ff39898c640 -1 osd.2 119 *** Got signal Terminated *** 2024-08-20T01:28:36.215 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2[28109]: 2024-08-20T01:28:35.791+0000 7ff39898c640 -1 osd.2 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:28:41.715 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:41 smithi167 podman[60764]: 2024-08-20 01:28:41.411910871 +0000 UTC m=+5.831034482 container died f724e22e2393a471ada3903a687cc2cce79d1185deea70432a0229b1fe8bbb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=) 2024-08-20T01:28:42.077 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:41 smithi167 podman[60764]: 2024-08-20 01:28:41.817008055 +0000 UTC m=+6.236131668 container cleanup f724e22e2393a471ada3903a687cc2cce79d1185deea70432a0229b1fe8bbb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T01:28:42.077 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:41 smithi167 bash[60764]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2 2024-08-20T01:28:42.383 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60781]: 2024-08-20 01:28:42.077092848 +0000 UTC m=+0.661674423 container remove f724e22e2393a471ada3903a687cc2cce79d1185deea70432a0229b1fe8bbb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-08-20T01:28:42.706 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.286381378 +0000 UTC m=+0.016078720 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:28:42.706 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.419703817 +0000 UTC m=+0.149401204 container create 742519b7ff7068f61d7247a80556831f24f38a4927eaec398d6fb9e76c2b4af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-20T01:28:42.965 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.706053194 +0000 UTC m=+0.435750537 container init 742519b7ff7068f61d7247a80556831f24f38a4927eaec398d6fb9e76c2b4af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-08-20T01:28:42.965 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.709661456 +0000 UTC m=+0.439358805 container start 742519b7ff7068f61d7247a80556831f24f38a4927eaec398d6fb9e76c2b4af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T01:28:42.966 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.806190877 +0000 UTC m=+0.535888219 container attach 742519b7ff7068f61d7247a80556831f24f38a4927eaec398d6fb9e76c2b4af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:28:42.966 INFO:journalctl@ceph.osd.2.smithi167.stdout:Aug 20 01:28:42 smithi167 podman[60869]: 2024-08-20 01:28:42.91497109 +0000 UTC m=+0.644668448 container died 742519b7ff7068f61d7247a80556831f24f38a4927eaec398d6fb9e76c2b4af1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:43.736 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.2.service' 2024-08-20T01:28:43.779 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:43.779 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-08-20T01:28:43.779 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-08-20T01:28:43.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.3 2024-08-20T01:28:44.248 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:43 smithi138 systemd[1]: Stopping Ceph osd.3 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:44.543 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:44 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:44.247+0000 7f1234600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:44.543 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:44 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:44.247+0000 7f1234600640 -1 osd.3 119 *** Got signal Terminated *** 2024-08-20T01:28:44.543 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:44 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:44.247+0000 7f1234600640 -1 osd.3 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:28:44.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:44.637+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:45.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:45.609+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:45.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:45 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:45.499+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:46.293 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:45 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:45.897+0000 7f1230000640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.791667+0000 front 2024-08-20T01:28:22.791494+0000 (oldest deadline 2024-08-20T01:28:45.691665+0000) 2024-08-20T01:28:46.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:46.122+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:46.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:46 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:46.487+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:46.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:46.643+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:47.293 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:46 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:46.909+0000 7f1230000640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.791667+0000 front 2024-08-20T01:28:22.791494+0000 (oldest deadline 2024-08-20T01:28:45.691665+0000) 2024-08-20T01:28:47.293 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:47 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:47.157+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:47.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:47.091+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:47.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:47 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:47.443+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:47.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:47.671+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:48.293 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:48 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:48.154+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:48.293 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:47 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:47.932+0000 7f1230000640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.791667+0000 front 2024-08-20T01:28:22.791494+0000 (oldest deadline 2024-08-20T01:28:45.691665+0000) 2024-08-20T01:28:48.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:48.078+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:48.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:48 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:48.442+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:48.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:48.558+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:48.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:48 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:48.718+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:49.293 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:48 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3[43363]: 2024-08-20T01:28:48.953+0000 7f1230000640 -1 osd.3 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.791667+0000 front 2024-08-20T01:28:22.791494+0000 (oldest deadline 2024-08-20T01:28:45.691665+0000) 2024-08-20T01:28:49.293 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:49 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:49.129+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:49.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:49.062+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:49.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:49.065+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:49.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:49 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:49.460+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:49.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:49.570+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:49.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:49 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:49.692+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:50.101 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:49 smithi138 podman[97116]: 2024-08-20 01:28:49.843102175 +0000 UTC m=+5.776899448 container died 7c1d6b704a3d020233fdbebd67e3f2e0644a060902eca440e39e7e5f6832e551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:28:50.416 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:50 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:50.098+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:50.416 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:50 smithi138 podman[97116]: 2024-08-20 01:28:50.302835297 +0000 UTC m=+6.236632558 container cleanup 7c1d6b704a3d020233fdbebd67e3f2e0644a060902eca440e39e7e5f6832e551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-08-20T01:28:50.417 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:50 smithi138 bash[97116]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3 2024-08-20T01:28:50.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:50.099+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:50.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:50.027+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:50.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:50 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:50.414+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:50.793 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:50 smithi138 podman[97131]: 2024-08-20 01:28:50.520809787 +0000 UTC m=+0.667497653 container remove 7c1d6b704a3d020233fdbebd67e3f2e0644a060902eca440e39e7e5f6832e551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:50.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:50.615+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:50.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:50 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:50.697+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:51.387 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:50.963677056 +0000 UTC m=+0.026895245 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:28:51.388 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:51.138592945 +0000 UTC m=+0.201811108 container create 1a63c3f15cd45a713e858b3e45d8b79ff033f5b4e8a53561080136d52d8a0d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-08-20T01:28:51.388 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:51 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:51.147+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:51.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:51.095+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:51.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:50.993+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:51.690 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:51 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:51.386+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:51.690 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:51.424559633 +0000 UTC m=+0.487777812 container init 1a63c3f15cd45a713e858b3e45d8b79ff033f5b4e8a53561080136d52d8a0d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-08-20T01:28:51.690 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:51.431262711 +0000 UTC m=+0.494480878 container start 1a63c3f15cd45a713e858b3e45d8b79ff033f5b4e8a53561080136d52d8a0d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:51.690 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:51.574791433 +0000 UTC m=+0.638009619 container attach 1a63c3f15cd45a713e858b3e45d8b79ff033f5b4e8a53561080136d52d8a0d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-08-20T01:28:51.946 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:51.660+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:51.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:51 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:51.683+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:52.043 INFO:journalctl@ceph.osd.3.smithi138.stdout:Aug 20 01:28:51 smithi138 podman[97216]: 2024-08-20 01:28:51.689819733 +0000 UTC m=+0.753037898 container died 1a63c3f15cd45a713e858b3e45d8b79ff033f5b4e8a53561080136d52d8a0d5d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-08-20T01:28:52.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:52.077+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:52.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:51.946+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:52.356 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:52 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:52.164+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:52.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.3.service' 2024-08-20T01:28:52.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:28:52.565 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-08-20T01:28:52.565 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-08-20T01:28:52.566 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.4 2024-08-20T01:28:52.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:52 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:52.354+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:52.930 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:52.651+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:52.993 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:52 smithi142 systemd[1]: Stopping Ceph osd.4 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:28:52.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:52 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:52.671+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:53.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:53.092+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:53.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:52.930+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:53.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:53 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:53.062+0000 7ff1b32b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:28:53.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:53 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:53.062+0000 7ff1b32b8640 -1 osd.4 119 *** Got signal Terminated *** 2024-08-20T01:28:53.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:53 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:53.062+0000 7ff1b32b8640 -1 osd.4 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:28:53.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:53 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:53.400+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:53.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:53 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:53.184+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:53.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:53.701+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:53.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:53.905+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:53.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:53 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:53.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:54.424 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:54.115+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:54.433 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:54 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:54.156+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:54.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:54 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:54.432+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:54.980 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:54.719+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:54.981 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:54.907+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:54.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:54 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:54.631+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:54.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:54 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:54.605+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:55.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:55.164+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:55.465 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:55 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:55.114+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:55.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:55 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:55.183+0000 7ff1af8cd640 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.249713+0000 front 2024-08-20T01:28:29.249688+0000 (oldest deadline 2024-08-20T01:28:55.050457+0000) 2024-08-20T01:28:55.746 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:55.669+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:55.792 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:55 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:55.464+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:55.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:55 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:55.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:55.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:55 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:55.628+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:56.205 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:55.932+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:56.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:56 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:56.204+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:56.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:56 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:56.194+0000 7ff1af8cd640 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.249713+0000 front 2024-08-20T01:28:29.249688+0000 (oldest deadline 2024-08-20T01:28:55.050457+0000) 2024-08-20T01:28:56.512 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:56 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:56.067+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:56.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:56 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:56.511+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:56.942 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:56 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:56.700+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:56.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:56 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:56.585+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:56.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:56 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:56.620+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:57.194 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:56 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:56.942+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:57.194 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:56 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:56.942+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:28:57.293 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:57.033+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:57.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:57.193+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:57.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4[34049]: 2024-08-20T01:28:57.200+0000 7ff1af8cd640 -1 osd.4 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.249713+0000 front 2024-08-20T01:28:29.249688+0000 (oldest deadline 2024-08-20T01:28:55.050457+0000) 2024-08-20T01:28:57.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:57 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:57.481+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:57.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:57.659+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:57.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:57.903+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:57.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:57 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:57.903+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:28:57.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:57.582+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:57.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:57.582+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:28:57.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:57 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:57.644+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:58.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:58.191+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:58.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:58.191+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:28:58.475 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:58.066+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:58.476 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:58.066+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:28:58.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:58.474+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:58.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:58.474+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:28:58.848 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:58 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:58.662+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:58.848 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:58 smithi142 podman[59864]: 2024-08-20 01:28:58.533868531 +0000 UTC m=+5.775796179 container died 258bb83c72d37b0d5a818818f21806feb70b01ecff6ab8513aa4a9ddbace99f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:28:58.848 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:58 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:58.541+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:58.848 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:58 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:58.541+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:28:58.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:58.633+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:58.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:58.854+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:58.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:58 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:58.854+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:28:59.174 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:58 smithi142 podman[59864]: 2024-08-20 01:28:58.964025565 +0000 UTC m=+6.205953209 container cleanup 258bb83c72d37b0d5a818818f21806feb70b01ecff6ab8513aa4a9ddbace99f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux ) 2024-08-20T01:28:59.174 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:58 smithi142 bash[59864]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4 2024-08-20T01:28:59.456 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:59 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:59.098+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:28:59.457 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:28:59 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:28:59.098+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:28:59.493 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59878]: 2024-08-20 01:28:59.173897793 +0000 UTC m=+0.636281882 container remove 258bb83c72d37b0d5a818818f21806feb70b01ecff6ab8513aa4a9ddbace99f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, ceph=True) 2024-08-20T01:28:59.650 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:59.222+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:28:59.650 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:28:59.222+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:28:59.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:59 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:59.455+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:28:59.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:28:59 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:28:59.455+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:28:59.822 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:28:59 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:28:59.635+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:28:59.822 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:59 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:59.568+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:28:59.822 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:28:59 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:28:59.568+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:28:59.823 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59967]: 2024-08-20 01:28:59.480655769 +0000 UTC m=+0.015508731 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:28:59.823 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59967]: 2024-08-20 01:28:59.582943159 +0000 UTC m=+0.117796151 container create a210ca957406e4344b5bae676036b6b5ac20691214f791b5aecc561f84d974e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-08-20T01:28:59.823 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59967]: 2024-08-20 01:28:59.81891845 +0000 UTC m=+0.353771404 container init a210ca957406e4344b5bae676036b6b5ac20691214f791b5aecc561f84d974e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-20T01:28:59.823 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59967]: 2024-08-20 01:28:59.822333288 +0000 UTC m=+0.357186238 container start a210ca957406e4344b5bae676036b6b5ac20691214f791b5aecc561f84d974e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0) 2024-08-20T01:28:59.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:59.649+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:28:59.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:28:59.649+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:28:59.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:59.821+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:28:59.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:28:59 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:28:59.821+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:00.244 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:28:59 smithi142 podman[59967]: 2024-08-20 01:28:59.927114138 +0000 UTC m=+0.461967104 container attach a210ca957406e4344b5bae676036b6b5ac20691214f791b5aecc561f84d974e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:00.244 INFO:journalctl@ceph.osd.4.smithi142.stdout:Aug 20 01:29:00 smithi142 podman[59967]: 2024-08-20 01:29:00.014114713 +0000 UTC m=+0.548967668 container died a210ca957406e4344b5bae676036b6b5ac20691214f791b5aecc561f84d974e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:00.473 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:00.111+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:00.473 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:00.111+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:00.618 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:00.234+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:00.618 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:00.234+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:00.681 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:00 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:00.602+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:00.681 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:00 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:00.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:00.681 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:00 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:00.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:00.705 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.4.service' 2024-08-20T01:29:00.746 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:00.746 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-08-20T01:29:00.746 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-08-20T01:29:00.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.5 2024-08-20T01:29:00.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:00.472+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:00.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:00 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:00.472+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:00.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:00.618+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:00.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:00.618+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:00.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:00 smithi167 systemd[1]: Stopping Ceph osd.5 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:00.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:00.827+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:00.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:00 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:00.827+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:01.439 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:01 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:01.067+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:01.439 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:01 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:01.067+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:01.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:01.240+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:01.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:01.240+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:01.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:01.081+0000 7f1abbd54640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:01.465 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:01.081+0000 7f1abbd54640 -1 osd.5 119 *** Got signal Terminated *** 2024-08-20T01:29:01.466 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:01.081+0000 7f1abbd54640 -1 osd.5 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:01.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:01 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:01.438+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:01.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:01 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:01.438+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:01.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:01.649+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:01.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:01.649+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:01.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:01.848+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:01.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:01 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:01.848+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:01.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:01 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:01.589+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:01.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:01 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:01.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:01.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:01 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:01.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:02.430 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:02 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:02.073+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:02.431 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:02 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:02.073+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:02.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:02.196+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:02.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:02.196+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:02.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:02 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:02.429+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:02.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:02 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:02.429+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:02.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:02.689+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:02.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:02.689+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:02.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:02.820+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:02.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:02.820+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:02.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:02 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:02.820+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:02.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:02 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:02.550+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:02.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:02 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:02.575+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:02.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:02 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:02.575+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:02.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:02 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:02.575+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:03.396 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:03 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:03.114+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:03.396 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:03 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:03.114+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:03.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:03.166+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:03.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:03.166+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:03.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:03 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:03.395+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:03.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:03 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:03.395+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:03.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:03.696+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:03.965 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:03.696+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:03.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:03.696+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.333847+0000 front 2024-08-20T01:28:39.333874+0000 (oldest deadline 2024-08-20T01:29:03.433965+0000) 2024-08-20T01:29:03.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:03.846+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:03.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:03.846+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:03.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:03 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:03.846+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:03.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:03.596+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:03.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:03.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:03.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:03.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:03.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:03 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:03.600+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:04.381 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:04.125+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:04.381 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:04.125+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:04.381 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:04.125+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:04.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:04.202+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:04.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:04.202+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:04.792 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:04.380+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:04.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:04.380+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:04.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:04.862+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:04.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:04.862+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:04.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:04.862+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:04.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:04.655+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:04.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:04.655+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:04.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:04 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:04.655+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.333847+0000 front 2024-08-20T01:28:39.333874+0000 (oldest deadline 2024-08-20T01:29:03.433965+0000) 2024-08-20T01:29:04.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:04.638+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:04.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:04.638+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:04.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:04.572+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:04.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:04.572+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:04.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:04 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:04.572+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:05.391 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:05.100+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:05.392 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:05.100+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:05.392 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:05.100+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:05.695 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:05.243+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:05.695 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:05.243+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:05.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:05.390+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:05.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:05.390+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:05.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:05 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:05.390+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:05.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:05.885+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:05.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:05.885+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:05.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:05.885+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:05.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:05.695+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.430763+0000 front 2024-08-20T01:28:19.430720+0000 (oldest deadline 2024-08-20T01:28:47.631382+0000) 2024-08-20T01:29:05.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:05.695+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.632070+0000 front 2024-08-20T01:28:27.632084+0000 (oldest deadline 2024-08-20T01:28:58.832858+0000) 2024-08-20T01:29:05.966 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:05 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5[34842]: 2024-08-20T01:29:05.695+0000 7f1ab8369640 -1 osd.5 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.333847+0000 front 2024-08-20T01:28:39.333874+0000 (oldest deadline 2024-08-20T01:29:03.433965+0000) 2024-08-20T01:29:05.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:05.644+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:05.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:05.644+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:05.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:05.576+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:05.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:05.576+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:05.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:05 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:05.576+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:06.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:06 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:06.274+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:06.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:06 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:06.274+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:06.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:06.356+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:06.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:06.356+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:06.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:06.356+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:06.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:06.118+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:06.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:06.118+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:06.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:06 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:06.118+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:06.904 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:06 smithi167 podman[60998]: 2024-08-20 01:29:06.590824728 +0000 UTC m=+5.642192635 container died 38d2f5814b3779589f416cd24f4a85e40b5a8666d9c6696487ccc5cc35ff8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-08-20T01:29:06.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:06 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:06.681+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:06.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:06 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:06.681+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:06.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:06 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:06.571+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:06.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:06 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:06.571+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:06.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:06 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:06.571+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:07.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:06 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:06.903+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:07.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:06 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:06.903+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:07.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:06 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:06.903+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:07.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[60998]: 2024-08-20 01:29:07.005707774 +0000 UTC m=+6.057075671 container cleanup 38d2f5814b3779589f416cd24f4a85e40b5a8666d9c6696487ccc5cc35ff8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-08-20T01:29:07.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 bash[60998]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5 2024-08-20T01:29:07.363 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:07.111+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:07.364 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:07.111+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:07.364 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:07.111+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:07.652 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[61014]: 2024-08-20 01:29:07.240500394 +0000 UTC m=+0.644810609 container remove 38d2f5814b3779589f416cd24f4a85e40b5a8666d9c6696487ccc5cc35ff8f2c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-20T01:29:07.652 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:07.245+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:07.652 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:07.245+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:07.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:07.362+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:07.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:07.362+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:07.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:07 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:07.362+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:07.934 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[61104]: 2024-08-20 01:29:07.554374241 +0000 UTC m=+0.015743458 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:07.934 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[61104]: 2024-08-20 01:29:07.674810895 +0000 UTC m=+0.136180155 container create 6361feb5a4e267a15e06c680ecd355e7b6e2f001d8957e0519abde7fb8c31426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-08-20T01:29:07.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:07 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:07.636+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:07.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:07 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:07.636+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:07.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:07 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:07.596+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:07.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:07 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:07.596+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:07.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:07 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:07.596+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:08.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:07.934+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:08.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:07.934+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:08.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:07 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:07.934+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:08.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[61104]: 2024-08-20 01:29:07.957698256 +0000 UTC m=+0.419067474 container init 6361feb5a4e267a15e06c680ecd355e7b6e2f001d8957e0519abde7fb8c31426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-08-20T01:29:08.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:07 smithi167 podman[61104]: 2024-08-20 01:29:07.961116268 +0000 UTC m=+0.422485488 container start 6361feb5a4e267a15e06c680ecd355e7b6e2f001d8957e0519abde7fb8c31426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-08-20T01:29:08.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:08 smithi167 podman[61104]: 2024-08-20 01:29:08.061151931 +0000 UTC m=+0.522521149 container attach 6361feb5a4e267a15e06c680ecd355e7b6e2f001d8957e0519abde7fb8c31426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5-deactivate, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:08.216 INFO:journalctl@ceph.osd.5.smithi167.stdout:Aug 20 01:29:08 smithi167 podman[61104]: 2024-08-20 01:29:08.153780426 +0000 UTC m=+0.615149646 container died 6361feb5a4e267a15e06c680ecd355e7b6e2f001d8957e0519abde7fb8c31426 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:08.369 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:08.110+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:08.369 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:08.110+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:08.369 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:08.110+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:08.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:08 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:08.284+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:08.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:08 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:08.284+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:08.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:08.368+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:08.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:08.368+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:08.793 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:08.368+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:08.801 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.5.service' 2024-08-20T01:29:08.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:08.840 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-08-20T01:29:08.840 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-08-20T01:29:08.841 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.6 2024-08-20T01:29:08.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:08 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:08.647+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:08.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:08 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:08.647+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:08.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:08 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:08.569+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:08.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:08 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:08.569+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:08.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:08 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:08.569+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:09.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:08 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:08.924+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:09.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:08 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:08.924+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:09.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:08 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:08.924+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:09.260 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:08 smithi138 systemd[1]: Stopping Ceph osd.6 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:09.260 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.141+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:09.260 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.141+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:09.260 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.141+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:09.261 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.259+0000 7f84d6a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:09.261 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.259+0000 7f84d6a00640 -1 osd.6 119 *** Got signal Terminated *** 2024-08-20T01:29:09.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:09.259+0000 7f84d6a00640 -1 osd.6 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:09.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:09.336+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:09.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:09.336+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:09.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:09 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:09.336+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:09.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:09.271+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:09.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:09.271+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:09.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:09 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:09.659+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:09.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:09 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:09.659+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:09.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:09 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:09.601+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:09.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:09 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:09.601+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:09.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:09 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:09.601+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:10.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:09.938+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:10.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:09.938+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:10.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:09 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:09.938+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:10.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:10.289+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:10.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:10.289+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:10.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:10.289+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:10.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:10.140+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:10.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:10.140+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:10.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:10.140+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:10.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:10 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:10.283+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:10.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:10 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:10.283+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:10.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:10 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:10.682+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:10.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:10 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:10.682+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:10.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:10 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:10.648+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:10.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:10 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:10.648+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:10.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:10 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:10.648+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:11.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:10 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:10.939+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:10 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:10.939+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:11.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:10 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:10.939+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:11.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:11.269+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:11.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:11.269+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:11.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:11.269+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:11.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:11.161+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:11.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:11.161+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:11.544 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:11 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:11.161+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:11.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:11 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:11.279+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:11.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:11 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:11.279+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:11.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:11.707+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:11.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:11.707+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:11.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:11.707+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:11.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:11.625+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:11.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:11.625+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:11.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:11 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:11.625+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:12.325 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:11 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:11.970+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:12.325 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:11 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:11.970+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:12.325 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:11 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:11.970+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:12.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:12.147+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:12.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:12.147+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:12.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:12.147+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:12.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:12.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:12.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:12.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:12.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:12 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:12.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:12.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:12 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:12.324+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:12.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:12 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:12.324+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:12.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:12.713+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:12.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:12.713+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:12.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:12.713+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:12.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:12.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:12.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:12.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:12.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:12 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:12.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:13.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:12 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:12.922+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:13.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:12 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:12.922+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:13.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:12 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:12.922+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:13.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:13.148+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:13.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:13.148+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:13.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:13.148+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:13.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:13.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:13.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:13.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:13.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:13 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:13.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:13.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:13.343+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:13.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:13.343+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:13.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:13.343+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:13.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:13.748+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:13.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:13.748+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:13.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:13.748+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:13.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:13.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:13.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:13.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:13.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:13 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:13.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:14.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:13.936+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:14.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:13.936+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:14.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:13 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:13.936+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:14.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:14.160+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.717249+0000 front 2024-08-20T01:28:21.717213+0000 (oldest deadline 2024-08-20T01:28:46.417343+0000) 2024-08-20T01:29:14.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:14.160+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:32.319046+0000 front 2024-08-20T01:28:32.319010+0000 (oldest deadline 2024-08-20T01:28:57.619278+0000) 2024-08-20T01:29:14.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6[50126]: 2024-08-20T01:29:14.160+0000 7f84d2400640 -1 osd.6 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.121136+0000 front 2024-08-20T01:28:38.121102+0000 (oldest deadline 2024-08-20T01:29:04.021231+0000) 2024-08-20T01:29:14.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:14.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:14.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:14.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:14.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:14 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:14.253+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:14.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:14.361+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:14.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:14.361+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:14.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:14.361+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:14.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:14.776+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:14.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:14.776+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:14.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:14.776+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:14.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:14.663+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:14.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:14.663+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:14.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:14.663+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:14.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:14 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:14.663+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:15.043 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:14 smithi138 podman[97317]: 2024-08-20 01:29:14.774846809 +0000 UTC m=+5.647533633 container died 844633c85995aed958d4f477fed47a70d69942d416c36c0b685c52021af8ea73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:29:15.358 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:14.980+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:15.358 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:14.980+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:15.358 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:14 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:14.980+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:15.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:15 smithi138 podman[97317]: 2024-08-20 01:29:15.298017404 +0000 UTC m=+6.170704217 container cleanup 844633c85995aed958d4f477fed47a70d69942d416c36c0b685c52021af8ea73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:15.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:15 smithi138 bash[97317]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6 2024-08-20T01:29:15.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:15 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:15.249+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:15.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:15 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:15.249+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:15.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:15 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:15.249+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:15.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:15 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:15.249+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:15.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:15 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:15.356+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:15.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:15 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:15.356+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:15.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:15 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:15.356+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:15.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:15.818+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:15.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:15.818+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:15.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:15.818+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:15.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:15.630+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:15.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:15.630+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:15.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:15.630+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:15.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:15.630+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:16.043 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:15 smithi138 podman[97334]: 2024-08-20 01:29:15.608378562 +0000 UTC m=+0.828249968 container remove 844633c85995aed958d4f477fed47a70d69942d416c36c0b685c52021af8ea73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-08-20T01:29:16.379 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.025+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:16.379 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.025+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:16.379 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.025+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:16.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.114460394 +0000 UTC m=+0.027718699 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:16.543 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.292354254 +0000 UTC m=+0.205612554 container create 16d1ab0fd64863194f1454f001219399811edab5ae8c1742cc2059960f09eab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6-deactivate, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:16.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:16 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:16.210+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:16.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:16 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:16.210+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:16.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:16 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:16.210+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:16.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:16 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:16.210+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:16.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:16.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:16.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:16.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:16.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:16.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:16.837 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.57816264 +0000 UTC m=+0.491420957 container init 16d1ab0fd64863194f1454f001219399811edab5ae8c1742cc2059960f09eab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:16.837 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.585001305 +0000 UTC m=+0.498259606 container start 16d1ab0fd64863194f1454f001219399811edab5ae8c1742cc2059960f09eab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:16.837 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.669797288 +0000 UTC m=+0.583055626 container attach 16d1ab0fd64863194f1454f001219399811edab5ae8c1742cc2059960f09eab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-08-20T01:29:16.856 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:16.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:16.857 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:16.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:16.857 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:16.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:16.857 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:16.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:17.215 INFO:journalctl@ceph.osd.6.smithi138.stdout:Aug 20 01:29:16 smithi138 podman[97423]: 2024-08-20 01:29:16.836326005 +0000 UTC m=+0.749584308 container died 16d1ab0fd64863194f1454f001219399811edab5ae8c1742cc2059960f09eab2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:29:17.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:16.856+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:17.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:16.856+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:17.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:16 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:16.856+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:17.384 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:17.384 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:17.384 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:16 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:16.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:17.515 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:17 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:17.221+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:17.516 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:17 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:17.221+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:17.516 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:17 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:17.221+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:17.516 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:17 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:17.221+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:17.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.6.service' 2024-08-20T01:29:17.625 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:17.626 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-08-20T01:29:17.626 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-08-20T01:29:17.626 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.7 2024-08-20T01:29:17.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:17 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:17.382+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:17.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:17 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:17.382+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:17.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:17 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:17.382+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:17.942 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 systemd[1]: Stopping Ceph osd.7 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:17.942 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:17.942 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:17.942 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:17.943 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:17.619+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:17.943 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:17.619+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:17.943 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:17.619+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:17.943 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:17.619+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:18.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.941+0000 7f4f6a9ac640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:18.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.941+0000 7f4f6a9ac640 -1 osd.7 119 *** Got signal Terminated *** 2024-08-20T01:29:18.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:17 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:17.941+0000 7f4f6a9ac640 -1 osd.7 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:18.387 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:18.388 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:18.388 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:18.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:18.173+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:18.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:18.173+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:18.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:18.173+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:18.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:18 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:18.173+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:18.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:18.386+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:18.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:18.386+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:18.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:18.386+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:18.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:18.845+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:18.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:18.845+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:18.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:18.845+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:18.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:18.845+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.760870+0000 front 2024-08-20T01:28:57.760894+0000 (oldest deadline 2024-08-20T01:29:18.761807+0000) 2024-08-20T01:29:18.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:18.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:18.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:18.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:18.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:18.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:18.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:18.634+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:19.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.977+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:19.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.977+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:19.387 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.977+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:19.387 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:18 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:18.977+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:19.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:19 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:19.177+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:19.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:19 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:19.177+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:19.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:19 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:19.177+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:19.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:19 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:19.177+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:19.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:19.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:19.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:19.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:19.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:19.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:19.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:19.385+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:19.843 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:19.589+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:19.844 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:19.589+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:19.844 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:19.589+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:19.844 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:19.589+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:20.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:19.842+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:20.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:19.842+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:20.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:19.842+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:20.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:19 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:19.842+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.760870+0000 front 2024-08-20T01:28:57.760894+0000 (oldest deadline 2024-08-20T01:29:18.761807+0000) 2024-08-20T01:29:20.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:19.965+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:20.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:19.965+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:20.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:19.965+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:20.387 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:19 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:19.965+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:20.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:20 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:20.163+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:20.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:20 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:20.163+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:20.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:20 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:20.163+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:20.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:20 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:20.163+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:20.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:20 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:20.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:20.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:20 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:20.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:20.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:20 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:20.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:20.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:20 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:20.384+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:20.993 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:20.829+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:20.830+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:20.830+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:20.830+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.760870+0000 front 2024-08-20T01:28:57.760894+0000 (oldest deadline 2024-08-20T01:29:18.761807+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:20.585+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:20.585+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:20.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:20.585+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:20.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:20 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:20.585+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:21.336 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:21.008+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:21.337 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:21.008+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:21.337 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:21.008+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:21.337 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:21.008+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:21.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:21 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:21.199+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:21.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:21 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:21.199+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:21.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:21 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:21.199+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:21.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:21 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:21.199+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:21.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:21.335+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:21.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:21.335+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:21.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:21.335+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:21.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:21 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:21.335+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:21.850 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:21.559+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:21.851 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:21.559+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:21.851 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:21.559+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:21.851 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:21.559+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:22.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:21.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:22.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:21.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:22.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:21.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:22.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:21 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:21.849+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.760870+0000 front 2024-08-20T01:28:57.760894+0000 (oldest deadline 2024-08-20T01:29:18.761807+0000) 2024-08-20T01:29:22.345 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.006+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:22.345 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.006+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:22.345 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.006+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:22.346 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.006+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:22.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:22 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:22.248+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:22.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:22 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:22.248+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:22.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:22 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:22.248+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:22.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:22 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:22.248+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:22.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:22.344+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:22.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:22.344+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:22.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:22.344+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:22.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:22.344+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:22.869 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:22.588+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:22.869 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:22.588+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:22.869 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:22.588+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:22.869 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:22.588+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:23.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:22.868+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:29.856245+0000 front 2024-08-20T01:28:29.856275+0000 (oldest deadline 2024-08-20T01:28:54.556404+0000) 2024-08-20T01:29:23.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:22.868+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:38.557955+0000 front 2024-08-20T01:28:38.557883+0000 (oldest deadline 2024-08-20T01:29:03.858026+0000) 2024-08-20T01:29:23.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:22.868+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:47.959053+0000 front 2024-08-20T01:28:47.959129+0000 (oldest deadline 2024-08-20T01:29:10.858987+0000) 2024-08-20T01:29:23.244 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7[40959]: 2024-08-20T01:29:22.868+0000 7f4f62fb9640 -1 osd.7 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.760870+0000 front 2024-08-20T01:28:57.760894+0000 (oldest deadline 2024-08-20T01:29:18.761807+0000) 2024-08-20T01:29:23.395 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:23.396 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:23.396 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:23.396 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:22 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:22.997+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:23.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:23 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:23.230+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:23.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:23 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:23.230+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:23.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:23 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:23.230+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:23.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:23 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:23.230+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:23.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:23 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:23.394+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:23.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:23 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:23.394+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:23.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:23 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:23.394+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:23.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:23 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:23.394+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:23.743 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:23 smithi142 podman[60072]: 2024-08-20 01:29:23.473153889 +0000 UTC m=+5.650758073 container died e138f425d54aec2031c6dce489d770d5fc74f509c94f6ffe03c4e12b97fde1c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T01:29:23.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:23.581+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:23.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:23.581+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:23.744 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:23.581+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:23.744 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:23.581+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:23.744 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:23 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:23.581+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:24.150 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:23 smithi142 podman[60072]: 2024-08-20 01:29:23.874044822 +0000 UTC m=+6.051649006 container cleanup e138f425d54aec2031c6dce489d770d5fc74f509c94f6ffe03c4e12b97fde1c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:24.150 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:23 smithi142 bash[60072]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7 2024-08-20T01:29:24.377 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:24.041+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:24.377 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:24.041+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:24.377 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:24.041+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:24.377 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:24.041+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:24.483 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60087]: 2024-08-20 01:29:24.150227721 +0000 UTC m=+0.672136949 container remove e138f425d54aec2031c6dce489d770d5fc74f509c94f6ffe03c4e12b97fde1c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:24.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:24.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:24.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:24.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:24.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:24.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:24.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:24.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:24.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:24 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:24.237+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:24.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:24.375+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:24.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:24.375+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:24.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:24.375+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:24.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:24 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:24.375+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:24.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:24 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:24.558+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:24.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:24 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:24.558+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:24.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:24 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:24.558+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:24.743 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:24 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:24.558+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:24.744 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:24 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:24.558+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:24.744 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.386202391 +0000 UTC m=+0.015608937 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:24.744 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.484775825 +0000 UTC m=+0.114182374 container create 4b01fde24746895cd3310aff1df11af95e839525de23646b98831c8ecae01d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS) 2024-08-20T01:29:24.744 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.720379002 +0000 UTC m=+0.349785551 container init 4b01fde24746895cd3310aff1df11af95e839525de23646b98831c8ecae01d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7-deactivate, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:24.744 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.723618459 +0000 UTC m=+0.353024999 container start 4b01fde24746895cd3310aff1df11af95e839525de23646b98831c8ecae01d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:25.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.795427049 +0000 UTC m=+0.424833591 container attach 4b01fde24746895cd3310aff1df11af95e839525de23646b98831c8ecae01d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-20T01:29:25.243 INFO:journalctl@ceph.osd.7.smithi142.stdout:Aug 20 01:29:24 smithi142 podman[60175]: 2024-08-20 01:29:24.914682401 +0000 UTC m=+0.544088952 container died 4b01fde24746895cd3310aff1df11af95e839525de23646b98831c8ecae01d47 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-08-20T01:29:25.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:25.031+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:25.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:25.031+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:25.399 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:25.031+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:25.400 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:25.031+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:25.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:25 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:25.206+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:25.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:25 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:25.206+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:25.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:25 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:25.206+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:25.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:25 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:25.206+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:25.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:25 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:25.206+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:25.575 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.7.service' 2024-08-20T01:29:25.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:25.615 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-08-20T01:29:25.616 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-08-20T01:29:25.616 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.8 2024-08-20T01:29:25.660 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.398+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:25.660 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.398+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:25.661 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.398+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:25.661 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.398+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:25.960 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 systemd[1]: Stopping Ceph osd.8 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:25.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:25 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:25.551+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:25.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:25 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:25.551+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:25.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:25 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:25.551+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:25.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:25 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:25.551+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:25.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:25 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:25.551+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:26.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.959+0000 7ff7fb461640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:26.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.959+0000 7ff7fb461640 -1 osd.8 119 *** Got signal Terminated *** 2024-08-20T01:29:26.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:25 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:25.959+0000 7ff7fb461640 -1 osd.8 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:26.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:26.064+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:26.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:26.064+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:26.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:26.064+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:26.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:26.064+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:26.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:26.212+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:26.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:26.212+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:26.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:26.212+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:26.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:26.212+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:26.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:26 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:26.212+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:26.690 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:26.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:26.690 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:26.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:26.690 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:26.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:26.690 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:26.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:26.690 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:26.599+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:26.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:26.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:26.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:26.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:26.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:26.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:26.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:26 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:26.378+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:27.421 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:27.065+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:27.421 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:27.065+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:27.421 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:27.065+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:27.421 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:27.065+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:27.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:27 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:27.257+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:27.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:27 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:27.257+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:27.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:27 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:27.257+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:27.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:27 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:27.257+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:27.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:27 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:27.257+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:27.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:27.420+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:27.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:27.420+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:27.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:27.420+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:27.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:27 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:27.420+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:27.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:27.644+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:27.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:27.644+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:27.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:27.644+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:27.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:27.644+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:27.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:27 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:27.644+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:28.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:28.020+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:28.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:28.020+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:28.386 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:28.020+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:28.387 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:28.020+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:28.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:28 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:28.285+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:28.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:28.385+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:28.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:28.385+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:28.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:28.385+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:28.716 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:28 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:28.385+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:28.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:28 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:28.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:28.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:28 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:28.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:28.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:28 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:28.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:28.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:28 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:28.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:28.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:28 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:28.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:29.404 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:29.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:29.404 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:29.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:29.405 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:29.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:29.405 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:29.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:29.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:29.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:29.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:29.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:29.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:29.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:29 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:29.256+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:29.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:29.403+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:29.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:29.403+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:29.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:29.403+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:29.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:29 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:29.403+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:29.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:29.676+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:29.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:29.676+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:29.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:29.676+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:29.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:29.676+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:29.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:29 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:29.676+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:30.443 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:30.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:30.443 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:30.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:30.444 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:30.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:30.444 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:30.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:30.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:30.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:30.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:30.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:30.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:30.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:30 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:30.254+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:30.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:30.441+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:19.683852+0000 front 2024-08-20T01:28:19.683747+0000 (oldest deadline 2024-08-20T01:28:45.583696+0000) 2024-08-20T01:29:30.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:30.441+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.484765+0000 front 2024-08-20T01:28:31.484689+0000 (oldest deadline 2024-08-20T01:28:57.285267+0000) 2024-08-20T01:29:30.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:30.441+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.288105+0000 front 2024-08-20T01:28:48.287900+0000 (oldest deadline 2024-08-20T01:29:12.988099+0000) 2024-08-20T01:29:30.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:30 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8[41339]: 2024-08-20T01:29:30.441+0000 7ff7f7a76640 -1 osd.8 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:52.988614+0000 front 2024-08-20T01:28:52.988599+0000 (oldest deadline 2024-08-20T01:29:18.888760+0000) 2024-08-20T01:29:30.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:30.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:30.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:30.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:30.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:30.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:30 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:30.650+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:31.455 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:31.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:31.455 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:31.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:31.456 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:31.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:31.456 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:31.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:31.456 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:31.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:31.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:31.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:31.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:31.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:31.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:31.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:31 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:31.229+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:31.715 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:31 smithi167 podman[61210]: 2024-08-20 01:29:31.455247491 +0000 UTC m=+5.639010269 container died 68806bf13213749547bdfcaff40b4f044b5e62fbc1f1a50c711296bac2d038ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:31.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:31.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:31.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:31.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:31.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:31.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:31 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:31.689+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:32.072 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:31 smithi167 podman[61210]: 2024-08-20 01:29:31.876668777 +0000 UTC m=+6.060431555 container cleanup 68806bf13213749547bdfcaff40b4f044b5e62fbc1f1a50c711296bac2d038ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:32.072 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:31 smithi167 bash[61210]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8 2024-08-20T01:29:32.465 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61232]: 2024-08-20 01:29:32.09457384 +0000 UTC m=+0.636914249 container remove 68806bf13213749547bdfcaff40b4f044b5e62fbc1f1a50c711296bac2d038ed (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, ceph=True) 2024-08-20T01:29:32.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:32 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:32.070+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:32.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:32 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:32.070+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:32.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:32 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:32.070+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:32.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:32 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:32.070+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:32.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:32 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:32.070+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:32.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:32.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:32.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:32.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:32.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:32.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:32 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:32.202+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:32.840 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.399172202 +0000 UTC m=+0.015741060 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:32.841 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.503761576 +0000 UTC m=+0.120330425 container create 8ab70b610d2526b4ffe29a3e53f8050d0e325a758985b1624fe8361c9a5d0c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-20T01:29:32.841 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.739663779 +0000 UTC m=+0.356232628 container init 8ab70b610d2526b4ffe29a3e53f8050d0e325a758985b1624fe8361c9a5d0c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:29:32.841 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.742957118 +0000 UTC m=+0.359525968 container start 8ab70b610d2526b4ffe29a3e53f8050d0e325a758985b1624fe8361c9a5d0c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-08-20T01:29:32.841 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.839894491 +0000 UTC m=+0.456463363 container attach 8ab70b610d2526b4ffe29a3e53f8050d0e325a758985b1624fe8361c9a5d0c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-08-20T01:29:32.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:32.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:32.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:32.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:32.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:32.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:32 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:32.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:33.215 INFO:journalctl@ceph.osd.8.smithi167.stdout:Aug 20 01:29:32 smithi167 podman[61320]: 2024-08-20 01:29:32.934774246 +0000 UTC m=+0.551343105 container died 8ab70b610d2526b4ffe29a3e53f8050d0e325a758985b1624fe8361c9a5d0c1f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-08-20T01:29:33.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:33 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:33.067+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:33.215 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:33 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:33.067+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:33.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:33 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:33.067+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:33.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:33 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:33.067+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:33.216 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:33 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:33.067+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:33.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:33.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:33.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:33.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:33.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:33.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:33.217+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:33.608 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.8.service' 2024-08-20T01:29:33.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:33.649 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-08-20T01:29:33.649 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-08-20T01:29:33.649 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.9 2024-08-20T01:29:33.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:33.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:33.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:33.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:33.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:33.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:33 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:33.656+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:34.042 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:33 smithi138 systemd[1]: Stopping Ceph osd.9 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:34.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:34.079+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:34.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:34.079+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:34.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:34.079+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:34.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:34.079+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:34.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:34.079+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:34.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.129+0000 7ff777c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:34.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.129+0000 7ff777c00640 -1 osd.9 119 *** Got signal Terminated *** 2024-08-20T01:29:34.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.129+0000 7ff777c00640 -1 osd.9 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:34.544 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:34 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:34.201+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:34.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:34.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:34.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:34.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:34.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:34.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:34 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:34.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:35.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:35.085+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:35.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:35.085+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:35.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:35.085+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:35.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:35.085+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:35.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:35 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:35.085+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:35.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:35 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:35.240+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:35.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:35.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:35.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:35.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:35.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:35.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:35 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:35.705+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:36.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:36.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:36.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:36.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:36.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:36.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:36 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:36.071+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:36.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:36 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:36.231+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:36.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:36.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:36.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:36.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:36.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:36.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:36.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:36 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:36.708+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:37.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:37.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:37.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:37.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:37.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:37.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:37.056+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:37.542 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:37.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:37.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:37.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:37.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:37.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:37 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:37.238+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:37.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:37.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:37 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:37.740+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:38.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:38.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:38.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:38.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:38.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:38.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:38 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:38.038+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:22.474545+0000 front 2024-08-20T01:28:22.474578+0000 (oldest deadline 2024-08-20T01:28:44.774702+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:31.677643+0000 front 2024-08-20T01:28:31.677612+0000 (oldest deadline 2024-08-20T01:28:57.577821+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:39.879905+0000 front 2024-08-20T01:28:39.879704+0000 (oldest deadline 2024-08-20T01:29:04.579746+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:44.580305+0000 front 2024-08-20T01:28:44.580280+0000 (oldest deadline 2024-08-20T01:29:15.081841+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.983198+0000 front 2024-08-20T01:28:57.983172+0000 (oldest deadline 2024-08-20T01:29:23.283107+0000) 2024-08-20T01:29:38.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:38 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9[56636]: 2024-08-20T01:29:38.250+0000 7ff76e600640 -1 osd.9 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:05.584400+0000 front 2024-08-20T01:29:05.584563+0000 (oldest deadline 2024-08-20T01:29:27.884277+0000) 2024-08-20T01:29:38.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:38.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:38.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:38.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:38.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:38.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:38.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:38 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:38.716+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:39.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:39.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:39.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:39.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:39.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:39.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:39 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:39.075+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:39.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:39.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:39.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:39.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:39.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:39.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:39.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:39 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:39.680+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:40.194 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:39 smithi138 podman[97525]: 2024-08-20 01:29:39.861998431 +0000 UTC m=+5.920765067 container died 4e434b3c058fa0b55b66618a4c8581754f4bdaf555411cbb6af535cb96160e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0) 2024-08-20T01:29:40.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:40.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:40.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:40.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:40.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:40.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:40 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:40.029+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:40.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:40 smithi138 podman[97525]: 2024-08-20 01:29:40.383583473 +0000 UTC m=+6.442350108 container cleanup 4e434b3c058fa0b55b66618a4c8581754f4bdaf555411cbb6af535cb96160e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:29:40.543 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:40 smithi138 bash[97525]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9 2024-08-20T01:29:40.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:40.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:40.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:40.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:40.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:40.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:40.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:40 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:40.649+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:41.043 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:40 smithi138 podman[97543]: 2024-08-20 01:29:40.571866959 +0000 UTC m=+0.699552886 container remove 4e434b3c058fa0b55b66618a4c8581754f4bdaf555411cbb6af535cb96160e5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-20T01:29:41.460 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.041002744 +0000 UTC m=+0.026630020 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:41.460 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.177833195 +0000 UTC m=+0.163460463 container create b2a5bc53b5d5d52d7f0a970d9dd8c1ac8f25e90f47b990716326d18c76c7540c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-20T01:29:41.460 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.460128207 +0000 UTC m=+0.445755487 container init b2a5bc53b5d5d52d7f0a970d9dd8c1ac8f25e90f47b990716326d18c76c7540c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.36.0) 2024-08-20T01:29:41.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:41.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:41.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:41.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:41.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:41.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:41 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:41.007+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:41.721 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.467013993 +0000 UTC m=+0.452641263 container start b2a5bc53b5d5d52d7f0a970d9dd8c1ac8f25e90f47b990716326d18c76c7540c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-08-20T01:29:41.722 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.554968503 +0000 UTC m=+0.540595788 container attach b2a5bc53b5d5d52d7f0a970d9dd8c1ac8f25e90f47b990716326d18c76c7540c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9-deactivate, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:29:41.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:41.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:41.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:41.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:41.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:41.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:41.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:41 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:41.690+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:42.043 INFO:journalctl@ceph.osd.9.smithi138.stdout:Aug 20 01:29:41 smithi138 podman[97630]: 2024-08-20 01:29:41.721271745 +0000 UTC m=+0.706899021 container died b2a5bc53b5d5d52d7f0a970d9dd8c1ac8f25e90f47b990716326d18c76c7540c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:42.464 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.9.service' 2024-08-20T01:29:42.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:42.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:42.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:42.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:42.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:42.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:42.022+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:42.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:42.541 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-08-20T01:29:42.541 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-08-20T01:29:42.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.10 2024-08-20T01:29:42.835 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 systemd[1]: Stopping Ceph osd.10 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:42.835 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:42.835 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.722+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.833+0000 7f5fc6992640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.833+0000 7f5fc6992640 -1 osd.10 119 *** Got signal Terminated *** 2024-08-20T01:29:42.836 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:42 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:42.833+0000 7f5fc6992640 -1 osd.10 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:43.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:43.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:43.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:43.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:43.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:43.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:43 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:43.026+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:43.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:43.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:43.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:43.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:43.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:43.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:43.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:43 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:43.720+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:44.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:44.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:44.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:44.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:44.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:44.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:44 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:44.060+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:44.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:44.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:44.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:44.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:44.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:44.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:44.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:44 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:44.726+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:45.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:45.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:45.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:45.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:45.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:45.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:45 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:45.055+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:45.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:45.995 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:45 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:45.693+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:46.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:46.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:46.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:46.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:46.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:46.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:46 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:46.042+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:46.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:46.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:46.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:46.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:46.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:46.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:46.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:46 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:46.646+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:47.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:47.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:47.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:47.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:47.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:47.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:47 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:47.074+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:21.036926+0000 front 2024-08-20T01:28:21.036885+0000 (oldest deadline 2024-08-20T01:28:44.536630+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:27.937828+0000 front 2024-08-20T01:28:27.937907+0000 (oldest deadline 2024-08-20T01:28:56.738473+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:36.739115+0000 front 2024-08-20T01:28:36.739155+0000 (oldest deadline 2024-08-20T01:29:02.039191+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6814 osd.3 since back 2024-08-20T01:28:48.941177+0000 front 2024-08-20T01:28:48.941224+0000 (oldest deadline 2024-08-20T01:29:13.641261+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.742144+0000 front 2024-08-20T01:28:57.742199+0000 (oldest deadline 2024-08-20T01:29:23.042234+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:04.643748+0000 front 2024-08-20T01:29:04.643601+0000 (oldest deadline 2024-08-20T01:29:29.943461+0000) 2024-08-20T01:29:47.994 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:47 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10[47482]: 2024-08-20T01:29:47.622+0000 7f5fbef9f640 -1 osd.10 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:09.944055+0000 front 2024-08-20T01:29:09.943997+0000 (oldest deadline 2024-08-20T01:29:35.844010+0000) 2024-08-20T01:29:48.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:48.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:48 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:48.037+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:48.652 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:48 smithi142 podman[60288]: 2024-08-20 01:29:48.397319789 +0000 UTC m=+5.668127420 container died d31acba36eb616c87c18fb8bdbf197649f4151ca03de2b8efa34b6975bbef079 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-08-20T01:29:48.935 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:48 smithi142 podman[60288]: 2024-08-20 01:29:48.733958769 +0000 UTC m=+6.004766398 container cleanup d31acba36eb616c87c18fb8bdbf197649f4151ca03de2b8efa34b6975bbef079 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:29:48.935 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:48 smithi142 bash[60288]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10 2024-08-20T01:29:49.243 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:48 smithi142 podman[60303]: 2024-08-20 01:29:48.935030062 +0000 UTC m=+0.536301765 container remove d31acba36eb616c87c18fb8bdbf197649f4151ca03de2b8efa34b6975bbef079 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-08-20T01:29:49.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:49.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:49 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:49.045+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:49.663 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.240622074 +0000 UTC m=+0.015442967 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:49.663 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.344025045 +0000 UTC m=+0.118845927 container create c66260c140f9f44d0a5135fa06606cac04373a5ca5bdfdbb5a89e67a30536a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:29:49.663 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.579553387 +0000 UTC m=+0.354374274 container init c66260c140f9f44d0a5135fa06606cac04373a5ca5bdfdbb5a89e67a30536a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:29:49.663 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.582814341 +0000 UTC m=+0.357635228 container start c66260c140f9f44d0a5135fa06606cac04373a5ca5bdfdbb5a89e67a30536a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-20T01:29:49.663 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.662691455 +0000 UTC m=+0.437512350 container attach c66260c140f9f44d0a5135fa06606cac04373a5ca5bdfdbb5a89e67a30536a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-08-20T01:29:49.993 INFO:journalctl@ceph.osd.10.smithi142.stdout:Aug 20 01:29:49 smithi142 podman[60394]: 2024-08-20 01:29:49.775910859 +0000 UTC m=+0.550731757 container died c66260c140f9f44d0a5135fa06606cac04373a5ca5bdfdbb5a89e67a30536a9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T01:29:50.358 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.10.service' 2024-08-20T01:29:50.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:50.399 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-08-20T01:29:50.399 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-08-20T01:29:50.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.11 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:50.427 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.069+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:50.704 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 systemd[1]: Stopping Ceph osd.11 for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:50.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.704+0000 7ff245bc4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:50.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.704+0000 7ff245bc4640 -1 osd.11 119 *** Got signal Terminated *** 2024-08-20T01:29:50.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:50 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:50.704+0000 7ff245bc4640 -1 osd.11 119 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-08-20T01:29:51.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:51.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:51 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:51.115+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:52.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:52.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:52.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:52.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:52.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:52.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:52.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:52 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:52.114+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:53.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:53.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:53.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:53 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:53.164+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-20T01:29:27.582504+0000 front 2024-08-20T01:29:27.582518+0000 (oldest deadline 2024-08-20T01:29:52.282527+0000) 2024-08-20T01:29:54.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:54.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:54.467 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:54 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:54.121+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-20T01:29:27.582504+0000 front 2024-08-20T01:29:27.582518+0000 (oldest deadline 2024-08-20T01:29:52.282527+0000) 2024-08-20T01:29:55.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6806 osd.0 since back 2024-08-20T01:28:23.274088+0000 front 2024-08-20T01:28:23.273966+0000 (oldest deadline 2024-08-20T01:28:48.573883+0000) 2024-08-20T01:29:55.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6804 osd.1 since back 2024-08-20T01:28:30.875099+0000 front 2024-08-20T01:28:30.874948+0000 (oldest deadline 2024-08-20T01:28:56.774971+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6804 osd.2 since back 2024-08-20T01:28:37.875990+0000 front 2024-08-20T01:28:37.876048+0000 (oldest deadline 2024-08-20T01:29:02.576052+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6812 osd.4 since back 2024-08-20T01:28:57.778747+0000 front 2024-08-20T01:28:57.778763+0000 (oldest deadline 2024-08-20T01:29:18.878780+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6812 osd.5 since back 2024-08-20T01:29:01.779777+0000 front 2024-08-20T01:29:01.779801+0000 (oldest deadline 2024-08-20T01:29:30.580560+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.138:6822 osd.6 since back 2024-08-20T01:29:10.581070+0000 front 2024-08-20T01:29:10.581204+0000 (oldest deadline 2024-08-20T01:29:35.280918+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.142:6820 osd.7 since back 2024-08-20T01:29:22.282332+0000 front 2024-08-20T01:29:22.282381+0000 (oldest deadline 2024-08-20T01:29:47.582150+0000) 2024-08-20T01:29:55.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:55 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11[47892]: 2024-08-20T01:29:55.122+0000 7ff23e1d1640 -1 osd.11 119 heartbeat_check: no reply from 172.21.15.167:6820 osd.8 since back 2024-08-20T01:29:27.582504+0000 front 2024-08-20T01:29:27.582518+0000 (oldest deadline 2024-08-20T01:29:52.282527+0000) 2024-08-20T01:29:56.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:56 smithi167 podman[61423]: 2024-08-20 01:29:56.213502029 +0000 UTC m=+5.616410065 container died 464de6d2d014cddf41096ffe938c2762f925de399fde86df48df14aa436c0887 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-08-20T01:29:56.855 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:56 smithi167 podman[61423]: 2024-08-20 01:29:56.595693397 +0000 UTC m=+5.998601430 container cleanup 464de6d2d014cddf41096ffe938c2762f925de399fde86df48df14aa436c0887 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:29:56.855 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:56 smithi167 bash[61423]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11 2024-08-20T01:29:57.204 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:56 smithi167 podman[61445]: 2024-08-20 01:29:56.855220294 +0000 UTC m=+0.631031176 container remove 464de6d2d014cddf41096ffe938c2762f925de399fde86df48df14aa436c0887 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-08-20T01:29:57.465 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.107321717 +0000 UTC m=+0.015945107 image pull 9f5c88d94bb632e604503b516326cdfcf31eea177420de126ce2ba09454a7f4a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8 2024-08-20T01:29:57.466 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.214340598 +0000 UTC m=+0.122964030 container create 5c27f397d39c700cd95344f22d1ef4031b1aeeec5af74f5641d9039ddbb6210e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-08-20T01:29:57.965 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.495376734 +0000 UTC m=+0.404000126 container init 5c27f397d39c700cd95344f22d1ef4031b1aeeec5af74f5641d9039ddbb6210e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-08-20T01:29:57.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.498639005 +0000 UTC m=+0.407262402 container start 5c27f397d39c700cd95344f22d1ef4031b1aeeec5af74f5641d9039ddbb6210e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-08-20T01:29:57.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.583641336 +0000 UTC m=+0.492264742 container attach 5c27f397d39c700cd95344f22d1ef4031b1aeeec5af74f5641d9039ddbb6210e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-08-20T01:29:57.966 INFO:journalctl@ceph.osd.11.smithi167.stdout:Aug 20 01:29:57 smithi167 podman[61533]: 2024-08-20 01:29:57.690538216 +0000 UTC m=+0.599161615 container died 5c27f397d39c700cd95344f22d1ef4031b1aeeec5af74f5641d9039ddbb6210e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0) 2024-08-20T01:29:58.287 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@osd.11.service' 2024-08-20T01:29:58.364 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:29:58.364 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-08-20T01:29:58.365 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-08-20T01:29:58.365 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.a 2024-08-20T01:29:58.748 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:29:58 smithi138 systemd[1]: Stopping Ceph mds.a for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:29:59.043 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:29:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a[63517]: 2024-08-20T01:29:58.746+0000 7efff5a00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:29:59.043 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:29:58 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a[63517]: 2024-08-20T01:29:58.746+0000 7efff5a00640 -1 mds.a *** got signal Terminated *** 2024-08-20T01:30:03.043 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:30:02 smithi138 podman[97734]: 2024-08-20 01:30:02.766308886 +0000 UTC m=+4.117317108 container died c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-08-20T01:30:03.419 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:30:03 smithi138 podman[97734]: 2024-08-20 01:30:03.192687094 +0000 UTC m=+4.543695315 container cleanup c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0) 2024-08-20T01:30:03.419 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:30:03 smithi138 bash[97734]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a 2024-08-20T01:30:03.676 INFO:journalctl@ceph.mds.a.smithi138.stdout:Aug 20 01:30:03 smithi138 podman[97749]: 2024-08-20 01:30:03.41880709 +0000 UTC m=+0.646161699 container remove c27bb9affd98d35961778e2737eac9b78a78655f0010fdfadc1486d17640b5cf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-08-20T01:30:03.684 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.a.service' 2024-08-20T01:30:03.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:03.756 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-08-20T01:30:03.756 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-08-20T01:30:03.756 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.d 2024-08-20T01:30:04.043 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:03 smithi138 systemd[1]: Stopping Ceph mds.d for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:04.543 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d[62340]: 2024-08-20T01:30:04.141+0000 7f1a44600640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:04.543 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:04 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d[62340]: 2024-08-20T01:30:04.141+0000 7f1a44600640 -1 mds.d *** got signal Terminated *** 2024-08-20T01:30:07.043 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:06 smithi138 podman[97858]: 2024-08-20 01:30:06.752116785 +0000 UTC m=+2.722342667 container died 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-08-20T01:30:07.398 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:07 smithi138 podman[97858]: 2024-08-20 01:30:07.146395622 +0000 UTC m=+3.116621504 container cleanup 3667677061f0849cbb1b96d0e3946c03182e6fc5abf9bc9c9ff265fa0fd506dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.build-date=20240731) 2024-08-20T01:30:07.399 INFO:journalctl@ceph.mds.d.smithi138.stdout:Aug 20 01:30:07 smithi138 bash[97858]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-d 2024-08-20T01:30:07.563 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.d.service' 2024-08-20T01:30:07.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:07.632 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-08-20T01:30:07.633 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-08-20T01:30:07.633 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.g 2024-08-20T01:30:08.012 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:07 smithi138 systemd[1]: Stopping Ceph mds.g for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:08.293 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: 2024-08-20T01:30:08.011+0000 7fe1d6200640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:08.293 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:08 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g[63133]: 2024-08-20T01:30:08.011+0000 7fe1d6200640 -1 mds.g *** got signal Terminated *** 2024-08-20T01:30:09.414 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:09 smithi138 podman[97973]: 2024-08-20 01:30:09.09902601 +0000 UTC m=+1.201228317 container died 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-08-20T01:30:09.793 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:09 smithi138 podman[97973]: 2024-08-20 01:30:09.583657231 +0000 UTC m=+1.685859525 container cleanup 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True) 2024-08-20T01:30:09.793 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:09 smithi138 bash[97973]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g 2024-08-20T01:30:10.099 INFO:journalctl@ceph.mds.g.smithi138.stdout:Aug 20 01:30:09 smithi138 podman[97984]: 2024-08-20 01:30:09.828990613 +0000 UTC m=+0.724010176 container remove 29665e90289f5afafe482d39b38ba9ac279a35fa6d1dacbc0c0740983fd22e56 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-g, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:30:10.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.g.service' 2024-08-20T01:30:10.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:10.186 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-08-20T01:30:10.186 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-08-20T01:30:10.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.j 2024-08-20T01:30:10.543 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:10 smithi138 systemd[1]: Stopping Ceph mds.j for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:11.043 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j[61959]: 2024-08-20T01:30:10.581+0000 7f7280800640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:11.043 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:10 smithi138 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j[61959]: 2024-08-20T01:30:10.581+0000 7f7280800640 -1 mds.j *** got signal Terminated *** 2024-08-20T01:30:14.922 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:14 smithi138 podman[98095]: 2024-08-20 01:30:14.584952991 +0000 UTC m=+4.130654449 container died 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-08-20T01:30:15.206 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:15 smithi138 podman[98095]: 2024-08-20 01:30:15.037017757 +0000 UTC m=+4.582719218 container cleanup 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T01:30:15.206 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:15 smithi138 bash[98095]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j 2024-08-20T01:30:15.471 INFO:journalctl@ceph.mds.j.smithi138.stdout:Aug 20 01:30:15 smithi138 podman[98106]: 2024-08-20 01:30:15.204929616 +0000 UTC m=+0.606985453 container remove 44f7f81bd3ce18f7a32697136aa8d9de21a3541d8447f17ed094b1fe5407aaa2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-08-20T01:30:15.479 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.j.service' 2024-08-20T01:30:15.547 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:15.547 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-08-20T01:30:15.547 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-08-20T01:30:15.547 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.b 2024-08-20T01:30:15.859 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:15 smithi142 systemd[1]: Stopping Ceph mds.b for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:16.114 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: 2024-08-20T01:30:15.858+0000 7f22701df640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:16.114 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:15 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b[52979]: 2024-08-20T01:30:15.858+0000 7f22701df640 -1 mds.b *** got signal Terminated *** 2024-08-20T01:30:17.292 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:17 smithi142 podman[60496]: 2024-08-20 01:30:17.050016009 +0000 UTC m=+1.301809687 container died 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-08-20T01:30:17.679 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:17 smithi142 podman[60496]: 2024-08-20 01:30:17.399358034 +0000 UTC m=+1.651151707 container cleanup 1e605f686df8c3c89a590e65e779207e2a7b3f5185edb32a29db3a95708f5896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , ceph=True) 2024-08-20T01:30:17.679 INFO:journalctl@ceph.mds.b.smithi142.stdout:Aug 20 01:30:17 smithi142 bash[60496]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-b 2024-08-20T01:30:17.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.b.service' 2024-08-20T01:30:17.853 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:17.853 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-08-20T01:30:17.854 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-08-20T01:30:17.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.e 2024-08-20T01:30:18.243 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:17 smithi142 systemd[1]: Stopping Ceph mds.e for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:18.244 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: 2024-08-20T01:30:18.178+0000 7f11a4325640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:18.244 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:18 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e[52379]: 2024-08-20T01:30:18.178+0000 7f11a4325640 -1 mds.e *** got signal Terminated *** 2024-08-20T01:30:21.370 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:21 smithi142 podman[60630]: 2024-08-20 01:30:21.047892895 +0000 UTC m=+2.977003279 container died ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:30:21.370 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:21 smithi142 podman[60630]: 2024-08-20 01:30:21.369470303 +0000 UTC m=+3.298580690 container cleanup ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-08-20T01:30:21.724 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:21 smithi142 bash[60630]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e 2024-08-20T01:30:21.724 INFO:journalctl@ceph.mds.e.smithi142.stdout:Aug 20 01:30:21 smithi142 podman[60642]: 2024-08-20 01:30:21.529018051 +0000 UTC m=+0.477774642 container remove ba3e383ad1a1fb73be5d6e5e3ee96b1afebae5e445a9f4ec0da6a1ad60216702 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-e, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-08-20T01:30:21.736 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.e.service' 2024-08-20T01:30:21.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:21.771 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-08-20T01:30:21.771 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-08-20T01:30:21.771 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.h 2024-08-20T01:30:21.993 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:21 smithi142 systemd[1]: Stopping Ceph mds.h for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:22.493 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: 2024-08-20T01:30:22.098+0000 7ff4a2569640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:22.493 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:22 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h[52680]: 2024-08-20T01:30:22.098+0000 7ff4a2569640 -1 mds.h *** got signal Terminated *** 2024-08-20T01:30:25.358 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:25 smithi142 podman[60751]: 2024-08-20 01:30:25.049184334 +0000 UTC m=+3.063111180 container died 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-08-20T01:30:25.666 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:25 smithi142 podman[60751]: 2024-08-20 01:30:25.439643642 +0000 UTC m=+3.453570487 container cleanup 32c560fc5f410cd325c14413d7461ce2791849de74e38f27c32c68c4af63ad97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0) 2024-08-20T01:30:25.666 INFO:journalctl@ceph.mds.h.smithi142.stdout:Aug 20 01:30:25 smithi142 bash[60751]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-h 2024-08-20T01:30:25.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.h.service' 2024-08-20T01:30:25.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:25.817 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-08-20T01:30:25.817 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-08-20T01:30:25.817 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.k 2024-08-20T01:30:25.993 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:25 smithi142 systemd[1]: Stopping Ceph mds.k for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:26.493 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k[52063]: 2024-08-20T01:30:26.134+0000 7f225f72b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:26.493 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:26 smithi142 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k[52063]: 2024-08-20T01:30:26.134+0000 7f225f72b640 -1 mds.k *** got signal Terminated *** 2024-08-20T01:30:30.410 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:30 smithi142 podman[60872]: 2024-08-20 01:30:30.148623332 +0000 UTC m=+4.116089513 container died ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, ceph=True) 2024-08-20T01:30:30.686 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:30 smithi142 podman[60872]: 2024-08-20 01:30:30.48453285 +0000 UTC m=+4.451999024 container cleanup ce44b9a59e15682d67e8aec34380fc6945dbce89794cacedb1de36e924dd9498 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-08-20T01:30:30.686 INFO:journalctl@ceph.mds.k.smithi142.stdout:Aug 20 01:30:30 smithi142 bash[60872]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-k 2024-08-20T01:30:30.903 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.k.service' 2024-08-20T01:30:30.935 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:30.936 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-08-20T01:30:30.936 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-08-20T01:30:30.936 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.c 2024-08-20T01:30:31.255 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:30 smithi167 systemd[1]: Stopping Ceph mds.c for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:31.715 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: 2024-08-20T01:30:31.253+0000 7fd9d1876640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:31.715 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:31 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c[53264]: 2024-08-20T01:30:31.253+0000 7fd9d1876640 -1 mds.c *** got signal Terminated *** 2024-08-20T01:30:33.331 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:33 smithi167 podman[61649]: 2024-08-20 01:30:33.046666107 +0000 UTC m=+1.908996196 container died bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-08-20T01:30:33.672 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:33 smithi167 podman[61649]: 2024-08-20 01:30:33.437579714 +0000 UTC m=+2.299909800 container cleanup bfc525c668aa9f5e7a4119d5341d02c3e8e80fd9c3ade955517e71ed88f37de4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-08-20T01:30:33.672 INFO:journalctl@ceph.mds.c.smithi167.stdout:Aug 20 01:30:33 smithi167 bash[61649]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-c 2024-08-20T01:30:33.825 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.c.service' 2024-08-20T01:30:33.861 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:33.861 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-08-20T01:30:33.861 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-08-20T01:30:33.862 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.f 2024-08-20T01:30:34.208 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:33 smithi167 systemd[1]: Stopping Ceph mds.f for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:34.465 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f[52966]: 2024-08-20T01:30:34.206+0000 7f082591f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:34.465 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:34 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f[52966]: 2024-08-20T01:30:34.206+0000 7f082591f640 -1 mds.f *** got signal Terminated *** 2024-08-20T01:30:36.367 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:36 smithi167 podman[61771]: 2024-08-20 01:30:36.031307982 +0000 UTC m=+1.952264207 container died faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-08-20T01:30:36.667 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:36 smithi167 podman[61771]: 2024-08-20 01:30:36.449393636 +0000 UTC m=+2.370349900 container cleanup faf0ca14437552e55336d583a9b2342d11302012877873184227f155d1f871c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, io.buildah.version=1.36.0) 2024-08-20T01:30:36.667 INFO:journalctl@ceph.mds.f.smithi167.stdout:Aug 20 01:30:36 smithi167 bash[61771]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-f 2024-08-20T01:30:36.883 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.f.service' 2024-08-20T01:30:36.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:36.917 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-08-20T01:30:36.918 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-08-20T01:30:36.918 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.i 2024-08-20T01:30:37.237 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:36 smithi167 systemd[1]: Stopping Ceph mds.i for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:37.715 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: 2024-08-20T01:30:37.235+0000 7f497d6b9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:37.715 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:37 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i[53565]: 2024-08-20T01:30:37.235+0000 7f497d6b9640 -1 mds.i *** got signal Terminated *** 2024-08-20T01:30:41.429 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:41 smithi167 podman[61894]: 2024-08-20 01:30:41.049087386 +0000 UTC m=+3.919213718 container died b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754) 2024-08-20T01:30:41.715 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:41 smithi167 podman[61894]: 2024-08-20 01:30:41.51134485 +0000 UTC m=+4.381471179 container cleanup b07db5da6ce83046611c39686589d796c5a6f9b43f32720e4b678749ee783d7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-08-20T01:30:41.715 INFO:journalctl@ceph.mds.i.smithi167.stdout:Aug 20 01:30:41 smithi167 bash[61894]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-i 2024-08-20T01:30:41.997 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.i.service' 2024-08-20T01:30:42.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:42.030 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-08-20T01:30:42.031 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-08-20T01:30:42.031 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.l 2024-08-20T01:30:42.365 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:42 smithi167 systemd[1]: Stopping Ceph mds.l for d876cd00-5e8e-11ef-bccf-c7b262605968... 2024-08-20T01:30:42.715 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l[52662]: 2024-08-20T01:30:42.364+0000 7f2af76ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-08-20T01:30:42.715 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:42 smithi167 ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l[52662]: 2024-08-20T01:30:42.364+0000 7f2af76ea640 -1 mds.l *** got signal Terminated *** 2024-08-20T01:30:43.949 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:43 smithi167 podman[62014]: 2024-08-20 01:30:43.696616868 +0000 UTC m=+1.444638933 container died 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-08-20T01:30:44.215 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:44 smithi167 podman[62014]: 2024-08-20 01:30:44.040323804 +0000 UTC m=+1.788345872 container cleanup 21b5e2d66fae16f334509586ad20d4395fec2112278478ce619510d2697fb282 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9fef3676e228a6a987e17bf85cedfa9ed4ad03e64c3b44ff5ea1f6d196d5afb8, name=ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-08-19-1357-ef42754, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS) 2024-08-20T01:30:44.215 INFO:journalctl@ceph.mds.l.smithi167.stdout:Aug 20 01:30:44 smithi167 bash[62014]: ceph-d876cd00-5e8e-11ef-bccf-c7b262605968-mds-l 2024-08-20T01:30:44.448 DEBUG:teuthology.orchestra.run.smithi167:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d876cd00-5e8e-11ef-bccf-c7b262605968@mds.l.service' 2024-08-20T01:30:44.479 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-08-20T01:30:44.480 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-08-20T01:30:44.480 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T01:30:44.752 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:30:48.650 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T01:30:48.851 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:30:51.345 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force --keep-logs 2024-08-20T01:30:51.548 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:30:53.922 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:30:53.985 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:30:54.018 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-08-20T01:30:54.045 INFO:tasks.cephadm:Archiving crash dumps... 2024-08-20T01:30:54.047 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi138/crash 2024-08-20T01:30:54.048 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash -- . 2024-08-20T01:30:54.113 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T01:30:54.114 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-08-20T01:30:54.117 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi142/crash 2024-08-20T01:30:54.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash -- . 2024-08-20T01:30:54.158 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T01:30:54.158 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-08-20T01:30:54.160 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi167/crash 2024-08-20T01:30:54.161 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash -- . 2024-08-20T01:30:54.196 INFO:teuthology.orchestra.run.smithi167.stderr:tar: /var/lib/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/crash: Cannot open: No such file or directory 2024-08-20T01:30:54.197 INFO:teuthology.orchestra.run.smithi167.stderr:tar: Error is not recoverable: exiting now 2024-08-20T01:30:54.198 INFO:tasks.cephadm:Checking cluster log for badness... 2024-08-20T01:30:54.198 DEBUG:teuthology.orchestra.run.smithi138:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-08-20T01:30:54.275 INFO:tasks.cephadm:Compressing logs... 2024-08-20T01:30:54.276 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T01:30:54.280 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T01:30:54.282 DEBUG:teuthology.orchestra.run.smithi167:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-08-20T01:30:54.307 INFO:teuthology.orchestra.run.smithi167.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T01:30:54.307 INFO:teuthology.orchestra.run.smithi167.stderr:: No such file or directory 2024-08-20T01:30:54.308 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T01:30:54.308 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.2.log 2024-08-20T01:30:54.308 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.8.log 2024-08-20T01:30:54.308 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.z.log 2024-08-20T01:30:54.313 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.5.log 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.c.log 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.z.log: /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.f.log 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.11.log 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.i.log 2024-08-20T01:30:54.314 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.11.log: 90.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T01:30:54.315 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T01:30:54.317 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.i.log: 93.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.z.log.gz 2024-08-20T01:30:54.319 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T01:30:54.319 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log: 82.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T01:30:54.319 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.l.log 2024-08-20T01:30:54.319 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log: 94.9% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.f.log.gz 2024-08-20T01:30:54.320 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-08-20T01:30:54.322 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.c.log 2024-08-20T01:30:54.323 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-08-20T01:30:54.324 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.4.log 2024-08-20T01:30:54.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T01:30:54.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.1.log 2024-08-20T01:30:54.325 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.b.log 2024-08-20T01:30:54.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.h.log 2024-08-20T01:30:54.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.10.log 2024-08-20T01:30:54.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.k.log 2024-08-20T01:30:54.329 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.y.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.e.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.7.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.b.log 2024-08-20T01:30:54.330 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log 2024-08-20T01:30:54.333 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.l.log: 95.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.l.log.gz 2024-08-20T01:30:54.333 INFO:teuthology.orchestra.run.smithi167.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log 2024-08-20T01:30:54.333 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.c.log: 91.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T01:30:54.334 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-08-20T01:30:54.334 INFO:teuthology.orchestra.run.smithi138.stderr: /var/log/ceph/cephadm.log 2024-08-20T01:30:54.334 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-08-20T01:30:54.335 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log 2024-08-20T01:30:54.336 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.admin.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.6.log 2024-08-20T01:30:54.336 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.g.log 2024-08-20T01:30:54.337 INFO:teuthology.orchestra.run.smithi167.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log: 86.9% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T01:30:54.337 INFO:teuthology.orchestra.run.smithi138.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-08-20T01:30:54.337 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.d.log 2024-08-20T01:30:54.338 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.0.log 2024-08-20T01:30:54.339 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.x.log 2024-08-20T01:30:54.340 INFO:teuthology.orchestra.run.smithi138.stderr: 90.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T01:30:54.340 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.0.log: 93.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T01:30:54.343 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.3.log 2024-08-20T01:30:54.344 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.a.log 2024-08-20T01:30:54.347 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.3.log: /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.a.log 2024-08-20T01:30:54.350 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.a.log: 95.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.d.log.gz 2024-08-20T01:30:54.351 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.j.log 2024-08-20T01:30:54.354 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.9.log 2024-08-20T01:30:54.357 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.j.log: /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log 2024-08-20T01:30:54.361 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log 2024-08-20T01:30:54.363 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log: 84.4%gzip -5 --verbose -- /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log 2024-08-20T01:30:54.363 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T01:30:54.369 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log: /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log: 95.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.j.log.gz 2024-08-20T01:30:54.376 INFO:teuthology.orchestra.run.smithi138.stderr: 86.9% 95.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.a.log.gz 2024-08-20T01:30:54.378 INFO:teuthology.orchestra.run.smithi138.stderr: -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T01:30:54.383 INFO:teuthology.orchestra.run.smithi138.stderr: 90.8% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T01:30:54.414 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.b.log: /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-08-20T01:30:54.525 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.y.log.gz 2024-08-20T01:30:54.574 INFO:teuthology.orchestra.run.smithi142.stderr: 91.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-volume.log.gz 2024-08-20T01:30:54.859 INFO:teuthology.orchestra.run.smithi138.stderr: 90.4% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mgr.x.log.gz 2024-08-20T01:30:54.963 INFO:teuthology.orchestra.run.smithi167.stderr: 91.5% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.c.log.gz 2024-08-20T01:30:55.223 INFO:teuthology.orchestra.run.smithi142.stderr: 94.9% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.k.log.gz 2024-08-20T01:30:55.319 INFO:teuthology.orchestra.run.smithi142.stderr: 82.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.cephadm.log.gz 2024-08-20T01:30:55.321 INFO:teuthology.orchestra.run.smithi142.stderr: 90.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.audit.log.gz 2024-08-20T01:30:55.937 INFO:teuthology.orchestra.run.smithi142.stderr: 86.9% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph.log.gz 2024-08-20T01:30:55.975 INFO:teuthology.orchestra.run.smithi138.stderr: 90.4% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.a.log.gz 2024-08-20T01:30:59.412 INFO:teuthology.orchestra.run.smithi167.stderr: 94.9% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.i.log.gz 2024-08-20T01:30:59.505 INFO:teuthology.orchestra.run.smithi167.stderr: 95.3% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.c.log.gz 2024-08-20T01:31:18.506 INFO:teuthology.orchestra.run.smithi167.stderr: 91.6% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.11.log.gz 2024-08-20T01:31:19.787 INFO:teuthology.orchestra.run.smithi138.stderr: 91.6% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.0.log.gz 2024-08-20T01:31:20.252 INFO:teuthology.orchestra.run.smithi142.stderr: 92.7% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mon.b.log.gz 2024-08-20T01:31:22.000 INFO:teuthology.orchestra.run.smithi167.stderr: 91.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.8.log.gz 2024-08-20T01:31:27.070 INFO:teuthology.orchestra.run.smithi138.stderr: 91.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.6.log.gz 2024-08-20T01:31:27.389 INFO:teuthology.orchestra.run.smithi138.stderr: 91.4% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.3.log.gz 2024-08-20T01:31:27.928 INFO:teuthology.orchestra.run.smithi167.stderr: 91.3% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.2.log.gz 2024-08-20T01:31:31.591 INFO:teuthology.orchestra.run.smithi167.stderr: 91.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.5.log.gz 2024-08-20T01:31:31.593 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-08-20T01:31:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:real 0m37.296s 2024-08-20T01:31:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:user 2m9.349s 2024-08-20T01:31:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:sys 0m4.192s 2024-08-20T01:31:39.638 INFO:teuthology.orchestra.run.smithi138.stderr: 91.3% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.9.log.gz 2024-08-20T01:31:58.979 INFO:teuthology.orchestra.run.smithi142.stderr: 91.6% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.10.log.gz 2024-08-20T01:32:14.570 INFO:teuthology.orchestra.run.smithi142.stderr: 95.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.e.log.gz 2024-08-20T01:32:19.059 INFO:teuthology.orchestra.run.smithi138.stderr: 93.2% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.g.log.gz 2024-08-20T01:32:19.062 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-08-20T01:32:19.062 INFO:teuthology.orchestra.run.smithi138.stderr:real 1m24.754s 2024-08-20T01:32:19.062 INFO:teuthology.orchestra.run.smithi138.stderr:user 3m27.278s 2024-08-20T01:32:19.062 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m15.858s 2024-08-20T01:32:40.513 INFO:teuthology.orchestra.run.smithi142.stderr: 91.5% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.7.log.gz 2024-08-20T01:33:14.963 INFO:teuthology.orchestra.run.smithi142.stderr: 91.7% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.1.log.gz 2024-08-20T01:33:58.939 INFO:teuthology.orchestra.run.smithi142.stderr: 91.4% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-osd.4.log.gz 2024-08-20T01:35:03.880 INFO:teuthology.orchestra.run.smithi142.stderr: 93.0% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.b.log.gz 2024-08-20T01:36:02.391 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% -- replaced with /var/log/ceph/d876cd00-5e8e-11ef-bccf-c7b262605968/ceph-mds.h.log.gz 2024-08-20T01:36:02.413 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-08-20T01:36:02.413 INFO:teuthology.orchestra.run.smithi142.stderr:real 5m8.095s 2024-08-20T01:36:02.413 INFO:teuthology.orchestra.run.smithi142.stderr:user 4m56.757s 2024-08-20T01:36:02.413 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m11.122s 2024-08-20T01:36:02.414 INFO:tasks.cephadm:Archiving logs... 2024-08-20T01:36:02.414 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi138/log 2024-08-20T01:36:02.445 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T01:36:32.817 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi142/log 2024-08-20T01:36:32.819 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T01:37:23.603 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi167/log 2024-08-20T01:37:23.604 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-08-20T01:37:46.916 INFO:tasks.cephadm:Removing cluster... 2024-08-20T01:37:46.917 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force 2024-08-20T01:37:47.193 INFO:teuthology.orchestra.run.smithi138.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:37:48.391 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force 2024-08-20T01:37:48.597 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:37:49.218 DEBUG:teuthology.orchestra.run.smithi167:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d876cd00-5e8e-11ef-bccf-c7b262605968 --force 2024-08-20T01:37:49.436 INFO:teuthology.orchestra.run.smithi167.stdout:Deleting cluster with fsid: d876cd00-5e8e-11ef-bccf-c7b262605968 2024-08-20T01:37:49.880 INFO:tasks.cephadm:Removing cephadm ... 2024-08-20T01:37:49.880 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T01:37:49.920 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T01:37:49.940 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf /home/ubuntu/cephtest/cephadm 2024-08-20T01:37:49.957 INFO:tasks.cephadm:Teardown complete 2024-08-20T01:37:49.957 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-08-20T01:37:50.738 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-08-20T01:37:50.738 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T01:37:50.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T01:37:50.746 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-08-20T01:37:50.899 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-08-20T01:37:51.236 INFO:teuthology.task.clock:Checking final clock skew... 2024-08-20T01:37:51.236 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T01:37:51.243 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T01:37:51.248 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-08-20T01:37:51.263 INFO:teuthology.orchestra.run.smithi142.stderr:bash: line 1: ntpq: command not found 2024-08-20T01:37:51.270 INFO:teuthology.orchestra.run.smithi167.stderr:bash: line 1: ntpq: command not found 2024-08-20T01:37:51.275 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T01:37:51.275 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2024-08-20T01:37:51.275 INFO:teuthology.orchestra.run.smithi167.stdout:^* hv01.front.sepia.ceph.com 3 6 377 36 -209us[ -175us] +/- 50ms 2024-08-20T01:37:51.276 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 34 +197us[ +197us] +/- 61ms 2024-08-20T01:37:51.276 INFO:teuthology.orchestra.run.smithi167.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +124us[ +124us] +/- 74ms 2024-08-20T01:37:51.276 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T01:37:51.276 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-08-20T01:37:51.280 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T01:37:51.280 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-08-20T01:37:51.280 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv01.front.sepia.ceph.com 3 6 377 37 -236us[ -255us] +/- 50ms 2024-08-20T01:37:51.280 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +170us[ +151us] +/- 61ms 2024-08-20T01:37:51.281 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 +102us[ +102us] +/- 74ms 2024-08-20T01:37:51.281 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -172us[ -136us] +/- 50ms 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +227us[ +227us] +/- 61ms 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 +165us[ +165us] +/- 74ms 2024-08-20T01:37:51.283 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-08-20T01:37:51.284 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-08-20T01:37:51.694 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-08-20T01:37:51.699 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-08-20T01:37:51.844 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T01:37:51.972 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T01:37:52.060 DEBUG:teuthology.orchestra.run.smithi167:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-08-20T01:37:52.125 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T01:37:52.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:52.187 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T01:37:52.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:52.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-08-20T01:37:52.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:52.250 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-08-20T01:37:52.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-08-20T01:37:52.355 INFO:teuthology.task.internal:Duration was 2826.745879 seconds 2024-08-20T01:37:52.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-08-20T01:37:52.392 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-08-20T01:37:52.392 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T01:37:52.397 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T01:37:52.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-08-20T01:37:52.445 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T01:37:52.467 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T01:37:52.480 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-08-20T01:37:52.858 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-08-20T01:37:52.859 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-08-20T01:37:52.859 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T01:37:52.908 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-08-20T01:37:52.908 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T01:37:52.946 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-08-20T01:37:52.947 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-08-20T01:37:52.979 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-08-20T01:37:52.979 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T01:37:52.983 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T01:37:52.988 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-08-20T01:37:53.051 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-08-20T01:37:53.051 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T01:37:53.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T01:37:53.058 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-08-20T01:37:54.005 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-08-20T01:37:54.017 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-08-20T01:37:54.018 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T01:37:54.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T01:37:54.116 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-08-20T01:37:54.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-08-20T01:37:54.158 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.210 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-08-20T01:37:54.212 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-08-20T01:37:54.228 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-08-20T01:37:54.258 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:54.291 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:54.307 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-08-20T01:37:54.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-08-20T01:37:54.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-08-20T01:37:54.335 INFO:teuthology.task.internal:Transferring archived files... 2024-08-20T01:37:54.336 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi138 2024-08-20T01:37:54.337 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T01:37:54.425 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi142 2024-08-20T01:37:54.425 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T01:37:54.462 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-08-19_19:57:23-fs-wip-hemanth-testing-2024-08-19-1357-distro-default-smithi/7861298/remote/smithi167 2024-08-20T01:37:54.463 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-08-20T01:37:54.505 INFO:teuthology.task.internal:Removing archive directory... 2024-08-20T01:37:54.506 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T01:37:54.510 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T01:37:54.514 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-08-20T01:37:54.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-08-20T01:37:54.579 INFO:teuthology.task.internal:Not uploading archives. 2024-08-20T01:37:54.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-08-20T01:37:54.616 INFO:teuthology.task.internal:Tidying up after the test... 2024-08-20T01:37:54.617 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T01:37:54.622 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T01:37:54.626 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-08-20T01:37:54.639 INFO:teuthology.orchestra.run.smithi142.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 01:37 /home/ubuntu/cephtest 2024-08-20T01:37:54.642 INFO:teuthology.orchestra.run.smithi167.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 01:37 /home/ubuntu/cephtest 2024-08-20T01:37:54.652 INFO:teuthology.orchestra.run.smithi138.stdout: 393246 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 20 01:37 /home/ubuntu/cephtest 2024-08-20T01:37:54.654 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-08-20T01:37:54.666 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-08-20T01:37:54.715 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 7e35574567e19d2ca345caad671b98560d512bea description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2826.745879173279 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi138 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ef4275410d8483698016fe484b2a3c664080b4c1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_hyelloji@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=595f38d07202411191bcc1cec4ad1ea9 status: fail success: false 2024-08-20T01:37:54.715 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-08-20T01:37:54.804 INFO:teuthology.run:FAIL